OpenAI published its first results for Jalapeño, its first custom inference chip, tested against Nvidia's GB200 and GB300 systems on InferenceX, a public, open-source benchmark from SemiAnalysis. That last detail is worth crediting before anything else: InferenceX is independently maintained, its recipes are committed to a public repo, and its runs are reproducible by outside parties rather than a metric OpenAI invented for this announcement — a real methodological upgrade over a purely internal benchmark. What OpenAI reports on top of that real infrastructure is where this post needs to slow down, because two very different kinds of numbers are sitting in the same charts, and they don't mean the same thing.
The number that's actually the headline: 1.5–1.9x more work per watt
Across three model families — GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, none of which OpenAI controls — Jalapeño delivered 1.5 to 1.9x more mixed-token throughput per kilowatt at each system's own best peak operating point, alongside 1.7–3.6x lower end-to-end latency and 2.7–4.1x higher peak per-user decode speed. That's a real, defensible efficiency gain, tested on hardware and model families outside OpenAI's control, on a benchmark outside parties can rerun. Worth naming directly as the actual news: a first-generation custom chip beating Nvidia's current top systems on efficiency at their own best settings is a genuine result, not a framing trick.
One caveat OpenAI discloses itself and is worth repeating rather than skipping: Jalapeño is rated at 700 watts, with measured sustained draw at or below 550W in these tests, against Nvidia's GB200 at 1,200W and GB300 at 1,400W — roughly half the power envelope of the systems it's being compared to. A chip rated for meaningfully less power posting better performance-per-watt numbers isn't surprising on its own; it's the whole point of designing custom silicon for a narrower job than a general-purpose GPU. Still worth stating plainly rather than letting "per watt" hide the absolute power gap driving part of the multiple.
The much bigger numbers, and what they're actually measuring
The charts OpenAI leads with show far larger multiples — 53.7x on GPT-OSS, 104.3x on DeepSeek R1, 56.1x on Kimi K2.5 — labeled "more throughput at previous TBT" (time-between-tokens). These aren't a second measurement of the same peak-efficiency claim; they measure something narrower: how much more efficient Jalapeño is when the Nvidia system is forced to run at Jalapeño's fastest per-user decode speed — a speed regime GPUs are known to become dramatically less efficient in, since squeezing out the lowest possible latency per user sacrifices the batching that makes GPU serving efficient in the first place. Comparing two systems at one system's most favorable point, after establishing the other system's efficiency craters away from its own comfortable operating range, is a real result about where Jalapeño's Pareto frontier sits — but it is not the same claim as "104x more efficient," full stop, and OpenAI's own appendix data makes the distinction available even though the headline framing doesn't lead with it. Worth being as precise about this as this blog has been about harness benchmarks claiming saturation under one specific, favorable framing before: the number is real, the generalization from it is doing more work than the number itself supports.
The chip was built with AI, and built to be programmed by AI
Two details in OpenAI's own writeup are worth taking at face value as genuinely interesting, separate from the benchmark numbers. First: Jalapeño went from initial design to tapeout in nine months, with earlier OpenAI models helping design and bring up the chip and current models accelerating optimization — a concrete, dated claim about AI-assisted hardware design timelines. Second, and more specific: using Codex with GPT-Astra, OpenAI says it brought three open-weight models that weren't part of Jalapeño's original production plan to full performance within two months, and for selected GPT-OSS attention and mixture-of-experts blocks, AI-generated kernel implementations ran 1.5–1.8x faster than existing human-expert-written ones. OpenAI's own caveat on that last figure deserves to be kept attached to it rather than dropped in retelling: "those figures apply to the selected blocks, not the full model." That's the same shape of result this blog covered when NVIDIA's AVO evolved attention kernels that beat cuDNN and FlashAttention-4 — AI-assisted low-level kernel optimization is showing up as a real, recurring pattern across labs this year, not a one-off claim from either company.
What's asserted but not shown
OpenAI states that "in our internal testing, Jalapeño's advantage widened further on frontier OpenAI models" — a claim with no published numbers behind it here, resting entirely on models and workloads nobody outside OpenAI can check. Worth treating as a direction rather than a data point until (or unless) OpenAI publishes something more specific about its own frontier models the way it did for the three public ones. Deployment is also still ahead of it, not behind it: OpenAI describes Jalapeño as entering "production qualification," with a plan to begin deploying within its own infrastructure by the end of the year, and says it will keep buying Nvidia and other partners' accelerators alongside it rather than replacing them. This is a first-generation chip's first published results, not a shipped, load-bearing part of OpenAI's serving fleet yet.
What to expect next
- Watch whether these specific runs land on InferenceX's public dashboard. InferenceX's own model is reproducible, auditable community runs; whether this particular Jalapeño-vs-GB200/GB300 comparison becomes independently rerunnable there, rather than living only in OpenAI's own blog post, is the real test of how much the "public benchmark" framing is worth here.
- Watch for the frontier-model numbers OpenAI didn't publish. The claim that Jalapeño's advantage "widens further" on OpenAI's own models is exactly the kind of specific, checkable claim this post would want actual figures for.
- Watch Jalapeño's actual deployment, not just its benchmark results. A chip in production qualification is not yet a chip serving real traffic at scale — the gap between "tested well" and "running OpenAI's fleet" is where a lot of custom-silicon efforts have historically slowed down.
References: OpenAI — Jalapeño: first results · OpenAI on X — announcement · InferenceX by SemiAnalysis · related coverage: NVIDIA's AVO Hits 100% on ARC-AGI-3 Too · Two Harnesses Solved the Same Benchmark by Disagreeing About Memory · Frontier Arcade: trends & predictions