By Thorsten Meyer
OpenAI just published first measured results for Jalapeño, its own custom inference chip, and the numbers are genuinely strong. They’re also, as always with first-party silicon, the vendor’s own — measured by OpenAI, framed by OpenAI, on the metric OpenAI chose. Before I walk through the performance and the architecture in the detail they deserve, I want to correct one thing up front, because it shapes how you should read everything else: the measured comparisons in this post are against NVIDIA, and only NVIDIA. There is no head-to-head here against AMD, Google, or Microsoft. So if you came for “Jalapeño beats everyone,” the honest answer is that OpenAI didn’t test everyone — it tested itself against NVIDIA’s Blackwell generation, on a power-efficiency metric, with a chip it hasn’t deployed yet. That’s still a real story. It’s just a narrower and more interesting one than the headline.
Let me give you the performance, the architecture, and a fair placement against the whole field.
The performance, with the caveats attached
OpenAI tested Jalapeño on InferenceX, a public benchmark from SemiAnalysis that measures the full path of serving an AI request, across three open models it doesn’t own: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Using external models matters — it’s evidence the architecture isn’t only tuned for OpenAI’s own networks. Across all three, the headline ranges are: 1.5 to 1.9 times more AI work per watt at peak throughput, 1.7 to 3.6 times lower end-to-end latency, and 2.1 to 4.1 times higher performance on highly interactive workloads. Those are the numbers to anchor on.
OpenAI’s first custom inference chip posts real per-watt wins on a public benchmark — measured by OpenAI, on the metric OpenAI chose, against NVIDIA only, on a chip not yet deployed.
Normalized by published TDP: Jalapeño 700W (measured ≤550W) vs GB200 1,200W / GB300 1,400W. Peak throughput per kW — higher is better.
The specifics, by model, all against NVIDIA systems: on GPT-OSS 120B against a GB200, roughly 1.9x the peak throughput-per-watt and 1.7x lower latency; on DeepSeek R1 against a GB300, about 1.7x per-watt and 3.6x lower latency; on Kimi K2.5, the largest tested, roughly 1.5x per-watt and 3.4x lower latency. You'll also see eye-popping multiples like 53x or 104x "more throughput at previous-best time-between-tokens" — ignore those as headline material. They're real but they describe one narrow operating point (matching a specific latency target the older system could barely hit), not general performance. The honest summary is the 1.5–1.9x efficiency and 1.7–3.6x latency ranges, not the triple-digit cherry.
Now the three caveats that determine what these numbers actually mean.
First, the metric is performance per watt, and that's a choice. OpenAI says outright it considers per-watt the more useful standard than per-chip, and normalizes using each accelerator's published power rating — Jalapeño at 700W against a GB200 at 1,200W and a GB300 at 1,400W. Per-watt is a defensible and arguably the right metric for a datacenter operator paying power bills. But it's not a neutral one: it structurally favors the lower-power part, and a per-chip or per-dollar comparison would read differently. To OpenAI's credit, they note Jalapeño's measured sustained power stayed at or below 550W while they normalized against the higher 700W rating — so on that axis they're being conservative about their own chip. Still: know that "per watt" is doing work in every one of these figures.
Second, this is a dedicated inference ASIC versus general-purpose GPUs. NVIDIA's Blackwell parts train and infer; Jalapeño does one job. A purpose-built inference chip beating a general-purpose GPU on inference-per-watt is close to the definition of why you'd build an ASIC in the first place. It's a real advantage for OpenAI's serving costs — but it's not an apples-to-apples verdict on NVIDIA's silicon, because half of what NVIDIA's chip is for isn't in the test.
Third, it's vendor-reported and not yet deployed. These are OpenAI's own measurements, not independent benchmarking, and Jalapeño won't begin deploying inside OpenAI's own infrastructure until the end of the year, with production qualification still ongoing. First-party silicon results always land favorably; that's not an accusation, it's a base rate. Treat this as a strong, specific claim awaiting independent confirmation, not a settled result.
Distributed AI Systems: A practical guide to building scalable training, inference, and serving systems for production AI
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The architecture, which is the genuinely interesting part
Strip away the competitive framing and the design is where Jalapeño earns real attention, because it's a clean example of building hardware around a workload rather than adapting a workload to hardware.
The genuinely interesting part isn't the benchmark — it's the design. LM inference is two phases with different bottlenecks, and Jalapeño is built to stop the waiting between them.
The core insight is that language-model inference isn't one thing — it's phases with different bottlenecks. Prefill, where the model digests your prompt, is compute-bound. Decode, where it generates the answer token by token, is bound more by memory bandwidth. And moving data between cores and chips adds latency that can leave expensive compute sitting idle, waiting. A chip that's brilliant at one phase can squander that edge waiting on another. OpenAI designed Jalapeño to attack the waiting: minimize data movement, and let model state — crucially the KV cache used during generation — be explicitly placed and kept local, so the system can activate the right mix of compute, memory, and networking for whichever phase it's in.
The network is treated as part of the architecture, not an afterthought bolted on around it. A large network domain lets an entire workload stay inside one connected system, so a request stays fast from first token to last without shuttling state across boundaries. The stated result is a "balanced and fungible" accelerator — good at both prefill and decode, and able to adapt as the ratio between them shifts. That last property is the one aimed squarely at agents: agentic workloads swing unpredictably between long prompt-processing and long generation, and a chip that rebalances between the two without losing efficiency is genuinely well-matched to where the work is going. This is inference-first silicon designed for the agentic era, and the design reasoning holds up.
As an affiliate, we earn on qualifying purchases.
The part that's a real signal: AI designed and programmed the chip
Here's the detail I'd underline for anyone thinking about where this is all heading. OpenAI says its own models were in the loop building Jalapeño — helping take it from initial design to tapeout in nine months, which is fast for custom silicon, and optimizing the chip's arithmetic circuits to fit more compute on schedule. They deliberately designed the chip to be a clean, predictable target that an AI can program: describe the work as local tensors with explicit communication, and let a model handle the notoriously hard problem of mapping and scheduling it across the system.
The concrete evidence they offer: using Codex with a GPT-Astra model, the team brought three open-weight models that weren't in the original plan up to high performance in two months, and for selected attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than the human-expert-written versions. Read that carefully — it's selected blocks, not the whole model, and OpenAI says so plainly. But the direction is the point: AI compressing the design and the programming of the hardware it runs on is a recursive loop worth watching soberly, without inflating it into something it isn't yet.
As an affiliate, we earn on qualifying purchases.
Now, honestly, against the whole field
You asked how Jalapeño stacks up against NVIDIA, AMD, Google, and Microsoft. The honest answer requires separating what OpenAI showed from what's true about the landscape.
Against NVIDIA, OpenAI showed measured per-watt inference wins against Blackwell — but framed the whole thing carefully, and in the same post committed to keep deploying NVIDIA "widely" for both training and inference. That's the tell. This isn't OpenAI leaving NVIDIA; it's OpenAI adding first-party inference silicon to reduce its dependence and improve its serving economics, while remaining a massive NVIDIA customer. "Ahead of NVIDIA" overstates a relationship OpenAI itself describes as ongoing.
Against Google, the framing inverts entirely. Google has been running custom AI silicon — its TPUs — in production for years, serving Gemini and Google Cloud at scale. If anyone has proven that vertical model-plus-silicon integration works, it's Google, and OpenAI is now following that playbook, not leading it. Jalapeño isn't even deployed yet. On the specific axis of "has shipped custom AI silicon in production," OpenAI is behind Google, not ahead.
Against AMD and Microsoft, there's simply no comparison in this post to draw a verdict from. AMD's Instinct line is the main merchant challenger to NVIDIA on inference, and OpenAI has its own supply deal with AMD. Microsoft — an OpenAI partner and investor — is building its own Maia accelerators for exactly the same vertical-integration reasons. Add Amazon's Trainium and Meta's MTIA and the real picture comes into focus: custom AI silicon is now table stakes for anyone operating at hyperscale, and Jalapeño is OpenAI joining that club, not vaulting past it. The club already had several members shipping in production.
So the accurate framing isn't "OpenAI's chip beats everyone." It's "OpenAI now makes its own competitive inference chip, with strong first-party per-watt numbers against NVIDIA, joining a vertical-integration race that Google and others have been running for years." That's a significant milestone. It's just not a coronation.
As an affiliate, we earn on qualifying purchases.
The part that's mine to make
Read through my usual lens, the important story here isn't the benchmark — it's the strategy, and it's the same one I keep tracing across everything this month. The battleground is cost-to-serve. Grok held price flat and called it a frontier launch; GLM and Qwen shipped efficient open models to win on price-per-token; Stripe bought the meter that prices the tokens. Jalapeño is OpenAI attacking the same variable from the hardware floor: more useful work per watt means lower cost per result, which means operating leverage — revenue growing faster than the cost to serve. Vertical integration down to the silicon is how you win a price war you're also helping to start.
And there's a sovereignty dimension I can't ignore, because it cuts against the trend I usually champion. Jalapeño is the opposite of open. It's maximally vertically integrated, proprietary, un-buyable, un-ownable — you will never run your agents on one unless you're renting OpenAI's stack. It points at one of the two futures the industry is splitting toward: on one side, closed labs integrating all the way down to custom chips, raising the moat and the capital bar; on the other, the open-weight efficiency wave running on merchant and commodity hardware that anyone can own. Jalapeño is a strong data point for the first future. The open models I've been covering are the counterweight. For most builders, Jalapeño changes nothing you can touch — you can't buy it — but it changes OpenAI's cost structure, and that pressure reaches everyone through the same price war.
Where I land
Jalapeño is real, working first-party silicon with genuinely strong first-party results: 1.5–1.9x more inference work per watt and meaningfully lower latency than NVIDIA's Blackwell on a public benchmark, an inference-first architecture that's thoughtfully matched to agentic workloads, and a credible early demonstration of AI helping design and program the hardware it runs on. Credit where it's due, and it's due.
But the honest scorecard reads: vendor-reported, per-watt, versus NVIDIA only, not yet deployed, and following a vertical-integration path Google has been on for years. "Ahead of NVIDIA, AMD, Google, and Microsoft" is a headline the post doesn't actually support — OpenAI beat NVIDIA Blackwell on a chosen metric and stayed silent on the other three, while itself pledging to keep buying NVIDIA. The truer and more useful read is that OpenAI has joined the custom-silicon club with a strong first entry, and turned its own cost-to-serve into a weapon in the price war that's defining this whole era. That's worth taking seriously. It just isn't the same as winning the race, and the difference is exactly the kind of thing worth being precise about.
Analysis and opinion from a builder, founder, and post-labor economist running a local-first inference operation. All performance figures (1.5–1.9x perf/watt, 1.7–3.6x latency, 2.1–4.1x interactive; per-model results vs NVIDIA GB200/GB300; 9-month tapeout; AI-generated kernels 1.5–1.8x faster on selected blocks) are OpenAI's own reported results on the SemiAnalysis InferenceX benchmark, power-normalized by published TDP, verified at time of writing against OpenAI's Jalapeño post and companion "full stack" post (25 August 2026); they are first-party vendor results pending independent verification, and the chip is not yet deployed. Competitive characterizations of AMD, Google, Microsoft, Amazon, and Meta silicon are the author's analysis of the public landscape, not head-to-head benchmarks published by OpenAI. This is analysis, not investment advice. Point-in-time as of 25 August 2026.
