Skip to main content

OpenAI Benchmarks Jalapeño Against Nvidia's Racks

OpenAI published first results for its Broadcom-built inference chip: more throughput and lower latency at once, with 40 to 60% less power. Vendor numbers.

OpenAI Benchmarks Jalapeño Against Nvidia's Racks

Ricardo Argüello

Ricardo Argüello
Ricardo Argüello

CEO & Founder

Business Strategy 3 min read

Nine months from design to tape-out.

That is the Jalapeño number I keep coming back to, and it is not the one making headlines. A chip of this size used to take two to three years. OpenAI says its own models helped compress the design cycle, and Broadcom did the silicon implementation.

The benchmarks are louder. On August 25, OpenAI published first results claiming 1.5x to 1.9x the throughput of Nvidia’s GB200 and GB300 racks, end-to-end latency 1.7x to 3.6x lower on GPT-OSS-120B, DeepSeek R1 and Kimi K2.5, and estimated power draw 40 to 60% lower.

Keep the label attached. OpenAI supplied every number. SemiAnalysis witnessed the InferenceX runs in its lab and confirmed the GSM8k evals landed on par with Nvidia, but it did not run the full suite or see AgentX results. The part is not shipping, and vendors publish the scenario where they win.

Your per-token price has a hardware floor

When you negotiate an API contract or compare model providers, you are looking at the far end of a chain. Under that price sits the real cost of running a model, and hardware sets it.

For three years that hardware was general-purpose. The same parts that train models also serve them, which leaves capability idle that inference never uses.

Jalapeño trains nothing. Richard Ho, OpenAI’s VP of hardware, described the goal as minimizing data movement and communication delay. One system carries 128 accelerators, 1.7 exaFLOPS of 4-bit compute, 27.5 TB of HBM4 and 2 petabytes per second of memory bandwidth.

Specializing usually beats serving two masters. That has been the hardware story for forty years.

A shortening cycle beats a faster part

If nine months holds as the new cadence, the refresh cycle for AI hardware roughly halves. Whoever holds the best part today holds it for far less time than an enterprise infrastructure decision lasts.

Sign a three-year commitment against a cost reality that will turn over twice inside the term, and the commitment is the risk, not the chip.

What this actually asks you to do

Nothing involving hardware purchases. Something cheaper.

Your company has a list of shelved use cases. It lives in a 2025 email thread or a slide, and the reason written beside each one is some version of “too expensive per call.”

That list was built against a token price that has already moved and will move again when this class of silicon reaches volume in 2027.

We wrote about that trap in the per-token price lies, and about a concrete instance of it in Astra’s ten math proofs and the cost of inference. Same conclusion both times: a use case killed on cost is not dead, it is paused, and it deserves a calendar entry.

Concretely, this week

Find the list. If it does not exist, writing it is the first job: every case someone proposed and someone killed, with the reason beside it.

Split it in two. Cases killed on cost, and cases killed for anything else. The ones that died because the process was unclear or because nobody owned the outcome are not fixed by any chip, ever.

Put a date on the cost pile. Every six months, recompute at current prices. It takes half an hour and it is the highest-return review I know of, because the expensive part, understanding the process, is already done.

Jalapeño reaches volume in 2027. The list can be reviewed Tuesday.

Recompute one case you shelved on cost

Frequently Asked Questions

OpenAI Broadcom Jalapeño inference cost Nvidia AI infrastructure B2B strategy

Related Articles

NVIDIA Guaranteed $105B of OpenAI's Lease Obligations
Business Strategy
· 7 min read

NVIDIA Guaranteed $105B of OpenAI's Lease Obligations

NVIDIA backstopped up to $105B of OpenAI's Ohio lease. It only pays if OpenAI fails, and it terminates the moment OpenAI earns a credit rating.

NVIDIA OpenAI circular financing
OpenAI Astra: ten open problems for $2,000 in tokens
Business Strategy
· 8 min read

OpenAI Astra: ten open problems for $2,000 in tokens

OpenAI says an internal version of Astra cracked ten long-open problems, each with a Lean 4 certificate. The tokens would cost about $2,000 at Sol API rates.

OpenAI Astra Lean 4
Perplexity at $30B on $750M, and Nvidia May Fund It
Business Strategy
· 4 min read

Perplexity at $30B on $750M, and Nvidia May Fund It

The Information reports Nvidia in talks to back Perplexity above $30B. In January it committed its entire annual revenue to three years of Azure compute.

Perplexity Nvidia answer engines
Pinecone Nexus Beat GPT-5.5 by One Point, at 77% Less
AI & Automation
· 4 min read

Pinecone Nexus Beat GPT-5.5 by One Point, at 77% Less

Coverage said Nexus outscored frontier models. The primary says 47.4% against 46.4%. What the knowledge layer actually bought was cost, on the cheaper model.

Pinecone Nexus Sierra
AI Prices Fell 40x. Most Marketing Teams Didn't Notice.
AI in Marketing
· 5 min read

AI Prices Fell 40x. Most Marketing Teams Didn't Notice.

Fixed-capability AI inference dropped 40x per year since 2023. The value is not in this week's frontier model, it's in the workflows you never systematized.

AI in marketing AI strategy marketing automation