Answer in brief
On 25 August 2026, OpenAI reported that Jalapeño delivered 1.5–1.9 times more AI work per watt at peak throughput and 1.7–3.6 times lower end-to-end latency than the comparison systems across three tested open-weight models. Those are benchmark results, not a universal promise for every model or production workload.
Verified facts
- Announcement date
- 25 August 2026
- Workloads tested
- GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T
- Benchmark
- SemiAnalysis InferenceX
- Deployment plan
- OpenAI says internal deployment is planned by the end of 2026
What OpenAI published on 25 August 2026
OpenAI published the first measured performance results for Jalapeño, its custom chip and rack-scale system for language-model inference. The company says the system was evaluated with SemiAnalysis's public InferenceX benchmark across GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. Across those tests, OpenAI reported 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the commercial systems used for comparison. For the highly interactive operating points it selected, the company reported 2.1 to 4.1 times higher performance.
Those figures are specific claims with a visible measurement frame, not a statement that Jalapeño is faster for every model, prompt, precision or service pattern. The OpenAI publication provides model names, nominal context and output lengths, comparison hardware, package power assumptions and several latency and throughput measures. That level of detail makes the release more useful than a single peak number, but the correct reading still begins with the conditions. A benchmark result describes what happened inside its test boundary; a production conclusion requires evidence that the boundary resembles the workload being purchased or operated.
Why throughput per watt and latency must be read together
Inference systems are often presented through one headline metric, usually tokens per second or aggregate throughput. That can hide the experience of an individual user. A system may batch many requests efficiently while making each user wait, or serve one interactive request quickly while leaving expensive capacity underused. OpenAI's release emphasizes a combination of work per unit of power, end-to-end latency and time between generated tokens. Reading the axes together is the right instinct because an interactive product has to balance responsiveness, capacity and cost rather than maximize only one of them.
The balance matters even more for agents. A chat response may require one generation, but an agent can make a long sequence of model calls, tool decisions and validations. Small delays compound across those steps. Lower latency may shorten the whole task, while higher useful throughput can let the same infrastructure serve more simultaneous work. Power efficiency matters because electricity and cooling constrain how much compute can be deployed in a data center. None of those benefits is automatic, however: the model, scheduler, cache behavior, network and software stack have to preserve them when the service moves from a controlled benchmark into production traffic.
What the InferenceX comparison actually covers
InferenceX is designed to measure the broader serving path rather than an isolated arithmetic peak. OpenAI says it tested Jalapeño across operating points that move between high throughput and highly interactive, low-latency service. The published appendix reports results for three open-weight models and compares Jalapeño with commercially available systems using stated power ratings. This gives readers more than one workload and more than one point on the performance curve, reducing the risk that the headline rests on a single convenient configuration.
The comparison still has boundaries. OpenAI rates Jalapeño at 700 watts and says measured sustained power remained at or below 550 watts for the tested workloads, while the normalized comparison uses published package ratings for each accelerator. Package power is useful for consistency, but it is not the same as measuring an entire rack, network, cooling system or data-center overhead under identical conditions. Model precision, kernel maturity, input and output lengths, batching, cache reuse and concurrency can also change the shape of a result. A technical buyer should preserve those variables when reproducing the test and should not mix a package-level numerator on one platform with a system-level denominator on another.
Why a model company would design its own inference system
OpenAI describes Jalapeño as a co-designed system spanning the chip, memory, network and serving software. Language-model inference moves through phases with different bottlenecks: prompt processing is compute-intensive, token generation is often constrained by memory bandwidth, and communication can leave processors waiting when model state has to move. A design built around those phases can place data, compute and networking differently from a general accelerator adapted to many workload types. The strategic attraction is tighter control over latency, energy use, capacity planning and the economics of serving models at large scale.
The project also sits inside a broader supplier portfolio. OpenAI says partner systems from Microsoft, NVIDIA and other providers remain foundational, and the Jalapeño publication says the company will continue to deploy NVIDIA accelerators for training and inference. Broadcom's June announcement framed the chip as a collaboration built around large-language-model inference rather than a universal replacement for external hardware. That distinction matters. First-party silicon can add leverage, a tailored path and negotiating power without eliminating the need for other accelerators. The operational question is how workloads will be divided across the portfolio once cost, availability, reliability and model compatibility are measured together.
What the first results do not establish yet
The release does not yet establish the all-in cost of producing and operating Jalapeño at scale. Performance per watt can improve economics, but a complete cost model also includes manufacturing yield, packaging, memory, networking, racks, power delivery, cooling, software engineering, qualification, maintenance, utilization and the opportunity cost of capacity. Nor does a benchmark prove availability. A service depends on stable supply, predictable failure behavior, monitoring, replacement processes and enough deployed volume to absorb demand spikes. Those questions are normal for a first generation; they are simply outside the scope of a performance chart.
The results also do not prove the same advantage for every closed frontier model or every agentic workflow. OpenAI says internal testing on frontier models showed a wider advantage, but the public comparison uses the named open-weight models. That is the evidence a reader can inspect today. The company also says production qualification is continuing, the software is maturing and more models are being validated. Until deployed measurements arrive, the responsible conclusion is narrower: Jalapeño produced strong results in the disclosed test setup and gives OpenAI a credible first-party inference path, while production efficiency, reliability and breadth remain subjects for the next evidence release.
A practical way to evaluate the claim for a real workload
Start with the workload rather than the vendor. Record the model, precision, average and tail input length, output length, concurrency, cache reuse, required time to first token, time between tokens and completion-rate target. Decide whether the product values a fast single-user experience, inexpensive batch throughput or a controlled balance. Then compare systems at a matched service level. If one platform is allowed slower responses or different numerical precision, the cost and efficiency result is not answering the same product question.
Next, keep the system boundary consistent. Measure accelerator, memory and network power on every platform, or use the same published package convention for every platform and label the limitation. Add rack, software, orchestration and reliability costs before estimating total cost. Use sustained throughput rather than a short peak, include p95 and p99 latency, and count useful completed requests instead of generated tokens alone when tools or retries are involved. Finally, rerun the test after a software update and with the actual traffic distribution. AI serving performance changes quickly, so an evaluation that cannot be repeated is a snapshot, not a procurement method.
What to watch between the benchmark and deployment
OpenAI says it plans to begin deploying Jalapeño in its own compute infrastructure by the end of 2026, while later generations are already in development. The next meaningful signals will be operational: which workloads move first, how much capacity is installed, what utilization the system sustains, whether software support broadens, how tail latency behaves and whether reliability remains stable across long-running production traffic. A customer-facing change in response speed, availability or price would be stronger evidence of business value than another isolated peak score.
There is also a market signal beyond OpenAI. Model companies, cloud providers and chip designers increasingly compete as full systems. The decisive unit is becoming the useful task completed within a latency, power, reliability and cost envelope, not the chip in isolation. Jalapeño's first published results strengthen that direction because they connect a model developer's serving needs to custom silicon and a public inference benchmark. The result deserves attention, but the disciplined interpretation is conditional: strong disclosed benchmark performance today, with production qualification and economic proof still to come.
Practical checklist
- Match the benchmark's model, precision, input length, output length and concurrency to the workload you actually operate.
- Check whether power is measured, sustained or based on a published package rating, and keep the same boundary for every system.
- Compare end-to-end latency and tokens per user alongside aggregate throughput; one metric cannot describe an interactive service.
- Include rack, network, memory, software, utilization and qualification costs before converting a benchmark result into total cost.
- Repeat the comparison after production deployment, using tail latency, reliability and useful completed work rather than a peak score alone.
Questions and answers
What is OpenAI's Jalapeño chip designed to do?
Jalapeño is a custom accelerator and system designed for language-model inference: serving trained models and returning responses. OpenAI says it is not a replacement for every training accelerator and will continue using partner hardware.
Which models were included in the first published benchmark results?
OpenAI published results for GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T on SemiAnalysis's InferenceX benchmark, covering models from more than one developer and several operating points.
Do the benchmark results prove a lower production cost?
No. Better performance per watt can improve serving economics, but total production cost also depends on manufacturing, rack design, networking, software, utilization, reliability, qualification and the workload mix.
When will Jalapeño be used in OpenAI's infrastructure?
OpenAI says it plans to begin deploying Jalapeño inside its compute infrastructure by the end of 2026. The company also says production qualification, software maturity and validation across more models are still in progress.