How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip
On 25 August, OpenAI fully unveiled Jalapeño, its debut AI accelerator chip. Jalapeño delivers up to 13.4 petaflops of 4-bit compute and accesses 232 gigabytes of the most advanced memory available, linked at 15.4 terabytes per second. Benchmarks cited by OpenAI show it can cut end-to-end latency — the time from prompt to last token — by up to 3.6 times versus Nvidia's GB300, a chip OpenAI currently relies on, while consuming less power. Whether those figures translate into real-world gains once Jalapeño is deployed widely in OpenAI's inference fleet remains to be seen.
Performance is only half the story; the other half is how the chip was designed, a process OpenAI says its LLMs accelerated. Jalapeño went from first architecture concept to first silicon in under 20 months, with only nine months separating the first RTL — the register-transfer level code defining the chip's logic — from tape-out, when the finished design goes to manufacturing. That is a rapid timeline, though experts believe it could soon look slow as LLMs improve and become more deeply integrated into chip design tools. OpenAI is bullish: "The models are giving superpowers to our engineers," said Richard Ho, vice president of hardware at OpenAI. "Our engineers are still driving the work. They're still the final arbiter of what's going on. But they can do things a lot faster. They can explore a lot more paths."
The design team was small. Ho says the group that designed Jalapeño averaged fewer than 100 people over the project and stands at roughly 100 today as the team pursues second- and third-generation designs. That count spans roles across the hardware team, from system design to software and supply chain, but excludes Broadcom employees, who partnered on the project. Labor was split between design and implementation: OpenAI's team owned end-to-end system design, including the inference accelerator, the memory hierarchy, and networking, while Broadcom handled "physical design from the gates onward," per Ho.
The Broadcom partnership tempered some outside assessments of OpenAI's speed. David Chin, co-founder of agentic chip design startup Verkor.io, called the schedule "quite credible" but said Broadcom's help was essential to the rapid timeline — "If you have somebody else start from scratch, it won't be possible." Verkor co-founder Ravi Krishna called OpenAI's speed "a relatively impressive result" while noting that LLM capability improvements mean a project started today could move even faster. Andrew Kahng, a distinguished professor at the University of California, San Diego, also found OpenAI's speed no [excerpt truncated].