NVIDIA hardware has powered much of the recent growth in AI, including training and serving large models. OpenAI’s work with Broadcom shows why large AI companies also want hardware designed around their own workloads.
That is why OpenAI’s latest announcement matters.
OpenAI revealed Jalapeño, its first custom inference chip, developed with Broadcom. Inference is the work a model performs after training, including answering prompts, generating code, and serving responses. That makes Jalapeño different from the large GPU clusters commonly used to train models.
The important question is not whether one chip replaces NVIDIA. It is whether OpenAI can lower the cost and power demands of the repeated inference work behind its services.
A custom chip can be tuned for OpenAI’s workloads instead of serving every type of customer. It may also give the company more control over cost, power use, and infrastructure planning. OpenAI has not established here that Jalapeño will outperform NVIDIA hardware across other workloads.
OpenAI is following a path already taken by Meta, Amazon, and Google, which have developed accelerators for their own services. Apple’s investment in U.S. chip manufacturing reflects a different part of the same desire for more control over hardware supply and planning.
This does not put NVIDIA in immediate trouble. OpenAI still depends on a broad computing ecosystem, and an inference processor does not replace every training or general-purpose GPU workload. Jalapeño gives OpenAI another option rather than proving that it can leave NVIDIA behind.
Jalapeño also shows that OpenAI is operating as an infrastructure company as well as a model developer. Its services depend on data centers, networking, power, and chips, not only the models visible to users.
The chip gives OpenAI more influence over one layer of that infrastructure. The unanswered questions are performance, deployment scale, manufacturing economics, and how much work will continue running on NVIDIA hardware.
Support independent tech journalism
NERDS.xyz is independently owned and operated. If you enjoy my coverage of Linux, AI, hardware, cybersecurity, and tech culture, consider supporting the site on Ko-fi.
Support NERDS.xyz