ByteDance is designing its own AI inference chip, and the architecture it has reportedly settled on looks a great deal like Groq’s. Groq is the inference-chip company whose Language Processing Unit was fast enough at running large language models that Nvidia, in December, paid roughly $20 billion to make the problem go away.

The structure of that payment is the part worth remembering. Nvidia did not buy Groq. Nvidia signed a non-exclusive licensing agreement for Groq’s LPU technology, the largest deal in Nvidia’s history, comfortably ahead of the $6.9 billion it paid for Mellanox in 2020. It was non-exclusive because an exclusive acquisition of the one architecture beating Blackwell on inference speed would have been a difficult thing to explain to an antitrust regulator. One analyst described the arrangement as structured to keep the fiction of competition alive, which is a polite way of saying Nvidia paid $20 billion to own a thing while maintaining that it did not own the thing.

ByteDance is now testing how non-exclusive non-exclusive really is. It is not licensing anything from Groq or from Nvidia. It is building a chip that works the way Groq’s chip works, in China, for its own use, which is permitted, because the license Nvidia bought was the kind that leaves the underlying ideas where they are. Nvidia spent $20 billion securing the commercial future of this architecture in the markets it sells to. It did not buy the right to stop a Chinese company from building the same kind of chip, because that right was not for sale.

The reason ByteDance wants one is the same reason everyone wants one. Inference, the part where a deployed model actually answers a query or runs an agent, leans harder on chips that are not Nvidia GPUs than training ever did. ByteDance is also designing its own CPUs, having watched Intel and AMD raise prices by as much as 35 percent quarter over quarter. ByteDance buys a very large number of chips, and paying 35 percent more for them every quarter is the kind of thing that makes a company want to make its own chips. So it is making its own chips, both kinds, because they are expensive and ByteDance would prefer they were not.

This slots into a now-familiar pattern. The chip supplier of record in China is Huawei, which recently proposed a new law of physics to explain why being three years behind TSMC counts as progress. ByteDance, the country’s largest single consumer of inference compute, already has Qualcomm building it export-compliant ASICs and a proprietary chip of its own. The frontier lab raises money from more or less all of China at once. The thing being assembled is a full domestic stack, and the workload owners are now designing their own corners of it rather than waiting for Huawei to catch up.

Nvidia’s $20 billion bought the inference era in the markets it can reach. The markets it cannot reach now include a Chinese company building the same kind of chip for itself, which paid Nvidia nothing, and which Nvidia’s $20 billion was never going to stop.

–
By the Control Plane Editorial Team