Artificial Intelligence – Silicon Game

Cerebras admits it can’t compete in training.

• Cerebras’ pivot towards inference is both an indication of where the market is going and an admission that when it comes to training, no one is close to competing against Nvidia.
• Cerebras Systems is a privately held semiconductor company that designs and sells silicon chips for AI training and inference and is a direct competitor to Nvidia.
• Its secret sauce is that it has figured out how to make massive chips and be able to deal with the inevitable manufacturing errors that creep into chips that are this large.
• Its latest WS-3 chip (announced in March 2024) is 46,225m² compared to the H100 which is 826mm² and has 900,000 cores compared to the H100 at 16,896.
• The idea with a chip this size is that the bottlenecks of shuttling data from one chip to another when training or inferencing massive LLMs is removed and the chip can run much faster than a collection of chips.
• I have long been of the opinion that Cerebras is a niche player where use cases of very high performance are more important than the ease and well-established practice of using Nvidia’s CUDA platform.
• However, I have long suspected that even in these use cases, it has been struggling in the training market especially now that Nvidia has released Blackwell against which the WS-3 will have much less favourable advantages.
• Nvidia’s advantage remains that all the developers prefer to use its CUDA platform to train their models and its product cadence is so fast that it is always at least one generation ahead of its competitors.
• While developer preference remains for CUDA, I don’t see anyone making a dent in Nvidia’s position in training which is why Cerebras is making this pivot towards inference.
• According to Nvidia and Cerebras, inference is now 40% – 50% of the AI chip market and I think that this percentage will grow rapidly as LLM-based models are deployed at scale, particularly by the big players.
• Presumably, model owners will be able to take models that they have trained using Nvidia chips and run them on Cerebras silicon which will be a great help in dealing with the disadvantages that Cerebras has when it comes to CUDA and product cadence.
• Cerebras is making this available as a service powered by three (soon to be four) data centres to developers and enterprises and has taken an aggressive stance on pricing to try and put a dent in Nvidia’s dominance.
• I think that this is a good move from Cerebras as while the focus of developers remains on the silicon development platform, it (and all its peers) have no chance against Nvidia.
• However, the inference market is wide open and for the moment, inference is going to be carried out in the cloud.
• I think in the long term, inference will migrate to the edge as the economic, speed, privacy and security arguments for inference-at-the-edge are a no-brainer in my opinion.
• This may leave Cerebras being squeezed by Nvidia in training from one side and inference at the edge from the other but here I think its positioning as a niche player will come into its own.
• I think that Cerebras works well for those who design their own AI systems and models from scratch and where the use case demands cutting-edge performance.
• In this regard, it has a very small position in training but I think the prospect is to win a higher amount of share in inference.
• Whether this is enough to justify its current valuation of around $4bn remains to be seen but I think there is a place for it in the AI market.
• None of this is enough to interrupt the Nvidia AI juggernaut which I expect will continue to sweep all before it when it reports its FQ2 results after the close on August 28^th, and so there are no immediate negative implications from this for Nvidia at the moment.
• I remain nervous about valuations being paid in AI at the moment which is why I continue to prefer the adjacencies of inference at the edge and nuclear power to run all of these data centres.