Artificial Intelligence – Hard Times in China

A potential benefit of hardship. 

• China’s lack of cutting-edge silicon and limited amounts of cash to spend on compute is forcing it to innovate in ways that may allow it to compete very effectively with AI especially when it comes to cost.
• The prevailing view in Silicon Valley is that the bigger the models become and the more data that is used to train them, the better they will perform.
• This is referred to as ‘The Scaling Law’ which is why the models are getting bigger and bigger and are costing more and more to train and run.
• It is also the main driver for silicon chips that can handle increasing model sizes and data sets and do so quickly and efficiently.
• The best example of this is Nvidia’s Blackwell chip which is designed for model sizes of 1tn+ parameters and costs far more than its H100 predecessor.
• However, its increased throughput and power efficiency more than makeup for its increased price, meaning that the same model trained on Blackwell will cost about half as much as it would on the H100.
• This abundance of compute and almost limitless amounts of financing to pay for it means that very few avenues other than bigger is better are being pursued in the West.
• China has none of this abundance as US restrictions mean that it cannot purchase the latest chips and the moribund state of its economy, venture capital and private equity sectors mean that there is very little cash available for investment.
• This has forced Chinese companies to do much more with less and while this leaves them down the league tables now, this may not always be the case especially when the AI bubble bursts and Western cash for AI investment dries up.
• 01.ai, DeepSeek, Alibaba, ByteDance, Tencent and Baidu all have models available for developers to create services on and are making them available at very low prices ($ per million tokens for inference) to encourage development.
• 01.ai’s model Yi-Lightening and Baidu’s ERNIE models are far from the cutting edge, but as the gap between the leaders and the rest of the pack narrows, this becomes less important.
• The lack of cutting-edge silicon has meant that these companies are forced to do more with less compute, and this has pushed them to seek other ways to make improvements.
• One of these is to use a series of smaller models each an expert in a particular field and then stitch them together to match the performance of larger models while another is to use smaller models that are trained with better-quality data.
• This makes sense as data labelling and scrubbing are tasks where China has no restrictions as well as plenty of high-quality labour to carry out the tasks.
• The result of this approach is that the models are cheaper to run which combined with brutal competition in the local market has made them far more cost-effective than their Western counterparts.
• For example Yi-Lightening which performs within touching distance of GPT-4o and Claude 3.5 costs $0.14 per million tokens compared to $4.16 for GPT-4o and $6 for Claude 3.5.
• This does not mean that Yi-Lightening will be any good for tasks outside of China, but it is an indication that limited resources are forcing Chinese AI companies to develop new techniques to be efficient and cost-effective that Western companies are largely not bothering with yet.
• When the generative AI bubble comes to an end, as it surely must, techniques for training and inference of LLMs with fewer resources and greater efficiency will suddenly become important and we may find at that time that China is ahead.
• This would be a blow to US policy which has been to limit China’s access to technology so that when it competes with US technology in 3^rd party countries it is no longer the ‘just-as-good’ but cheaper option.
• A lot of this depends on whether Chinese companies can make money at $0.14 per million tokens and there are so many variables (including subsidies) which are unknown, that it is impossible to tell one way or the other at the moment.
• This could be an unintended consequence of both the export restrictions and regulation that have greatly damaged the Chinese technology sector.
• Consequently, I still see China as being highly competitive in AI albeit not at the cutting-edge specification that most people would measure it by.
• Hence while the restrictions are effective at keeping China out of the cutting edge of AI, it is well represented and is faring well in the fast followers segment which in most use cases is probably good enough.
• Even with the stimulus-related rally in the Chinese stock market, China remains by far the cheapest way to invest in AI although the regulatory risk remains very high and very uncertain.
• This is what continues to put me off jumping back into China (although I still hold Alibaba) which is why I am sticking with my AI adjacencies of inference at the edge and nuclear power.