USA vs. China – King Qwen

Good Chinese AI but at what cost?

• Alibaba’s models have topped the new rankings at Hugging Face in a sign that China is a force to be reckoned with in AI even without leading-edge semiconductors.
• However, like in advanced semiconductors, the battle will be won or lost on who can produce the product most economically, and here the US still has a big advantage.
• Hugging Face has revamped its Open LLM leaderboard to make it more challenging so that the differences between the different models become much clearer and to prevent model creators from gaming the system.
• Hugging Face has added new benchmarks that are more difficult to score highly on as well as ensured that the evaluation is completely fair for every contender.
• Somewhat surprisingly, it is not Meta, Mistral or Microsoft that has topped the table but Alibaba with its Qwen-72b-Instruct model.
• Furthermore, the difference between Qwen and second place Llama-3 70bn was significant with Qwen scoring 43.02 compared to Llama-3 which scored 36.67.
• It is important to note that not all models have been tested yet and so things might change but Alibaba getting positions 1,7 and 8 in the top 10 and 01.ai, another Chinese AI startup, getting positions 4 and 10 is a surprisingly good achievement.
• This underpins RFM and Alavan Independent’s long-held view that in newer technologies, the Chinese are very competitive and are in a position to challenge the USA for global leadership.
• We find this to be also true in autonomous driving, The Metaverse, quantum computing and several other very new technologies.
• This is why we have seen the USA tighten export controls on semiconductors and semiconductor capital equipment further over the last few years.
• All of these new technologies need semiconductors to function and, in many cases, the Chinese variants made using older technology will be bigger, heavier, consume more battery life and, most importantly, be much more expensive to make.
• This is exactly what we have seen with all the furore around SMIC and Huawei making 7nm chips where they have succeeded in making working advanced chips using older technology, but the yield is so low that it is economically ruinous to do so at scale.
• I suspect that we are seeing the same in AI where Chinese companies are demonstrating that they can produce models that perform just as well as everyone else, but the question is how much does it cost to build and train these models?
• Given that China is limited to older generations of AI chips from Nvidia and AMD and will be unable to make its own economically at any node smaller than 14nm, I suspect that it costs China a lot more.
• It is important to note that the really big guns are absent from this assessment as this test is limited to those that are available in the open-source community.
• This means that the multi-trillion models of OpenAI, Gemini, Anthropic and so on which are closed source, have not been assessed using this benchmark.
• The admissions from the industry are indicating that what is in the labs is not massively better than what is already available and so I continue to see the rate of improvements slowing making it easier for the laggards to catch up.
• Hence, I suspect that the massive Chinese models (not tested on Hugging Face) will perform comfortably within the range of their Western counterparts, but the real question is at what cost?
• Nvidia’s Blackwell chip cuts the cost of training by 75% which Nvidia splits with the customer leading to a 50% cost saving for the customer but Chinese competitors will be being trained on chips that are at least one, and probably more, generations behind.
• This means that while they will be able to perform in line with the Western versions, they will cost orders of magnitude more to train and run and consume far more power.
• Hence, this will make Chinese technology less economic at home and when it comes to convincing other countries that they should choose Chinese technology, the argument of low cost will start to break down unless there is massive subsidisation.
• The problem there is that China cannot afford to subsidise on this scale as its economy is in the doldrums and its demographic indicators are all pointing the wrong way.
• Hence the USA and its allies have an advantage not dissimilar to the one they had against the Soviet Union during the Cold War although not nearly on the same scale.
• Consequently, we see China continuing to decouple from the USA and increasingly it will set its own technology standards which are incompatible with anything else.
• In the long run, multiple and incompatible standards are bad news for everyone because the total opportunity for value creation will be orders of magnitude less.
• This means lower long-term growth for the entirety of the technology sector unless there is a rapprochement of some sort.
• At the moment, this is a very distant and dim hope.