Baidu – ERNIE 1.0

The real ERNIE 1.0.

• Baidu rolled out the latest version of its ERNIE chatbot which is a great improvement over previous versions and in my opinion, is what the company wanted to launch back in March but was simply not ready.
• Instead, like Google, it had to rush to market with something to demonstrate that China was not being left behind in generative AI.
• Baidu held its annual Baidu World 2023 event in-person for the first time in several years where it showed off its upgraded ERNIE chatbot which is already in its 4^th version.
• RFM Research (see here) has long rated Baidu as a leader in AI although more recently, it has been overtaken by OpenAI and Microsoft and so the fact that it has the best chatbot in China does not come as a big surprise.
• ERNIE 4.0 makes a number of improvements over its predecessors mostly in the areas of its ability to understand natural Chinese language and its ability to respond.
• The main difference between the initial launch in March and today is that Baidu felt confident enough to demonstrate ERNIE live rather than rely on a pre-record.
• This is a big step forward as all of these models are unpredictable and in China, the consequences for releasing an LLM that goes off the rails are far greater than they are elsewhere in the world.
• Back in March, ERNIE was clearly not ready, but international pressure meant that China needed to come up with something, even if it was not ready for release.
• ERNIE 4.0 headlines with most of the same abilities of ChatGPT based on GPT-4 and Baidu went to great lengths to state that ERNIE’s abilities were at least on par with GPT-4.
• The problem with this statement is that it is impossible to verify because ChatGPT and ERNIE are trained on completely different datasets.
• Hence, if they were asked to perform the same task one would get different results regardless of any differences in quality between the two.
• I suspect that ERNIE 4.0 remains quite some distance behind the best that OpenAI and Google can come up with for two reasons.

• First, controls. All chatbots that are released to the Chinese public are required to have a licence from the state that certifies that they have been tested such that they will not produce material or opinions that contravene Chinese socialist values.
• All LLMs are essentially black boxes meaning that that their creators have very little idea of how they work.
• Furthermore, when the LLMs go off the rails, they don’t know that they are going off the rails which makes them even more difficult to control.
• Hence, most LLMs are controlled by curtailing functionality entirely in certain areas to ensure that they don’t misbehave.
• In ERNIE’s case, the penalties are severe and so Baidu will have had to curtail functionality in many more areas than OpenAI will have had to.
• This will mean that ERNIE’s functionality will be more limited than ChatGPT’s.
• Second, data: A well-known problem in non-English language LLMs is the availability of training data as the vast majority of the Internet is in English.
• ERNIE is a Chinese bot that has been trained on predominantly Chinese data which will greatly limit the scope of the dataset that it has ingested to create its functionality.
• Consequently, ERNIE’s knowledge will be more limited than GPT4’s which in turn will reduce functionality outside of the Chinese language and the Chinese Internet.

• ERNIE will still be useful in China, and I suspect that on Chinese tasks, it will easily outperform ChatGPT but its functionality will be almost entirely restricted to China.
• This is yet another sign of the bifurcation of the Internet that RFM Research and Alavan Independent see evolving over the next 5 to 10 years.
• Eventually, this will lead to Chinese technology standards being incompatible with global standards further splitting the technology sector and reducing its capacity for value creation.
• This is a scenario where everyone is worse off and lower long-term growth of the technology sector is likely to result unless there is some form of rapprochement.
• This looks to be further away than ever.