Google vs. ChatGPT – Scroogled again? pt. II

Bing chat is not ChatGPT.

• Samsung is weighing up a switch to Bing on the basis that its GPT-generated search results are better than Google’s which is a proposition that I think has little basis in reality.
• Google is currently the default search engine on both Samsung Android devices and Apple devices which is a reflection that in the current world of Internet search, it is the undisputed king.
• This is also reflected in its market share in search which is 90%+ and it’s $224bn in advertising revenues generated in 2022.
• The idea that Samsung might switch to Bing as a result of OpenAI’s superiority in GPT-based chat systems has caused another panic at Google which I take as a bad sign as Google has proved to perform very badly when it is running around like a headless chicken (see here).
• I continue to think that Bing and GPT-based systems are very far from challenging Google’s dominance in search for the reasons I lay out below.
• Ever since ChatGPT exploded into the public consciousness, the question that many are asking is whether these new systems make Google search obsolete.
• I think that they certainly have the possibility to change the nature of the search business and that does raise some risk of a market shift, but I have not seen anything from Microsoft that would lead to believe that Bing is suddenly miles better than Google.
• I have been using ChatGPT, Midjourney, Bing-powered by GPT-4 and Bard over the last few months and a few things become immediately obvious.
  • First, ChatGPT vs. Bing: which are two separate and distinct services and in terms of what they do, they are very different.
  • ChatGPT is a massive language model (175bn parameters) that has encapsulated the average knowledge of the internet frozen in time at the end of September 2021.
  • It is very good at digging up data from its knowledge base as well as working out what the main points of a piece of writing are and highlighting those to the user.
  • Unfortunately, it also has a tendency to make things up out of nowhere which means that every piece of data that it produces has to be independently verified.
  • ChatGPT is frozen in time because these models are incapable of dealing with changes in the dataset and for search, this renders them useless for many of the things that people normally search for.
  • The new Bing chat is a completely different animal as it makes use of the conversational capabilities of ChatGPT but when it looks for information, it does so using the Bing search engine.
  • This means that the ultimate performance of Bing chat for real use cases like finding out train times or buying a new pair of shoes, it is simply the old Bing search with a conversational front end.
  • Second, Google Bard: which is very similar to the new Bing chat.
  • Google is using one of its smaller language models (50bn parameters I believe) to provide the conversational front-end but when it looks for information it is using the Google search engine.
  • Over the last few weeks, I have used both of these systems extensively for research purposes and I find that Bard is better than Bing chat.
  • ChatGPT is still much better at having a conversation or summarising text but ChatGPT is not providing a proper search function.
• This is why I think that Samsung is mixing up two different products when it is thinking about what search to set as default on its devices.
• If it chooses Bing to replace Google, I suspect that it will be expecting to get ChatGPT.
• However, it will end up with Bing which in my testing is simply not as good as Bard when it comes to search.
• Bard and Bing are pretty much the same search engines that were in use before with a conversation function stuck on the front end.
• Hence, their performance will be determined by the quality of the underlying search engine and not the front end, and here Google still meaningfully outperforms Microsoft.
• To make Google obsolete, OpenAI and Microsoft would need to make their GPT models able to deal with a dataset that is constantly changing which is something no one in AI has been able to reliably demonstrate to date.
• Furthermore, I suspect that in order to do this, a fundamental breakthrough in the methodology of how AI is created would be needed which has not happened despite the sudden jump in terms of the quality of conversational AI.
• Instead, what has happened is that a combination of Google’s 2017 transformers innovation and massive data and massive compute have been put together to produce GPT-based systems.
• All of the long-standing limitations of AI still exist in these systems which is what causes them to hallucinate and make things up and this is unlikely to change anytime soon.
• Hence, I do not think that Google’s search business is under any imminent threat, and should it lose Samsung to Bing, I suspect that it would come back quite quickly once it realizes that Bing chat is not ChatGPT.
• Therefore, if Google’s shares suffer a large correction, this would represent a very interesting opportunity to pick them up cheap.
• This has not happened yet, but it is something to watch out for.