Google AI – Penny Pincher

Gen AI is clearly very expensive to operate.

• Google is toying with the idea of putting new features of its flagship search service behind a paywall which indicates that the cost to deploy these features is so high, that advertising cannot support them.
• Since ChatGPT first appeared, there has been constant concern that generative AI would undermine Google’s position in search, but 18 months later there is no sign of this.
• Google’s market share has not materially changed and its revenues and profitability show no sign of competitive pressure.
• However, it is prudent to look at ways of maintaining distance from the competition which is what Google is doing in examining how it might use generative AI in its search service.
• The first problem with using generative AI for search is that search needs to access information in real time but the LLMs which power generative AI are fixed at a single point in time.
• This has been patched over with an architecture called Retrieval Augmentation Generation (RAG) that allows an AI to conduct a search in real-time, but these are still a lot less than perfect.
• The other problem is that generative AI provides a single answer to a query while a traditional search provides several, some of which are advertisements from which Google earns its money.
• Hence, a move to generative AI-powered search has the potential to upend Google’s business model.
• Instead, what I have thought most likely for a while is that Google will use generative AI to make its existing products better and more competitive, thereby preserving monetisation and keeping the competitive threat at bay.
• The problem with this is that at the moment, generative AI is very expensive to implement given the sizes of the models involved (1tn+ parameters) and the amount of compute power required to train and then process queries.
• Consequently, if Google was to add these features to its current search service, the chances are that margins would decline because revenues would increase enough to compensate for the added processing costs incurred.
• This is most likely to be why Google is considering putting the gen-AI-supported search features behind a paywall because then there will be an incremental revenue stream attached to the added compute power being deployed to provide the service.
• This combined with Nvidia’s exploding top line and OpenAI’s voracious capital requirements are all signs that generative AI is expensive to create and run.
• In Google’s case, the real problem is not the training of the AI but in supporting the requests as Google has billions of users many of whom use the service multiple times every day.
• This is in line with RFM Research’s conclusion that the real cost of AI is likely to be in inference as opposed to training which is where most of the money is going at the moment.
• I suspect that if Google is going to go ahead with this, it will announce it at its developer conference Google I/O which will be held on May 14^th.
• AI will almost certainly be the main focus of Google I/O where I am expecting Google to launch an SDK for Gemini that will allow developers to make their own gen AI services based on Gemini and distribute them through the app store.
• This is the beginning of Google’s move to create its own AI ecosystem and offset the threat presented by Microsoft and OpenAI.
• I expect all the others will follow suit this year.
• I think that fears of Google being left behind in AI are overblown and that alongside Baidu, Google offers one of the cheapest ways to gain exposure to AI.
• The purest play is Nvidia which thanks to its explosive growth does not trade at a crazy valuation, but its market share is so high that should the bubble burst, there will be very little it can do beyond hang on for dear life.
• However, in this environment of plentiful and almost free money, there is probably still room for Nvidia to go higher although the biggest move in its share price has already happened.