Nvidia – The moat

Nvidia makes the moat even bigger.

• Nvidia has launched an update to the H100 which offers mostly incremental improvements, but the cadence of new products combined with the power of CUDA leaves Nvidia looking pretty much unassailable for now.
• At the Supercomputing show, Nvidia unveiled its latest processor the H200 which is the first chip of its type to make use of a new type of memory called HBM3e.
• This memory offers both greater capacity and crucially, faster speed as when training algorithms on clusters of chips, the speed with which chips can communicate becomes a crucial factor.
• Most of the comparisons being made by Nvidia when it comes to training are against the A100 where the H200 offers double the capacity and 2.4x the bandwidth.
• This is a similar marketing strategy to Apple which made most of the comparisons of the M3 processor against the M1 as the M2 comparisons were much less interesting.
• Compared to the H100, the figures are much less impressive, but it is not in training where I think the real benefit of the H200 is to be found.
• Generative AI has two main segments which are the training of the algorithm (which is where all the excitement currently is) and the execution of requests from users by the trained model (inference).
• RFM Research has long believed that the market for inference will dwarf the size of the training market and that the majority of services that run at scale will execute inference at the edge.
• There will also be a significant amount of inference in the cloud as well (given the size limitations of inference on smartphones) and it is here where the H200 offers a stepwise change in performance.
• Here the H200 can run inference on LlaMa 2 (70bn) at double the speed that the H100 can offer with more to come with software updates.
• In my opinion, this is the most interesting upgrade as once all the hype dissipates and models are in the market in volume, it is going to all be about inference and less about training.
• The H200 pushes Nvidia ahead of its rivals yet again and outside of the giant digital ecosystems that have plenty of resources, the cadence of updates makes life very difficult for the smaller players who simply cannot afford to keep up.
• Furthermore, with more than a decade lead with its CUDA compute platform, CUDA is the go-to place to develop and train generative AI algorithms.
• Its tools a better known, more mature and wider reaching than anyone else’s which is what makes them so popular and they are the foundation of Nvidia’s ability to sell its chips at 70% gross margins.
• This update widens the moat slightly but adds more weight to the proposition of the best tools also running on the best silicon available.
• This is evidenced by AWS, Azure, Google Cloud and Oracle all agreeing to put the H200 in their data centres even though 2 of them have chips of their own and Azure is working on one.
• This is because if they don’t offer Nvidia, they will lose customers in yet another sign of just how strong Nvidia’s position is at the moment.
• There are signs of how this might weaken over the long term but for the next 12 to 24 months, there is very little on the horizon that is going to put a dent in Nvidia’s armour.
• However, all of this looks to have been accounted for as Nvidia shares are priced for perfection and allow for no bursting of the AI bubble which I think is inevitable.
• Hence, I would have sold Nvidia long ago if I had it.
• Inference looks to be a better place to look as no one is really talking about it at the moment and here Qualcomm (which I hold) is the one to look at in terms of both exposure and valuation.