Computex 2024 Day 0 – Nvidia – Stealing Thunder

Jensen consumerises GTC.

• Nvidia stole the early Computex limelight away from AMD with another massively attended keynote that mostly rehashed GTC but made the story more accessible and lifted the curtain on the 3-to-4-year roadmap.
• This had the added advantage of giving away the fact that the AI industry intends to continue down the road of making models bigger in the hope that they will continue to get better.
• Jensen Huang spent most of his keynote making a convincing case for accelerated computing in an easy-to-consume and accessible way but didn’t really announce anything of substance until almost at the end.
• This case is relatively straightforward in that the computing use cases where there is real growth can be addressed by parallel processing as opposed to logical steps that need to occur in a sequence where one follows the other.
• This allows massive acceleration by running tasks in parallel on multiple processors which is what Nvidia has been doing for decades.
• Jensen also spelled out what makes Nvidia so special which is the CUDA development platform which he claims is now past the tipping point and has entered a virtuous cycle.
• Nvidia now has 5m developers using CUDA which with its 350 libraries, is adding users daily which in turn is encouraging more developers to use the platform and so on.
• It is this, much more than its silicon offering, that cements its dominance of AI in my opinion.
• Apart from the good story of how Nvidia got to where it is today, Jensen’s main announcements were:
  • First, silicon roadmap: which will now operate on a 12-month cadence which I suspect that its smaller competitors will struggle to keep pace with.
  • 2025 will see the launch (presumably at GTC) of Blackwell Ultra, and 2026 will see Rubin with Rubin Ultra in 2027.
  • No real details of these were given, but given the commentary, I would suspect that they will all offer a continuation of the path that Blackwell is taking.
  • Here, Blackwell can train the same model as Hopper on ¼ the number of GPUs and ¼ the power consumption.
  • Assuming that Blackwell costs twice as much as Hopper, Nvidia is splitting the saving with the customer underlining Jensen’s well-known catchphrase “The more you buy, the more you save”!
  • Blackwell Ultra and Rubin are very likely to offer more of this.
  • Second, inter-chip communication: which is becoming a big issue as more and more chips all work together to train and execute larger and larger models.
  • This is why Nvidia purchased Mellanox in 2019 and last night it detailed its roadmap for inter-chip communication to keep the bottleneck of chip communication at a minimum.
  • For those that wish to use Ethernet rather than Infiniband, Nvidia revealed the roadmap of its Spectrum X Ethernet product which said more about the industry than it did about Nvidia.
  • Here the X600 is designed to link 10’s of thousands of GPUs with the X800 Ultra being designed to link 100s of thousands of GPUs and the X1600 to link millions of GPUs.
  • Together with the commentary around the roadmap for its data centre GPUs, it is clear that the AI industry intends to keep growing its model size to trillions and then tens of trillions of parameters in size.
  • The hope is that Kaplan’s scaling law of 2020 will hold in that bigger models are better but there are some signs that exponential increases in model size now deliver only linear improvements in performance.
  • This indicates that the performance of large language models may be closing on its maximum which, if true, will reduce the return on investment of training ever larger models.
  • This is why RFM research concluded in 2023 that smaller models trained with more data might be a more economical and profitable route, but the debate is still ongoing.
  • Third, AI on graphics cards: where PCs that use Nvidia graphics cards will also be able to run AI and 4 laptops (3 from Asus) were briefly displayed.
  • This looks to be Nvidia’s answer for Copilot + PCs where the current specification requires an NPU that supports at least 40TOPS.
  • At the moment, Qualcomm is the only company with a processor of this specification shipping commercially, but Intel is hot on its heels and AMD will not be far behind.
  • I am pretty sure that Microsoft will enable Copilot + to run on Nvidia graphics cards as it needs as many devices as possible to run Copilot + regardless of which supplier provides the silicon.
• Jensen also spent some time quietly cementing Nvidia’s dominance in AI but also went on to future-proof this position.
• I think that this is what Nvidia NIMs are all about.
• NIMs are pre-trained models for specific tasks that Nvidia will offer as part of its toolkit for building multipurpose agents or systems that have specific capabilities.
• Crucially, if these prove popular, NIMs take Nvidia’s stickiness up through the technology stack such that if developers are no longer using CUDA directly because they are developing directly on foundation models, they will still be demanding Nvidia silicon.
• Nvidia monetises AI by selling chips just like Apple monetises iOS by selling iPhones and iPads which is why NIMs could be crucial to long-term revenues and margins.
• Nvidia clearly intends to keep its dominance for as long as possible and in the current generation, there is nothing really on the horizon to challenge that.
• Hence, I still think that Nvidia’s stock can continue to rally in line with the rest of the AI frenzy, but the easy money has been made and I continue to look more laterally.
• Nuclear power and inference at the edge are my two favourite related themes.