Autonomous Autos 2019 – All change

The field tightens a long way away from commercial offerings.

• Disengagement data from the California DMV (see here) shows that Waymo has lost its lead in autonomous driving to Baidu explaining why Waymo has criticised this data for the first time.
• The use of disengagements to measure the quality of an autonomous driving system is deeply flawed but in the absence of any other measure, it at least gives an idea of where the field lies.
• Furthermore, for the last 3 years, it has provided a reasonable approximation of reality backed up by RFM’s qualitative research in this area.
• Hence, I am comfortable with drawing some broad conclusions from the data.
• The main conclusions I draw from the data are as follows:
  • First, the field has tightened at the top: Waymo has lost its lead to Baidu but the difference between the top 4 has closed substantially in the last 12 months.
  • The 2018 data showed that the difference in performance between No 1 (Waymo) and No. 4 (Pony.ai) was a factor of 12x.
  • This means that every time Waymo disengaged once, Pony.ai had disengaged 12 times.
  • This year a factor of 1.7x separates the top four with Baidu now at No. 1 and AutoX at No. 4.
  • Waymo has slipped to No.2 disengaging 1.4x every time Baidu has 1 and Cruise is at No.3, disengaging 1.5x every time Baidu suffers one.
  • There is still a broad correlation between miles driven and performance as the top 5 had all driven at least 32,000 miles in the last 12 months.
  • Given the weaknesses in this data and the fact that many of these companies are also testing in other locations which do not require data to be reported, I think it is impossible to distinguish between the top 4.
  • Second, improvements are plateauing: In 2018 Waymo was the clear leader driving 11,040 miles with no disengagements up from around 5,000 miles the year before that.
  • The number 2 was cruise which managed 5,204 miles with no disengagement on average.
  • This year, Waymo has made only marginal improvements but everyone else has caught up.
  • In 2019 Baidu managed 18,050 miles with no disengagements with Waymo managing 13,219 and Cruise 12,221.
  • Waymo’s competitors have improved far more than Waymo has.
  • This suggests that either the limits of what the current systems are capable of are being approached or that Waymo is now only deploying its best technology in areas (Arizona) where no disclosure is required.
  • Given, Waymo’s first-time criticism of this measure, I suspect that the pace of improvement is beginning to plateau and the plateau is not at a level good enough to support a commercial solution.
  • This is supported by RFM’s AI research which is still predicting a 3^rd AI Winter (see here) and a very long period of time before autonomous driving hits the market in any meaningful way (see here).
  • Third, a clear tier-II emerges: The second tier of three companies Pony.ai, Nuro and Zoox has emerged which disengage between 3x – 10x every time Baidu has one.
  • These were in the leading pack last year and they have been leapfrogged by Baidu and AutoX both of whom have shown a big improvement in the last 12 months.
  • The field then rapidly declines in performance to newcomer BoxBot which disengaged a massive 50,447x more frequently than Baidu.
  • However, to be fair it only drove a total of 39 miles so I am far from convinced that this means anything.
  • Fourth, the Chinese are improving rapidly: 3 out of the top 5 players are Chinese companies (Baidu, AutoX and Pony.ai) with Baidu claiming the top spot.
  • Those cynical of this measure (for which there are plenty of grounds) will say that the Chinese have simply worked out how to game the measure.
  • That being said, given the resources being thrown at this by the Chinese players, I think that this is a credible conclusion to reach and it also matches RFM’s qualitative research.
  • Fifth, Apple is not taking this seriously: Apple has improved materially in the last 12 months improving its relative performance relative to the leader.
  • In 2018 Apple disengaged 9,787x more frequently than the leader and it drove 78,605 miles.
  • In 2019 Apple disengaged 153x more frequently than the leader but it only drove 7,544 miles, a 90% decline compared to 2018.
  • Apple has a total of 70 vehicles of which none did any driving between December 2018 and June 2019 and then only 23 out of 70 (33%) drove any miles at all in the following months.
  • This leads me to believe that Apple is no longer serious about autonomous driving which makes complete sense as this has never made any sense when considering how the company makes money.
• There will be endless criticisms of using this measure, but I think that one can draw some broad conclusions from the data that has been presented.
• These are: that the field at the top is narrowing because the techniques being used are not good enough to bring a meaningful commercial solution to market.
• The Chinese have been and continue to invest hard in this area and are seeing some of the fruits of that investment.
• Yandex and Mobileye still report no data although Intel (Mobileye’s owner) has been testing something in the last 12 months.
• What this is is very unclear but if it is a Mobileye solution, it should be concerned as Intel scored poorly disengaging 2,300x more frequently than Baidu.
• However, it is worth noting that its driving patterns were very sporadic and it only drove 1,295 miles making this not a very representative sample.
• There are still way too many players in this market and given RFM’s expectations that there will be no meaningful revenues before 2028, I continue to expect substantial consolidation.
• There is still plenty of bloodletting to come.