Autonomous Autos – Dark horses.

Reply to this post

Waymo’s lead shrinks to 5 to 1.

• Analysis of autonomous testing data submitted to the California DMV for 2017, reveals that Waymo is still out front, but GM Cruise has improved massively with Nissan holding onto its No.3 position.
• The best measure of an autonomous driving solution remains how often the driver has to take over to correct shortcomings in the autonomous driving software.
• Regulations in California require those that test in the state to submit this data but typically, they all submit it in different ways (see here).
• There are also different types of disengagement such as when the car is going to hit something (critical) or when the safety driver feels uncomfortable (ordinary).
• Furthermore, companies test their cars in different conditions meaning that the data can really only be used as indication.
• However, the contrast between the have’s and have-nots is so stark that meaningful conclusions can still be drawn by parsing the data submitted.
• In order of performance the data for 2017 shows:
  • No. 1 Waymo (Google) is now only 5x better than its nearest rival compared to 8x in 2016.
  • However, Waymo’s performance has only improved marginally with 5,596 miles per disengagement in 2017 compared to 5,128 in 2016.
  • Waymo has still driven more miles than anyone else but its lead in miles driven has dropped from 155x its next rival to 2.8x.
  • Curiously, Waymo drove 45% fewer miles in 2017 (353K) compared to 2016 (636K).
  • No. 2 GM Cruise comes from nowhere to comfortably take the no.2 position with 125K miles driven with 105 disengagements (1190 miles / disengagement).
  • This is 5x more than Waymo, but GM Cruise drove all of its miles in downtown San Francisco which it argues is the most difficult environment within which to operate an autonomous vehicle.
  • I would argue that it is certainly the most complex but also the slowest, giving the computer much more thinking time than it has on suburban street or a highway.
  • Either way, this is an impressive performance and for the first time, there is a credible challenge to Waymo’s dominance.
  • No. 3 Nissan which is the other dark horse in this race with 5007 miles driven and 24 disengagements (209 miles per disengagement).
  • Last place Mercedes which saw a meaningful deterioration in its performance during 2017.
  • In 2017 Mercedes had nearly 1 disengagement for every mile driven compared to 2 per mile in 2016.
  • It is possible that Mercedes decided to test in more challenging conditions in 2017 which caused the deterioration but regardless, it is the laggard in the race for an autonomous car driving solution.
• Those that drove the most miles (Waymo and GM Cruise) still had the best performance, again underlining that the key to artificial intelligence (the heart of all autonomous driving systems) remains the amount data collected (see here).
• Uber and Tesla have no data for 2017 as they did not test in California but given the distractions that Uber has suffered in 2017 combined with the ongoing trade secrets trial against Waymo, is likely to have meant that it has not improved much on its dreadful performance in 2016 (see here).
• I am certain that Waymo is the best because it began working on autonomous driving in 2009 (far earlier than anyone else) and in the last 2 years has also driven substantially more miles than anyone else.
• I continue to believe that there is not much point in rushing to get an autonomous driving solution to market as I still do not expect the market to be ready for autonomous vehicles much before 2028 (see here)
• Consequently, those rushing to market may find that they have a solution but no deployments.
• This could easily result in a number of viable solutions being available once the market is ready, making sitting on the sidelines for now a wise choice.
• I suspect that this is why Google and Waymo are currently pushing to do a number of large long-term deals for technology as their lead over the rest of the market has already passed its peak.