Tesla – Fact or fantasy

Tesla’s bold claims do not hold up to scrutiny.

• Elon Musk has made some pretty bold claims on Tesla’s behalf regarding the state of its autonomous driving solution but with no official hard data being offered and its record, it is hard to take the claim seriously.
• Tesla’s CEO, Elon Musk, has given an interview to an MIT researcher (see here) where he claims “game, set and match” for Tesla in the race to reach autonomy and “I could be wrong, but it appears to be the case that Tesla is vastly ahead of everyone.”
• I suspect that he is wrong as there is some unofficial data available (see below).
• RFM research has found that miles driven is an indicator of excellence in autonomous driving but in no way is it a given.
• For example, the best two players Waymo and Cruise (see here) have driven more than anyone else but at the same time, Apple drove a lot of miles and was only slightly better than Uber, which was worst of all.
• On miles driven, it would not be difficult to believe that Tesla has gone further than anyone else as its autopilot software is in thousands of cars and is used to drive thousands of miles.
• Drivers have to keep their hands on the wheel at all times, but this could all be valuable experience from which the autopilot system can learn.
• However, what is clear is that just like everyone else, the Tesla system cannot handle infinite and unstable environments and has already chalked up two fatalities and a hacking incident to that effect.
• Furthermore, Tesla has published no data since 2017 meaning that very little credibility can be attached to its claims without some hard numbers.
• A study by MIT (see here) has provided some data which puts Tesla towards the bottom of the pack.
• In this study, Tesla’s drove 112,427 miles in autopilot mode and suffered 18,800 disengagements or about 1 every 6 miles which is 1,846x more frequent than Waymo.
• This puts Tesla way below the leaders in 20th position which, ironically, is not that far from where it was when it last published data in 2017.
• In the last 6 months, most autonomous driving players have accepted that the problem is more complex than originally anticipated and have brought expectations of commerciality back to “some time in the next 10 years”.
• This complexity is due to the fact that the deep learning models have no causal understanding of what it is that they are doing and so they tend to fail catastrophically when something unexpected happens.
• I think that it is Musk’s belief that Tesla will soon solve this problem that is driving his opinion about Tesla’s lead in the race.
• The evidence from the industry and from academia suggests that he will be disappointed.
• However, the solution to the problem may involve a combination of traditional rules based software and deep learning models but changing the architecture will take time.
• This is why everyone is pushing out the timeline and if Tesla wants users to be able to take their hands off the wheel, it will have to do the same.
• Based on the data available (which is not great), RFM ranks Waymo as No.1, Cruise as No.2 and Zoux as No.3 and puts Tesla in 20th out of 30.