We wrote an article when Telsa first released the data on accident rates with autopilot that essentially highlighted that the statistics are not comparing like for like. That article is below and the key points of that article still hold true today.
As more results get published we can however start to see more insight into the data, including trend data, which highlights a number of other points which reinforce the view that these things are not like for like comparisons.
Let us first look at the NHTSAs statistics. Over the last year this has fallen from 498k miles to 479k, the worst quarters effectively being winter, but overall, there is fairly low variability in the numbers. If these figures are correct and not averages over a longer time periods then there seems to be little (under 4%) seasonal effect on accident rates.
The second data point to look at is the no AP features including passive. The data has trended
The next set of numbers are the passive AP figures. These have essentially got worse over the last 3 quarters as follows:
One hypothesis could be that passive safety systems are more susceptible to weather conditions than normal drivers (i.e. passive systems do not take into consideration the weather). Alternatively, it could be that with the influx of Model 3 owners over the last 2 years, the demographic of the average Tesla driver has changed. This was one of the points we raised as a variable between the all miles driven and the Tesla figures with the assumption the majority of Tesla drivers were older and generally professional in occupation. The model 3 with its lower price point has opened that up to a wider audience. Another potential variable is the buyer attitude. As the buyer type changes from Innovator and early adopter to the early majority, the expectation from these new buyers is that the cars will perform safely and reliably and not acknowledge the Beta warnings. This would in turn lead to greater risk taking. Finally, drivers may be increasingly using Active AP resulting in the results being skewed as the mix of road type is becoming higher risk. Put differently, if active AP is used increasingly more on the relatively safe freeways, then these 'safe' miles will be excluded from the passive AP results which will in turn make these figures worse. It's difficult to conclude on the true cause, it may well be a mixture of the above points, however the variability in passive safety results is only marginally better than the difference between the passive safety system and no safety system, albeit in each quarter the passive safety systems have been at least 30% better. Given the cause of the variability cannot be explained, it's hard to conclude the software is actually still performing well as the figures getting apparently worse, although we would be surprised if it was now worse.
The final set of numbers is the Active AP. While Q1 figures show a dramatic improvement over Q4, Q4 was a relatively poor quarter. Comparing the last quarters results with the 2 quarters before shows only modest gains.:
Our previous comments including driver demographics and Active AP being used on the safest roads (another reference is US Department for transportation road safety which shows interstate roads have approx half the fatality rate compared to all roads) are all relevant factors when making meaningful comparisons.
A lot of numbers have been discussed so what is our TL;DR? Firstly, the points we previously made about active AP being used on the safest roads, at the safest time, in the safest weather and by the safest drivers are all still true. No new evidence has been provided that help address these variables. The trend that 3 quarters of falling passive safety is a concern. While passive safety undoubtedly works the falling performance with no explanation leaves no room for complacency and we believe is a result of the shifting driver demographic as the cheaper M3 is now available to an younger and less affluent market. Finally, active Autopilot performance fluctuates between quarters and while we appreciate the raw numbers are better, we are not satisfied that this can be attributed to the software over the external conditions when in use.
The TL:DR on our TL;DR is that drivers intuitively use active AP when they themselves recognise that the conditions are very safe which probably explains the numbers, and if active AP was genuinely the safest way to drive, why would we any Tesla owner ever chose to disengage it? We simply don't, so we know active AP is not universally safer.
Tesla's Vehicle Safety Report stated some interesting, and we use the word lightly, facts.
Let's break this down a bit.
We can add in some additional data points. Tesla's are expensive cars and it's also not unreasonable to assume that the demographic of the typical driver is someone in their 30s and above. These are by far the safest class of drivers per mile with accident rates up to 4x safer than the worst age demographic according to AAAFoundation. This is relevant to the overall stats against the field and not AP engaged v not.
There's also an article on general safety features available on modern cars, some of these features only becoming more common place in recent years such as lane assist and so rare on many older cars. We saw material safety improvements when seatbelts became standard and general reduction in accident rates due to ABS and stability aids, we're now seeing a similar increase from these new features. Forbes top 7 car safety features goes into a little more detail.
Technology improving safety is far from new, the insurance industry for highway safety found a 40% reduction in single vehicle accident rates from the introduction of stability control, technology which some cars still don't have and are part of the baseline statistics. What it this suggests is any modern car with passive safety systems have a marked safety improvement of those that don't, and we don't have the data points for a comparable statistic. Tesla could produce statistics that compare pre AP hardware cars, the Mobileye system and their own systems if they wanted to, although in fairness the mileage rates may be small in the earlier models to be statistically meaningful.
We do not fully subscribe to any argument on the relative safety of Tesla, especially on Autopilot, as the data available is far too weak and any number of assumptions can be made. Through the addition of a small amount of additional data to add context, the picture becomes very different and the argument can be skewed to any political agenda you wish to portray.
It would be beneficial if Tesla put more information out there and allowed some impartial and detailed analysis. They should be mindful that when putting out incomplete data sets that give a superficial view on safety it can do more harm than good and potentially give a false sense of security. We have seen too many instances of people abusing Autopilot, and in a number of occasions with fatal consequences. The suggestion that Autopilot is twice as safe as without must not be taken as an excuse to rely on Autopilot.
But let's take the positives, firstly Tesla are one of the first companies to publish any data like this that we are aware of, that's to be commended. Secondly, the cars appear to be able to cover many more miles between accidents than the average, whether it's the passive safety systems that are also on other similar priced and aged cars or driver demographic we don't know, but if you're in a Tesla, you're safer than average. And thirdly, Tesla like several other car manufacturers, are continuing to innovate and push the bar higher, and while we could be sceptical on the current metrics, merits and safety of the systems, innovation and change is how improvements will come about. For that we should all be grateful. Just don't abuse it.
We are Tesla owners and in general supporters. We don't hold stock in Tesla, nor do we short Tesla stock. Our views are our own, based on data points and research that we conduct ourselves. Our goal is to take an impartial view as far too many organisations are either extremely pro or anti Tesla.