Last updated 15-Jan-2022
We have updated this article a number of times over the years and this update follows the publication of the Q4 2021 figures.
Tesla have also decided to drop the radar from the Model 3 and Model Y in the US although the new Model S is quoted as having it, as are cars made in China. Tesla have also hinted that the Cybertruck will have FSD HW4 hardware although we know no details of what this may be.
The August AI day exposed a number of issues that Tesla are working to overcome on the limited beta release that has a limited roll out to high scoring (i.e. safe drivers in safe areas) owners to bed in the new features and capabilities although YouTube videos are circulating showing a generally impressive feature set albeit with a lot of significant edge case fails. Tesla are building a new super computer called Dojo which is expected to help with the training process although this is still under development.
This all points to a lot of change on a product that Tesla claimed was nearly ready for full release a few years ago.
When we look at the numbers, what they show is Tesla have plateau'd in performance with respect to safety and they need a step change in capabilities. The last year has shown some improvement in the safety of the cars while operating under autopilot but this is relatively limited and lighter traffic volumes due to Covid lock downs in many countries may have influenced the results.
Passive safety systems in Tesla were no better, in fact some quarters had resulted in worse figures than the equivalent quarter the year before. We find it surprising that Tesla have now chosen to omit the Tesla Passive Safety statistics figure and we can only guess to the reasons why. One explanation may be because the figure is not as favourable as Tesla would like, another is the transition to vision only safety systems has caused a blip in the data.
We have 3 issues with the numbers as presented. The first is what the numbers actually represent and whether they are statistically valid. The second is they are presented in such a way that we can easily jump to conclusions that the cars are significantly safer than not driving a Tesla, but there are a number of external stats that we can bring into the mix that explains away a lot of the benefit. And finally, their is concern that the figures are not improving despite all the learning that is said to be happening. We examine these points below.
The numbers presented are a fairly crude "miles per accident". What we don't have is any context behind them other than "cars with autopilot activsted". Some example questions we don't know the details of which can all have a bearing on the way the results are interpreted and whether a statistical assessment can even be made:
A mix of these fundementals may mean that there is no point even examining the figures furthyer because there is no veracity in the data presented to examine further. For the sake of argument, the following points assume they're sufficiently valid.
Comparing all cars to Tesla cars can be adjusted in a couple of ways. We can make some reasonable assumptions about the average age of a Tesla car, the demographics of a Tesla owner and even the roads where Autopilot is likely to be used.
Tesla cars are not cheap and are likely to be bought by a mixture of elder drivers or younger drivers in a professional occupation. There will of course be owners outside these demographics but for statistics, it's where the majority will be that we care about. The NHTSA statistcs will of course cover all drivers as well as all cars.
On age alone. it is known that drivers between 35 and 60 are the lowest risk group and which was almost certainly the primary demographic. The AAAFoundation data show that high 30's and above are 4x safer than the average. If this correlates witht he typical Tesla demographic, then this stats alone suggests a Tesla would record an accident every 2 million miles v the half million for all cars. Tesla accidents are every 1.59 Million in the 2021 Q4 figures. It would not be hard to conclude that the predominent reason a Tesla has fewer accidents is because the type of driver who would buy a Tesla is in a demographic that has fewer accidents irrespective of the car they drive.
Active autopilot is used predominantly on Motorways/Freeways as this is the road type Tesla advise it is only suitable for and which was initially restricted to. While it can now be used on other road types, we believe the volume of miles driven under active Autopilot will be on these type of road. This road type is known to be statistically safer than the average road.
UK figures from 2017 state that there were 99 fatalities in 69 Billion miles on motorways v 1068 in 145 Billion rural roads and 626 in 117 Billion urban miles. A reference to the above data can be found in this UK Government report.
Another reference is US Department for transportation road safety which shows interstate roads have approx half the fatality rate compared to all roads) are all relevant factors when making meaningful comparisons.
Tesla compares accident (or crash) rates not fatality or injury rates, but logic dictates that on these faster roads an accident is more likely to result in an injury accidents than slower roads because the speeds and therefore forces are greater. While exact numbers may be difficult to determine, the respective ratios of accident rates resulting in injury can be inferred.
The data therefore suggests that just based on the road type where Autopilot is typically engaged, you would expect to see a 2-4 fold increase in miles between accidents. The data shows it is a little under a 3x increase compared to Tesla without autopilot.
The examples above can pretty much explain both improvements in Tesla safety improvements just based on the type of driver and the type of road. But is active autopilot worse than that? We also know that drivers tend to use autopilot when the conditions allow and indeed autopilot will not engage if the conditions do not allow. Low sun can obscure cameras, fog and heavy mist can prevent activation, heavy rain, snow and ice may cause owners to not want to engage autopilot, and all these factors increase the likihood of an accident. In other words, autopilot is not used in the most dangerous weather situations thereby artifically increasing its apparant performance.
Similar can be said for traffic conditions and junctions. While EAP and FSD allows the car to take certain junctions, roundabouts, side junctions, and changing lane in very busy traffic situations are all performed by the driver and offer the most risk.
What we're suggesting is that active autopilot is typically (accepting some drivers will use the system in other ways) reserved for driving on the safest roads in the safest conditions so it is only natural that the accident rates should be much better. The question though, is a near 3x improvement good enough given the data presented?
It is certainly not true to say the Tesla is the cause of the safer results, to do so would be an invalid conclusion.
Let us first look at the NHTSAs statistics. While there is some fluctuation, the change year on year is relatively small with little seasonal adjustment.
If we look at the trend of accidents involving Teslas with no AP features including no passive safety we see fairly limited variability over time. There is some seasonaility in the data, but 2021Q3 is slightly worse than the equivalent 2020 figure, whereas 2021Q4 is slightly better. We don't believe there is any material change going on.
Tesla have decided to stop reporting passive safety performance which we find as being very perculiar and potentially worrying.
Active autopilot has seen an improvement since 2019 however comparing 2020 to 2021 shows a mixed picture with some quarters better and other quarters worse than the equivalent quarter the year before. Without much more data it is hard to deduce any conclusions. There is arguably a very small improvement, could this be the increased number of cars with HW3 computers? Could this be because the nag system has tightened up on abuse? We simply don't know, but over the last 2 years we don't believe the figures suggest any significant increase in performance.
We can not subscribe to any argument that a Tesla car, in itself, is safer. Any improvements can relatively easily be explained by the driver demographic or the type of road and its associated conditions.
It would be beneficial if Tesla put more information out there and allowed some impartial and detailed analysis. They should also be mindful that when putting out incomplete data sets, and now even less data than they did, that it can give a false view on safety. We have seen too many instances of people abusing Autopilot, and in a number of occasions with fatal consequences. The suggestion that active Autopilot is twice as safe must not be taken as an excuse to rely on Autopilot. The data really suggests it is not safer and arguably less safe.
But let's take a the positive, Tesla are one of the first companies to publish any data like this that we are aware of, that's to be commended, we just wish it meant something useful.