Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Tesla ADAS Incident Reports

This site may earn commission on affiliate links.
I'm starting this thread to analyze incident reports that Tesla files with NHTSA regarding crashes involving cars that might be related to FSD or AP or NOA. Note that Tesla has to report any incidence where the crash happens even if AP/FSD/NOA was engaged within 30 seconds prior to the crash.

Here is the NHTSA site where you can see details about the report and download the data. The data is for all OEMs.


Data sheet, 2022 : https://static.nhtsa.gov/odi/ffdd/sgo-2021-01/SGO-2021-01_Incident_Reports_ADAS.csv

Data Definitions : https://static.nhtsa.gov/odi/ffdd/sgo-2021-01/SGO-2021-01_Data_Element_Definitions.pdf

Tesla withholds a large number of data in the fields as "[REDACTED, MAY CONTAIN CONFIDENTIAL BUSINESS INFORMATION]". Still there are a lot of useful fields for analysis. We can use these fields to try to figure out how many crashes are AP/NOA and how many are FSDb, which is my first objective.

For eg. below I've a pivot table by "Roadway Type". Yellow is clearly AP/NOA, Green is FSD and the other two could be mixed.

1674936307508.png


Here is the breakdown by posted speed limit. Again we can assume anything below 60 is FSDb (though there are edge cases).


1674936455065.png
 
Before people start comparing a smart Tesla car to other non-smart car mfgers:

Furthermore, Level 2 ADAS-equipped vehicles are generally privately owned; as a result, when a reportable crash does occur, manufacturers may not know of it unless contacted by the vehicle owner. These limitations are important to keep in mind when reviewing the summary incident report data.

Manufacturers of Level 2 ADAS-equipped vehicles with limited data recording and telemetry capabilities may only receive consumer reports of driving automation system involvement in a crash outcome, and there may be a time delay before the manufacturer is notified, if the manufacturer is notified at all. In general, timeliness of the General Order reporting is dependent on if and when the manufacturer becomes aware of the crash and not on when the crash occurs. Due to variation in data recording and telemetry capabilities, the summary incident report data should not be assumed to be statistically representative of all crashes.
 
For eg. below I've a pivot table by "Roadway Type". Yellow is clearly AP/NOA, Green is FSD and the other two could be mixed.

1674936307508.png
Any of these categories could be involved within 30 seconds of AP usage. The bulk of the Highway/Freeway accidents are almost certainly assignable to AP/NOA, but the others are all ambiguous.

You also cannot assume that anything below 60 mph is FSDb. Before I got FSDb, I drove many city miles on AP so that my safety score would stay high. I would say that many people without FSD use AP on city streets.
 
Any of these categories could be involved within 30 seconds of AP usage. The bulk of the Highway/Freeway accidents are almost certainly assignable to AP/NOA, but the others are all ambiguous.

You also cannot assume that anything below 60 mph is FSDb. Before I got FSDb, I drove many city miles on AP so that my safety score would stay high. I would say that many people without FSD use AP on city streets.
We can't obviously be 100% sure or correct about the classification since Tesla is not giving information. The idea is to estimate.

Have you ever worked with real-world noisy, incomplete and inaccurate data ?
 
Last edited:
  • Like
Reactions: EbS-P
There's a Parking Lot accident with a 30mph speed limit that was most definitely not FSD Beta as it was on a 2015 Model S. Unfortunately both the Tesla driver and passenger were killed when they crashed into the back of a parked tractor-trailer at 64mph.
 
There's a Parking Lot accident with a 30mph speed limit that was most definitely not FSD Beta as it was on a 2015 Model S. Unfortunately both the Tesla driver and passenger were killed when they crashed into the back of a parked tractor-trailer at 64mph.
Yes, I saw that. No idea how they managed to do that - and why AP/FSD was involved. I think they started AP/FSD and then instead of braking applied accelerator after disengaging ?
 
We can't obviously be 100% sure or correct about the classification since Tesla is not giving information. The idea is to estimate.

Have you ever worked with real-world noisy, incomplete and inaccurate data ?
I made a career out of working with noisy, incomplete and inaccurate data. I did a quick first approximation of how many of the 1100+ accidents might be the fault of FSDb and came up with only 10 accidents. But, my gut feel was that it was a high number based on the assumptions that I made. I did not use individual reports, merely allocated the accidents based on some simple ratios of things like numbers of FSD cars vs non-FSD cars, applying probability of the Tesla being at fault and probability of the accident being caused by whatever ADAS had been in use.

If you relish combing through 1100 reports to find those 10, all power to you. I'll look forward to reading your methods and results.
 
We can't obviously be 100% sure or correct about the classification since Tesla is not giving information. The idea is to estimate.

Have you ever worked with real-world noisy, incomplete and inaccurate data ?
Arbitrarily declaring that anything below 60 mph is FSD is a horrendous way to estimate. If you continue to use that method, I would conclude any analysis you publish to be irrelevant.

James Douma recently posted a Twitter thread about the difficulty of making heads or tails of crash data beyond top-level metrics. Any analysis worthwhile would have to demonstrate why he is wrong.


Here it is consolidated:

@1LoafOfMeat Graph is their public data with averaging applied. I can't rule out them lying. I have put significant effort into trying to find a better way that they could release the data to inform users without making the confusion worse - and I couldn't come up with one. /1
@1LoafOfMeat After pulling the NHTSA crash database and breaking it down many different ways I was unable to find a simple, clear, and accurate way to convey crash statistics. Every approach I came up with had the potential to be gamed by cherry picking thresholds and definitions. /2
@1LoafOfMeat There are dozes of road types and many categories of accidents with a panoply of causes and confounding factors. And there aren't all that many accidents so you can't get statistically significant for any narrow category. /3
@1LoafOfMeat On top of this (for NHTSA data) reporting is done locally and varies by jurisdiction - and quality can be poor so there's subjectivity in deciding what to throw out. A lot of it is clearly erroneous. The NHTSA's own summaries are quite nonspecific for this reason. /4

@1LoafOfMeat Just trying to find incident rates for highway versus city driving is maddeningly and depends on subjective decisions. Any number you present can validly be argued away. The only simple and true statements are about broad categories with tight definitions. /5
@1LoafOfMeat Which is one reason fatalities feature so prominently in statistics. Death is well defined but injuries less so, accident even less so. Incident, crash, damage - all have subjective thresholds of many types. /6
@1LoafOfMeat And definitions are critical because adding or removing just a few 'incidents' to the list can change the 'statistics' a lot. NHTSA has hundreds of pages of categorization rules and they still aren't enough. /7
@1LoafOfMeat In the end broad and simple, if perhaps vague, is probably the best we can do for public facing information. Why trust it? Well for one thing NHTSA gets access to the raw data so lying in a big way would be 1) hard to get away with and 2) extremely damaging when found out. /end
 
I took a look at the current report data. Right off the bat, I noticed that there are up to three reports for each accident. There can by a 1-day, 10-day Update and Monthly Update as well as a column for the report number (1, 2 or 3). If you filter on report number 1, you should get all unique vehicle accidents. This reduces the total number of accidents to only 588. So, my earlier estimate of 10 FSDb accidents should be reduced to only 5.
 
  • Informative
Reactions: EVNow
I made a career out of working with noisy, incomplete and inaccurate data. I did a quick first approximation of how many of the 1100+ accidents might be the fault of FSDb and came up with only 10 accidents. But, my gut feel was that it was a high number based on the assumptions that I made. I did not use individual reports, merely allocated the accidents based on some simple ratios of things like numbers of FSD cars vs non-FSD cars, applying probability of the Tesla being at fault and probability of the accident being caused by whatever ADAS had been in use.

If you relish combing through 1100 reports to find those 10, all power to you. I'll look forward to reading your methods and results.
Arbitrarily declaring that anything below 60 mph is FSD is a horrendous way to estimate.
Instead of this - what I'm actually trying to do is to combine multiple fields to figure out the likelihood of an accident being FSD. May be make a ML model of it to get better numbers.
 
Instead of this - what I'm actually trying to do is to combine multiple fields to figure out the likelihood of an accident being FSD.
But there likely isn't enough data there to even begin to make that determination.

For example, TACC is considered ADAS is it not? Lots of people use TACC on city streets below 60 MPH. Then there is the fact that only a portion of the fleet has paid for FSD, and even fewer have installed/activated FSDb. And of course the ADAS being active within 30 seconds sort of invalidates the whole thing. 30 seconds is long time, you could have been on AP on the freeway, disengaed on the offramp and been into a parking lot within 30 seconds.
 
Instead of this - what I'm actually trying to do is to combine multiple fields to figure out the likelihood of an accident being FSD. May be make a ML model of it to get better numbers.
How do you train an ML model without some examples with known results? With only 588 examples, it's unlikely that you can get a very good model.

And I am curious which fields you plan to use to separate AP from FSD usage on non-highway accidents. From what I can tell, there is very little useful data aside from which parts of the car were hit, the speed limit and the impact speed. Nothing specifies whether the ADAS system was engaged at the time of the impact (or how long prior). All of the useful data has been redacted

Perhaps you could provide one or two examples of accidents that you believe involved FSDb and why they are not likely to be AP or manual driving?
 
  • Like
Reactions: MP3Mike
What's blowing my mind in these data is the amount of redacted entries for Tesla and it's particularly striking compared to the other entities, almost everything of value is redacted in Tesla's reports. Zero narrative, zero ODD, no address or zip code. I'm very accustomed to working with large nebulous data sets but I don't know what value can be gleaned from this.

Really unfortunate tbh, and one can only imagine why this is the case
 
  • Like
Reactions: 2101Guy
Agree with the others - this is potentially very interesting and valuable but the data is so limited and dirty that it’s impossible to draw any conclusions from it.

Take the infamous SF tunnel crash - we don’t even know for sure if an ADAS system was being used, other than the driver’s statement. Assuming it was we dont’ know if it was NAP or FSD. We don’t know if the driver disengaged the system, pressed the brake by mistake, etc.

A few years ago there was another crash for which people quickly blamed Tesla/auto pilot. Some weeks later it came out that autopilot wasn’t being used - it was simply a reckless driver that happened to be driving a Tesla.

Tesla undoubtedly has telemetry data for the majority of these crashes but without that data it’s impossible to draw any conclusions.
 
  • Like
Reactions: MP3Mike