Why Autopilot Safety Claims are BS

[clydeiii]

Why do people think Teslas are safer than other cars?

To me it appears Teslas have less safety features making them inherently more dangerous, when it comes to accident avoidance.

The reason why Tesla doesn't compare itself to other similar cars is probably that it's worse.

Examples:

- No rear cross-traffic detection
- No blind-spot warning in mirror
- Lane-keeping assist was added fairly recently as far as I remember
- No advanced features such as night vision
- No driver monitoring or drowsiness monitoring system
- Phantom braking when using ACC persists (never had any other car where I have to keep my foot by the accelerator pedal when on cruise control. Dangerous.)
- The efficacy of the AEB at different speeds is still questionable

When it comes to active safety features, I can't really think of any that Tesla has that other cars don't, so why would it be any safer?

We probably think they are safer than other cars because we compare them to our previous cars. I compare my Model 3 to my old 2009 Mazda 3 and it's way, way safer. That's all that matters for me.

 

[PungoteagDave]

I found some interesting data from the EU. It is in this report: https://ec.europa.eu/transport/road_safety/sites/roadsafety/files/pdf/scoreboard_2020.pdf
It seems that the only statistic is counted by the governments in europe is only accidents with person injury or a fatality.

As one can see, the fatalities at highways are probably around only 9% of the total only. But fatality numbers also count accident where cars are not involved like bicycling and pedestrians. Probably those numbers are low, so we can assume 9% is a decent estimate for car only fatalities.

Then another assumption could be that there is a fixed ratio of fatalities pr accident, independent of location. Then Tesla's reference should subract 90% of the accidents.

But this is probably not correct either, becuase it probably is a much higher rate of accidents in urban areas, while parking, and suburban than on highways.

Look also at the difference in fatalities pr million inhabitant per country. Does it vary this much in the US?

But conclusion is still, the claim from Tesla is just silly and reduces the reputation and trust in the company.

View attachment 655966

Thanks for that - EXACTLY my point - Tesla and Elon are comparing apples and oranges - claiming safety for a thing that does not generally even exist statistically - 90+% of accidents happen in places where FSD (current iteration) and AP are not even intended to be used. He insists that the general data that includes those accidents which would never occur with AP/FSD turned on in ANY case, apply against the generalized auto driving population when compared to Teslas that have FSD/AP turned on, which only occurs in places where only 9% of accidents occur in the first place. Sheesh. Figures don't lie but liars figure. Or as Mark Twain put it, "Lies, damned lies, and statistics."

 

[gearchruncher]

I am curious what the national accident rate is on highways.

The DOT publishes all of this. TL;DR:

Rural Interstate: 0.79 per 100M miles
Rural Arterial: 2.42 per 100M (3x worse)
Urban Interstate: 0.48
Urban Arterial: 1.34 (2.8 worse)

So the Interstate is 3X safer per mile than surface roads on average. More miles are travelled on interstates as well. This is for fatalities. Tesla statistics are for an accident that caused airbag deployment which could cause large changes in the Interstate/Arterial ratios too.

 

[gearchruncher]

I compare my Model 3 to my old 2009 Mazda 3 and it's way, way safer.

How many accidents per mile did you have in your Mazda 3 that set off the airbags and how many have you had in your Tesla?

 

[scottiev]

Per their disclosure, it's 5 seconds. If an accident happens 5 seconds after AP disengages, it's not counted. So if you suddenly come onto a stretch of highway that is iced over, and disengage AP, you're still at a massively higher risk of a collision because the ice doesn't just dissapear after 5 seconds. Same with other weather conditions and driving conditions, including being surrounded by trucks or shitty drivers.

5 seconds is an eternity in a crash situation. anything that happens after that is on you.

 

[gearchruncher]

5 seconds is an eternity in a crash situation. anything that happens after that is on you.

The time to stop from 60 MPH is 5.4-6.4 seconds on dry roads with good tires. Add another second at 75 MPH.
So, if AP fails to see something and you brake manually and hard, you can still easily hit something 5+ seconds later.

 

[gearchruncher]

How is there any argument being had that the drivers in Teslas are already safer as a baseline? The Tesla stats say this:

In the 1st quarter, we registered one accident for every 4.19 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.05 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 978 thousand miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 484,000 miles.

Tesla themselves says their drivers are 2X as safe as the average driver, and that the non-AP safety features add another doubling.
The fact that AP is only 2X as safe as an average Tesla not on AP actually makes AP look more dangerous, given the average crash rate on the highway is 1/3 that of on surface streets.

 

[Redbrick]

I more or less agree with cherry picking data, but Tesla does count crashes that occur just after autopilot was disengaged in their stats. So if someone sees an imminent accident and disengages autopilot and then crashes, that will still count as an accident with autopilot engaged.

Serious question....how do you know this? This seems like inside info....

This is my natural tendency to take over as I don't trust autopilot/NOA too. I've had too many close calls.

 

[gearchruncher]

Serious question....how do you know this? This seems like inside info....

It's on the Tesla Statistics Page in their methodology:

We collect the exact amount of miles traveled by each vehicle with Autopilot active or in manual driving, and do so without identifying specific vehicles to protect privacy. We also receive a crash alert anytime there is a crash that is correlated to the exact vehicle state at the time. This is not from a sampled data set, but rather this is exact summations. To ensure our statistics are conservative, we count any crash in which Autopilot was deactivated within 5 seconds before a crash, and we count all crashes in which the crash alert indicated an airbag or other active restraint deployed. In practice, this correlates to nearly any crash at about 12 mph (20 kph) or above, depending on the crash forces generated. On the other hand, police-reported crashes from government databases are notoriously under-reported, by some estimates as much as 50%, in large part because most fender benders are not investigated. We also do not differentiate based on the type of crash or fault, and in fact, more than 35% of all Autopilot crashes occur when the Tesla vehicle is rear-ended by another vehicle. In this way, we are confident that the statistics we share unquestionably show the benefits of Autopilot.

 

[qdeathstar]

The time to stop from 60 MPH is 5.4-6.4 seconds on dry roads with good tires. Add another second at 75 MPH.
So, if AP fails to see something and you brake manually and hard, you can still easily hit something 5+ seconds later.

You can also use that thing in front of your hands called a steering wheel.

 

[AlanSubie4Life]

The time to stop from 60 MPH is 5.4-6.4 seconds on dry roads with good tires. Add another second at 75 MPH.
So, if AP fails to see something and you brake manually and hard, you can still easily hit something 5+ seconds later.

Totally off topic (and this correction should not be taken as a dispute over the original point about 5 seconds and whether or not that is a good metric for Tesla to use - I tend to think they should make that 10 seconds and see how much worse it makes their data!), but this is bad information (or dated). With my PS4S the braking time is ~2.6 seconds from 60mph. Include reaction time and it's 3.6 seconds. With the MXM4 tires, they can stop from 60mph in under 3.1 seconds.

Here is instrumented data (see further up for the MXM4 data): POLL: What will my 60-0 stopping distance be with Pilot Sport 4S 265/40R18s be?

So total stopping time for someone paying attention from 60mph is going to be below 4.5 seconds and could easily be below 4 seconds in a properly equipped vehicle.

 

[Cheburashka]

How is there any argument being had that the drivers in Teslas are already safer as a baseline? The Tesla stats say this:


Tesla themselves says their drivers are 2X as safe as the average driver, and that the non-AP safety features add another doubling.
The fact that AP is only 2X as safe as an average Tesla not on AP actually makes AP look more dangerous, given the average crash rate on the highway is 1/3 that of on surface streets.

Good catch. Would be interesting to see Tesla break down where the accidents happen between highway and streets.

 

[ProtonSF]

5 seconds is an eternity in a crash situation. anything that happens after that is on you.

Wrong. If you come onto a stretch of super icy road, or it starts raining cats and dogs, most drivers will disengage AP, and the accident could very well happen after 5 seconds.

 

[Barklikeadog]

We probably think they are safer than other cars because we compare them to our previous cars. I compare my Model 3 to my old 2009 Mazda 3 and it's way, way safer. That's all that matters for me.

Funny thing is, that 2009 Mazda IS the 12 year old 'average car' on the road that elon puts his safety numbers up against.

 

[Dan D.]

Here's some scenarios that Elon may not count as an Autopilot accident assuming no airbags go off nor other restraint systems, based on the vague wording of their methodology:

• Two cars sideswipe causing extensive cosmetic damage but no airbags go off
• Rear ender damage, front bumper damage from bumping together
• Car drives through neighbors garden destroying their flimsy decorative picket fence and all the gnomes
• Car serves off the undivided highway, across the double yellow lines, through oncoming traffic, into a cross street, clips its mirror on a sign and slows down without crashing
• Car fails to notice or react to animals/pets in the road flattening them
• Car drives into the water at a boat launch ramp
• Car drives into some pedestrians but none are injured and the incident is not reported
• Car t-bones another car at walking speed
• Autopilot is disengaged and car crashes >5 seconds later
• Car fails to transmit its data to Tesla so they don't count that crash

 

[scottiev]

Wrong. If you come onto a stretch of super icy road, or it starts raining cats and dogs, most drivers will disengage AP, and the accident could very well happen after 5 seconds.

Why would that be AP's fault then, if you disengaged? How did AP cause the crash?

 

[scottiev]

The time to stop from 60 MPH is 5.4-6.4 seconds on dry roads with good tires. Add another second at 75 MPH.
So, if AP fails to see something and you brake manually and hard, you can still easily hit something 5+ seconds later.

Where are you getting those numbers? 60mph is 88fps. If an average driver can decelerate at 20fpsps, 88/20 = 4.4 seconds. Teslas are capable of stopping even faster.

 

[GZDongles]

Here's some scenarios that Elon may not count as an Autopilot accident assuming no airbags go off nor other restraint systems, based on the vague wording of their methodology:

• Two cars sideswipe causing extensive cosmetic damage but no airbags go off
• Rear ender damage, front bumper damage from bumping together
• Car drives through neighbors garden destroying their flimsy decorative picket fence and all the gnomes
• Car serves off the undivided highway, across the double yellow lines, through oncoming traffic, into a cross street, clips its mirror on a sign and slows down without crashing
• Car fails to notice or react to animals/pets in the road flattening them
• Car drives into the water at a boat launch ramp
• Car drives into some pedestrians but none are injured and the incident is not reported
• Car t-bones another car at walking speed
• Autopilot is disengaged and car crashes >5 seconds later
• Car fails to transmit its data to Tesla so they don't count that crash

To be fair, most of these scenarios apply to the non-autopilot comparative data set too! Underreported and mis-reported accidents impact both sides of data for sure. Rather than the absolute number of accidents or the number of accidents per mile, the key question to me is: what is a baseline accident rate with no passive or active safety systems (e.g. emergency braking, lane departure warning, blind spot warning), what is the accident rate under the same conditions with those systems working, and what is the accident rate with those systems working plus TACC or autosteer. I'd imagine that the jump in safety or reduction in accident rate from a car without those safety systems active to a car with those safety systems is substantial, but there is a much smaller increase in safety going from that to a car with autosteer.

However, regardless of the magnitude of the increase, I doubt there is anyone with a serious argument that autosteer and TACC make driving less safe. I'm not sure if these accident rates tell us anything meaningful about a L3 or higher system though.

 

[ProtonSF]

Why would that be AP's fault then, if you disengaged? How did AP cause the crash?

You're completely missing the point. I'm not saying it's AP's fault. I'm saying that if people disengage AP every time they are in precarious situations, then that majorly taints the data, because you have way fewer accidents being recorded as due to AP. So when Elon/Tesla compares AP data to NHTSA data as a whole, it looks like AP is the most amazing safe auto-driver in the world, but in reality it may only be slightly safer than a human driver considering how little it can do to maneuver complex situations, like let's say two trucks merging into the same lane on either side of you.

I'm not trying to be an a-hole when I say this, but it honesty shocks me at how little awareness the general public seems to have about statistics. I'm truly having a hard time processing it.

 

[Dan D.]

Wrong. If you come onto a stretch of super icy road, or it starts raining cats and dogs, most drivers will disengage AP, and the accident could very well happen after 5 seconds.

Why would that be AP's fault then, if you disengaged? How did AP cause the crash?

You're completely missing the point. I'm not saying it's AP's fault. I'm saying that if people disengage AP every time they are in precarious situations, then that majorly taints the data, because you have way fewer accidents being recorded as due to AP. So when Elon/Tesla compares AP data to NHTSA data as a whole, it looks like AP is the most amazing safe auto-driver in the world, but in reality it may only be slightly safer than a human driver considering how little it can do to maneuver complex situations, like let's say two trucks merging into the same lane on either side of you.

I'm not trying to be an a-hole when I say this, but it honesty shocks me at how little awareness the general public seems to have about statistics. I'm truly having a hard time processing it.

Let's see if I can clarify your point:

• Driver A is running Autopilot for 1000 miles. Every time he sees a risky situation (that has nothing to do with Autopilot's behavior) he disengages Autopilot and handles the situation himself (total 10 miles). He never crashes
• Driver B is running Autopilot for 1000 miles. Every time he sees a risky situation (that has nothing to do with Autopilot's behavior) he disengages Autopilot and handles the situation himself (total 10 miles). He crashes once
• Driver C is running manually for 1000 miles. He never crashes
• Driver D is running manually for 1000 miles. He crashes once
• Driver E is running Autopilot for 1000 miles. AP never crashes
• Driver F is running Autopilot for 1000 miles. Driver does nothing in crash-imminent situation. AP crashes once
• Driver G is running Autopilot for 1000 miles. One crash-imminent situation occurs due to AP. He disengages 2 seconds before the crash. Crash occurs
• Driver H is running Autopilot for 1000 miles. One crash-imminent situation occurs due to AP. He disengages 6 seconds before the crash. Crash occurs, enough criteria are met that Tesla doesn't count it as AP caused

Result:
Driver A: 1000 miles on Autopilot. 10 miles non-Autopilot. Zero crashes AP. Zero crashes non-AP
Driver B: 1000 miles on Autopilot. 10 miles non-Autopilot. Zero crashes AP. One crash non-AP
Driver C: 0 miles on Autopilot. 1000 miles non-Autopilot. N/A crashes AP. Zero crashes non-AP
Driver D: 0 miles on Autopilot. 1000 miles non-Autopilot. N/A crashes AP. One crash non-AP
Driver E: 1000 miles on Autopilot. 0 miles non-Autopilot. Zero crashes AP. N/A crashes non-AP
Driver F: 1000 miles on Autopilot. 0 miles non-Autopilot. One crash AP. N/A crashes non-AP
Driver G: 999.98 miles on Autopilot. 0.02 miles non-Autopilot. One crash AP. Zero crashes non-AP
Driver H: 999.95 miles on Autopilot. 0.05 miles non-Autopilot. Zero crashes AP. One crash non-AP

Analysis:
Driver A and B are skewing the data. They are letting AP run during safe times but taking over during risky times. They are not letting AP take responsibility for handling the risk
Driver C, D, E, and F are either taking full responsibility or letting AP take full responsibility. Any crashes are assigned appropriately
Driver G has an AP crash that is assigned appropriately to AP
Driver H is Tesla skewing the data. It has an AP crash that is not assigned to AP. Tesla assumes 6 seconds is enough time that the driver should have avoided the crash

If a lot of drivers act like A and B then AP gets the benefit of all the good miles and none of the crashes. Drivers A and B have skewed the data themselves and made AP look better.
Drivers C, D, E, F, and G are correctly recorded statistics.
If there are many situations like with Driver H then Tesla has skewed the data and AP gets no penalty for crashes that it caused.

ProtonSF is that what you meant?