Blog NTSB Releases Preliminary Report on Autopilot Crash

The U.S. National Transportation Safety Board issued a preliminary report on a fatal March 23 crash involving a Tesla Model X using Autopilot near Mountain View, Calif.

Investigators leveraged data pulled from the car’s computer that shows the driver’s hands were on the steering wheel for just 34 seconds during the minute before impact.

Data also showed that the Model X sped up to 71 miles per hour just before hitting a highway barrier. Tesla issued a release in March that included most of the info in the report. Tesla said “the driver had received several visual and one audible hands-on warning earlier in the drive” and the driver had about five seconds and 150 meters of unobstructed view of the concrete…but the vehicle logs show that no action was taken.”

The NTSB report said the crash remains under investigation, with the intent of issuing safety recommendations to prevent similar crashes. No pre-crash braking or evasive steering movement was detected, according to the report.

“Tesla Autopilot does not prevent all accidents — such a standard would be impossible — but it makes them much less likely to occur,” Tesla wrote in its March post. “It unequivocally makes the world safer for the vehicle occupants, pedestrians and cyclists.”

Read the full report here.

 

[dhanson865]

Instead, that is the job of forward collision warning and automatic emergency braking, whose owners-manual description is "Automatic Emergency Braking operates only when driving between 5 mph (8 km/h) and 85 mph (140 km/h)." AEB is NOT designed to ignore stationary objects, so a reasonable question is what (didn't) happen in this accident, and whatever the answer, it was more complex than the radar or AEB not being able/designed to react to stationary objects.

You need to reread the Model S owners manual. IIHS tested at 25 and below were AEB will bring the car to a stop. But AEB at speeds above 28 mph behaves differently as quoted below

AEB only kicks in for frontal collisions where there is another object moving in front of the Tesla going the same way or otherwise moving in front of the sensor. It doesn't kick in for stopped objects of those kinds and it sure doesn't kick in for every possible stopped object. So what does it kick in for? According to the manuals

• moving vehicles
• moving motorcycles
• moving bicycles
• moving pedestrians

For example from the Model 3 manual PUBLISHED MARCH 13, 2018

Automatic Emergency Braking

The forward looking camera(s) and the radar sensor are designed to determine the distance from an object (vehicle, motorcycle, bicycle, or pedestrian) traveling in front of Model 3. When a frontal collision is considered unavoidable, Automatic Emergency Braking is designed to apply the brakes to reduce the severity of the impact.
But the Police vehicle hit wasn't traveling in front. It was parked on the side of the road.

and the same from the Model S manual dated March 22, 2018

Automatic Emergency Braking
The forward looking camera(s) and the radar sensor are designed to determine the distance from an object (vehicle, motorcycle, bicycle, or pedestrian) traveling in front of Model S. When a frontal collision is considered unavoidable, Automatic Emergency Braking is designed to apply the brakes to reduce the severity of the impact.

Hmm, traveling in front, not parked on the side of the road, not barriers, not random debris. I'm thinking the owners manuals all sing the same tune. Stopped vehicles don't get any consideration from AEB. Sationary objects of any kind are ignored by AEB per the owners manual.


Going one step further the car had accelerated to 71 mph

When Automatic Emergency Braking applies the brakes, the touchscreen displays a visual warning and sounds a chime. You may also notice abrupt downward movement of the brake pedal. The brake lights turn on to alert other road users that you are slowing down.

If driving 29 mph (46 km/h) or faster, the brakes are released after Automatic Emergency Braking has reduced your driving speed by 25 mph (40 km/h). For example, if Automatic Emergency Braking applies braking when driving 56 mph (90 km/h), it releases the brakes when your speed has been reduced to 31 mph (50 km/h).

Automatic Emergency Braking operates only when driving between approximately 7 mph (10 km/h) and 90 mph (150 km/h).

The brakes are released after Automatic Emergency Braking has reduced your driving speed by 25 mph.

So 71 - 25 = 46. The car would have still hit the barrier at 46 mph even if AEB knew to kick in. Since AEB doesn't kick in for stationary objects, it hit at 71 mph.

 

[Barklikeadog]

You need to reread the Model S owners manual. IIHS tested at 25 and below were AEB will bring the car to a stop. But AEB at speeds above 28 mph behaves differently as quoted below

AEB only kicks in for frontal collisions where there is another object moving in front of the Tesla going the same way or otherwise moving in front of the sensor. It doesn't kick in for stopped objects of those kinds and it sure doesn't kick in for every possible stopped object. So what does it kick in for? According to the manuals

• moving vehicles
• moving motorcycles
• moving bicycles
• moving pedestrians

For example from the Model 3 manual PUBLISHED MARCH 13, 2018



and the same from the Model S manual dated March 22, 2018


Hmm, traveling in front, not parked on the side of the road, not barriers, not random debris. I'm thinking the owners manuals all sing the same tune. Stopped vehicles don't get any consideration from AEB. Sationary objects of any kind are ignored by AEB per the owners manual.


Going one step further the car had accelerated to 71 mph



The brakes are released after Automatic Emergency Braking has reduced your driving speed by 25 mph.

So 71 - 25 = 46. The car would have still hit the barrier at 46 mph even if AEB knew to kick in. Since AEB doesn't kick in for stationary objects, it hit at 71 mph.

If the car is hitting the brakes that hard... The driver will) finish the job as this is the warning to him. Driver isnt getting much of a warning if the car is accelerating into a stationary object. That kind of feedback says all is well.

 

[benfar]

You need to reread the Model S owners manual. IIHS tested at 25 and below were AEB will bring the car to a stop. But AEB at speeds above 28 mph behaves differently as quoted below

AEB only kicks in for frontal collisions where there is another object moving in front of the Tesla going the same way or otherwise moving in front of the sensor. It doesn't kick in for stopped objects of those kinds and it sure doesn't kick in for every possible stopped object. So what does it kick in for? According to the manuals

• moving vehicles
• moving motorcycles
• moving bicycles
• moving pedestrians

For example from the Model 3 manual PUBLISHED MARCH 13, 2018



and the same from the Model S manual dated March 22, 2018


Hmm, traveling in front, not parked on the side of the road, not barriers, not random debris. I'm thinking the owners manuals all sing the same tune. Stopped vehicles don't get any consideration from AEB. Sationary objects of any kind are ignored by AEB per the owners manual.


Going one step further the car had accelerated to 71 mph



The brakes are released after Automatic Emergency Braking has reduced your driving speed by 25 mph.

So 71 - 25 = 46. The car would have still hit the barrier at 46 mph even if AEB knew to kick in. Since AEB doesn't kick in for stationary objects, it hit at 71 mph.

You continue to express your opinion as fact, in spite of the test data I quoted for you. The FACT is that the IIHS test for AEB is for a stationary target. Other tests, especially by the European NCAP program, involve moving targets too, but not the quoted IIHS test.

Tesla's implementation of AEB described in their owners manual is to reduce speed by 25 miles per hour, but it doesn't explain why. I suspect it is to make sure the driver realizes there is a serious problem and takes over with either further braking, steering, and/or some combination.

The IIHS test of AEB is with a stationary target carefully designed to simulate a vehicle. From here http://www.iihs.org/media/a582abfb-...tings/Protocols/current/test_protocol_aeb.pdf , this section describes their AEB test:

Stationary Target System for AEB Testing The ADAC Advanced Emergency Braking System (AEBS), which is produced by Messring Systembau GmbH, is used as the stationary target (MESSRING Systembau MSG GmbH). The stationary carriage and balloon car are shown in Figure 2. The balloon car is equipped with a U.S.- specific license plate. At the beginning of testing, the target will be inflated to 250 mbar and maintained at that pressure throughout testing. The target cover should also be free of wrinkles during testing.

 

[benfar]

You continue to express your opinion as fact, in spite of the test data I quoted for you. The FACT is that the IIHS test for AEB is for a stationary target. Other tests, especially by the European NCAP program, involve moving targets too, but not the quoted IIHS test.

Tesla's implementation of AEB described in their owners manual is to reduce speed by 25 miles per hour, but it doesn't explain why. I suspect it is to make sure the driver realizes there is a serious problem and takes over with either further braking, steering, and/or some combination.

The IIHS test of AEB is with a stationary target carefully designed to simulate a vehicle. From here http://www.iihs.org/media/a582abfb-...tings/Protocols/current/test_protocol_aeb.pdf , this section describes their AEB test:

Stationary Target System for AEB Testing The ADAC Advanced Emergency Braking System (AEBS), which is produced by Messring Systembau GmbH, is used as the stationary target (MESSRING Systembau MSG GmbH). The stationary carriage and balloon car are shown in Figure 2. The balloon car is equipped with a U.S.- specific license plate. At the beginning of testing, the target will be inflated to 250 mbar and maintained at that pressure throughout testing. The target cover should also be free of wrinkles during testing.

One more data point is this video that shows the Tesla traveling at 30 mph, detecting a truck stopped partially blocking its lane ahead, and slowing until the driver took control. (That section starts about timestamp 6:11.)

A caveat is that the truck was moving but did not appear in the instrument-panel display until it was stopped. That suggests the radar did not detect it while moving, but it's possible it did but it wasn't shown in the display for some reason. If that was true, then this is not a good example because it would have classified the originally-moving object that became stationary as dangerous. If that wasn't the case, then this is an example of a Tesla traveling at 30 mph whose radar detected a stationary object blocking its lane and reacted appropriately.

 

[mongo]

A caveat is that the truck was moving but did not appear in the instrument-panel display until it was stopped. That suggests the radar did not detect it while moving, but it's possible it did but it wasn't shown in the display for some reason.

Likely that. It takes a bunch more SW to get target data on the cluster display. Along with the ability to filter what is displayed.

 

[mongo]

Tesla's implementation of AEB described in their owners manual is to reduce speed by 25 miles per hour, but it doesn't explain why. I suspect it is to make sure the driver realizes there is a serious problem and takes over with either further braking, steering, and/or some combination.

Also to reduce severity of false positives. 70-45 on freeway may be recoverable before being rear ended 70-0 much less so. Actual drop in speed is dependent on braking force vs reaction time.

 

[benfar]

Also to reduce severity of false positives. 70-45 on freeway may be recoverable before being rear ended 70-0 much less so. Actual drop in speed is dependent on braking force vs reaction time.

I absolutely agree. It's all about confidence in the sensor data and resulting analysis, and the auto companies and manufacturers of collision warning systems (typically for commercial vehicles) have varying strategies for balancing true positives & true negatives with false positives & false negatives. Better data from the sensors and more independent sources of data all help raise confidence and allow more aggressive warnings &/or vehicle actions when something potentially dangerous is detected.

 

[dhanson865]

You continue to express your opinion as fact, in spite of the test data I quoted for you. The FACT is that the IIHS test for AEB is for a stationary target. Other tests, especially by the European NCAP program, involve moving targets too, but not the quoted IIHS test.

Tesla's implementation of AEB described in their owners manual is to reduce speed by 25 miles per hour, but it doesn't explain why. I suspect it is to make sure the driver realizes there is a serious problem and takes over with either further braking, steering, and/or some combination.

The IIHS test of AEB is with a stationary target carefully designed to simulate a vehicle. From here http://www.iihs.org/media/a582abfb-...tings/Protocols/current/test_protocol_aeb.pdf , this section describes their AEB test:

Stationary Target System for AEB Testing The ADAC Advanced Emergency Braking System (AEBS), which is produced by Messring Systembau GmbH, is used as the stationary target (MESSRING Systembau MSG GmbH). The stationary carriage and balloon car are shown in Figure 2. The balloon car is equipped with a U.S.- specific license plate. At the beginning of testing, the target will be inflated to 250 mbar and maintained at that pressure throughout testing. The target cover should also be free of wrinkles during testing.

You keep confusing low speed (under 29 mph) behavior with high speed (over 29mph) behavior. No amount of testing you cite for low speed applies to high speed.

 

[benfar]

You keep confusing low speed (under 29 mph) behavior with high speed (over 29mph) behavior. No amount of testing you cite for low speed applies to high speed.

I'm not confused. You said "Since AEB doesn't kick in for stationary objects" and i gave proof that it does at least in the 25 mph and under cases tested by the IIHS. Neither of us has proof it does or does not at speeds over that. Lacking the proof, I say I suspect it does, and without physical-world evidence you keep simply claiming it does not. I'll come back with proof either way when I find or generate it with tests myself.

 

[Lasttoy]

Who cares. The man told his friends it was not working properly . Yet he was going 70 on hwy 101, why? That section of road is so dangerous for any aware person, much less a not perfect AP.
The man in Florida was asleep. The lady in utah was texting.
We can't fix stupid. The boy in Florida was speeding, got a ticke for 112mph.

 

[benfar]

Who cares. The man told his friends it was not working properly . Yet he was going 70 on hwy 101, why? That section of road is so dangerous for any aware person, much less a not perfect AP.
The man in Florida was asleep. The lady in utah was texting.
We can't fix stupid. The boy in Florida was speeding, got a ticke for 112mph.

I care because I want people to understand what a vehicle they own or consider buying should be expected to do in dangerous situations. The IIHS and European NCAP tests are a good start, but as this discussion illustrates, their published results are not adequate at this point to answer questions about latest-technology capabilities like the 2018 Cadillac CT6 with Super Cruise and all Teslas equipped with radar. Drivers should always pay careful attention even if their Level 2 system (Super Cruise or Autopilot) is active, and those who don't even briefly are risking their lives.

 

[Mark_T]

The only safe assumption is that you should not expect it to do anything to prevent an accident and remain vigilant and ready to react at all times.

There are still too many situations where the vehicle response will be ‘wrong’ or absent for it to be safe to expect anything at all.

 

[104 gauss]

This report points out a fatal flaw with Autopilot. It can see a car in front of you only if the closing speed is under 30 mph. If not, the system is blind to anything in front of it. That includes street sweepers in China, tractor trailers in Florida and firetrucks in California and Utah, a car at a stop light in Arizona, and a concrete barrier in Mountain View.

Tesla needs to come clean what is causing these accidents so that people know what to be watching for. Guess this is why the IIHS only tests AEB systems (along with Autopilot) at 12 and 25 mph. They don't work at faster closure rates.

 

[104 gauss]

Answer to your Question! "Tesla needs to come clean what is causing these accidents"
Drivers who are not taking there responsibility seriously. The Ultimate responsibility for your life and safety of others is YOU!

 

[TaoJones]

On a daily basis when crossing an intersection with AS engaged, if I let it the car would run directly into an oncoming raised median *after turning to the left* so that that median would be at the center of the front end of the vehicle rather than to the left as one would vastly prefer. I check after every firmware update to see if/when this will be fixed and of course nothing yet.

Now, this is at 30-40mph rather than at 70mph, and clearly the car is within camera and radar range of the median on the other side of the intersection.

Not exactly confidence-inspiring as there would be much less reaction time at 70mph as was experienced by the unfortunate soul who impacted the un-repaired barrier in the Bay Area recently. Ironic that that same barrier saved the life of a drunk driver only ~10 days prior (he reportedly walked away after his imported car hit the correctly-functioning barrier in excess of 60mph).

In any case, unless and until AS can manage to not drive my car into a median on the other end of an intersection at relatively low speed, I have exactly zero confidence that it will avoid a median or barrier in front of a median area at higher speed.

It would be nice if/when Tesla gets around to fixing this problem. In the meantime, Caltrans had better get on the stick with their repair windows for damaged barriers, ffs.

 

[benfar]

Whether AEB works at a faster initial speed of 25 mph when approaching a stationary object is what I was exploring with the original poster above. It definitely works up to the speed that IIHS tests, as the report I linked shows. Whether it works at higher speeds is not clear. I suspect it does, the original poster believes it does not, and neither of us has presented real-world evidence to prove either view yet.

I want to point out some things here about the above list of Tesla crashes involving stationary objects. I haven't seen an explanation of the Chinese accident, but keep in mind it was a hardware 1 vehicle. The Florida crash was explained by Tesla, and their "fleet learning for radar" software update in version 8.0 ( Upgrading Autopilot: Seeing the World in Radar ) was designed to improve recognition of stationary objects that are blocking the Tesla's path, versus those that are normally present in the same place but not dangerous because they are overhead signs, bridges, etc. So whatever the issues earlier, at least with software 8.0 and presumably all updates afterwards, any Tesla vehicle with those updates benefit from the "fleet learning for radar" improvements.

Both fire truck crashes reportedly were by Teslas originally following another vehicle that swerved out of the way. I suspect both triggered AEB as soon as the radar's view of the fire truck was no longer blocked, but am not sure reports have answered yet whether that was true.

I agree that the car at a stop light and the concrete barrier crashes are puzzling. I haven't seen a similar report about the stop light crash, but the preliminary NTSB report on the barrier crash states that the Tesla did no braking before the crash. If true, and also if true in the stop light case, then I'm puzzled why. Perhaps the earlier poster is correct, and AEB just doesn't trigger at those vehicle's speeds. Alternatively, it could be more complicated than that, but I don't plan to speculate. For my own knowledge, I will continue to look for video examples or other reliable test cases of similar cases with initial speeds above the 25 - 30 mph range in which tests show braking. As I told the original poster above, I'll share anything I find as a follow-on to this discussion. If I don't find anything, then I'm done here with this topic.

 

[mongo]

On a daily basis when crossing an intersection with AS engaged, if I let it the car would run directly into an oncoming raised median *after turning to the left* so that that median would be at the center of the front end of the vehicle rather than to the left as one would vastly prefer. I check after every firmware update to see if/when this will be fixed and of course nothing yet.

Now, this is at 30-40mph rather than at 70mph, and clearly the car is within camera and radar range of the median on the other side of the intersection.

Not exactly confidence-inspiring as there would be much less reaction time at 70mph as was experienced by the unfortunate soul who impacted the un-repaired barrier in the Bay Area recently. Ironic that that same barrier saved the life of a drunk driver only ~10 days prior (he reportedly walked away after his imported car hit the correctly-functioning barrier in excess of 60mph).

In any case, unless and until AS can manage to not drive my car into a median on the other end of an intersection at relatively low speed, I have exactly zero confidence that it will avoid a median or barrier in front of a median area at higher speed.

It would be nice if/when Tesla gets around to fixing this problem. In the meantime, Caltrans had better get on the stick with their repair windows for damaged barriers, ffs.

First off, do what you are comfortable with.

However, AP through an unmarked intersection on surface streets (which are not the specified realm of AP usage) is a different use case than highway.

Warning: Autosteer is intended for use only on highways and limited-access roads with a fully attentive driver. When using Autosteer, hold the steering wheel and be mindful of road conditions and surrounding traffic. Do not use Autosteer on city streets, in construction zones, or in areas where bicyclists or pedestrians may be present. Never depend on Autosteer to determine an appropriate driving path. Always be prepared to take immediate action. Failure to follow these instructions could cause serious property damage, injury or death.

 

[SlicedBr3ad]

All we can say is the car sensed torque on the wheel.
I can steer with my knees...

https://m.imgur.com/g4cJUmM

 

[benfar]

Whether AEB works at a faster initial speed of 25 mph when approaching a stationary object is what I was exploring with the original poster above. It definitely works up to the speed that IIHS tests, as the report I linked shows. Whether it works at higher speeds is not clear. I suspect it does, the original poster believes it does not, and neither of us has presented real-world evidence to prove either view yet.

I want to point out some things here about the above list of Tesla crashes involving stationary objects. I haven't seen an explanation of the Chinese accident, but keep in mind it was a hardware 1 vehicle. The Florida crash was explained by Tesla, and their "fleet learning for radar" software update in version 8.0 ( Upgrading Autopilot: Seeing the World in Radar ) was designed to improve recognition of stationary objects that are blocking the Tesla's path, versus those that are normally present in the same place but not dangerous because they are overhead signs, bridges, etc. So whatever the issues earlier, at least with software 8.0 and presumably all updates afterwards, any Tesla vehicle with those updates benefit from the "fleet learning for radar" improvements.

Both fire truck crashes reportedly were by Teslas originally following another vehicle that swerved out of the way. I suspect both triggered AEB as soon as the radar's view of the fire truck was no longer blocked, but am not sure reports have answered yet whether that was true.

I agree that the car at a stop light and the concrete barrier crashes are puzzling. I haven't seen a similar report about the stop light crash, but the preliminary NTSB report on the barrier crash states that the Tesla did no braking before the crash. If true, and also if true in the stop light case, then I'm puzzled why. Perhaps the earlier poster is correct, and AEB just doesn't trigger at those vehicle's speeds. Alternatively, it could be more complicated than that, but I don't plan to speculate. For my own knowledge, I will continue to look for video examples or other reliable test cases of similar cases with initial speeds above the 25 - 30 mph range in which tests show braking. As I told the original poster above, I'll share anything I find as a follow-on to this discussion. If I don't find anything, then I'm done here with this topic.

I'm not confused. You said "Since AEB doesn't kick in for stationary objects" and i gave proof that it does at least in the 25 mph and under cases tested by the IIHS. Neither of us has proof it does or does not at speeds over that. Lacking the proof, I say I suspect it does, and without physical-world evidence you keep simply claiming it does not. I'll come back with proof either way when I find or generate it with tests myself.

I don't have evidence about higher-speed Tesla AEB performance yet, but did find encouraging actions that the European New Car Assessment Program ("NCAP") recently started to test AEB, FCW, and the combination for speeds up to 48 miles per hour (80 kilometers per hour). From here http://cdn.euroncap.com/media/32278/euro-ncap-aeb-c2c-test-protocol-v201.pdf is this diagram showing tests for a stationary target designed to appear like a vehicle to radar, lidar, and camera sensors. Tests will be run straight-on and with lateral offset left and right up to 50% of the width of the target. (There are also tests for slower-moving vehicles and braking vehicles.)

 

[SO16]

In fact, all of the crashes in which AP was involved likely would not have occurred had the driver not been using AP. That makes it less safe, not more.

LOL. Funny how you school people about facts and then turn around and say this.