NTSB report on fatal Joshua Brown accident in Florida

[feslatan]

It felt to me reading the news about it that they were drawing a comparison between his attention and his hands being on the wheel when that may not be a fair comparison. Just because it didn't detect him holding the wheel for minutes prior to the accident doesn't mean he wasn't paying attention during that time, he may have been watching a movie or he may have been paying attention most of the time and only took his eyes off the road for a few seconds, as you said that's all it takes to have an accident.

Yes. I feel like if the average Tesla driver reads this and thinks "well I don't take my hands off the wheel for long periods of time so what happened to him won't happen to me" then it's important that the report be as accurate as possible and people realize that having your hands on the wheel isn't a substitute for paying visual attention. The former is great for sensing the car making movements it shouldn't make but hands on the wheel alone are worthless for sensing the need for evasive maneuvers that the car isn't seeing.

Put a different way: You keep your hands on the wheel to understand what the car is doing. You keep your eyes on the road to understand what the car isn't doing.

 

[Skotty]

Did you miss my post? No DVD or movie file was found in the vehicle. He most certainly wasn't watching or listening to a DVD or a ripped version of the movie. See post #18 two posts above this one for details.

Right. I said there was no evidence. I was just taking the "absence of evidence is not evidence of absence" position, though I was still making the point that people shouldn't claim or suggest he was watching a movie based on limited circumstantial evidence and heresay.

 

[dhanson865]

Right. I said there was no evidence. I was just taking the "absence of evidence is not evidence of absence" position, though I was still making the point that people shouldn't claim or suggest he was watching a movie based on limited circumstantial evidence and heresay.

It's just after months of people saying he was watching a DVD it should be repeated loud and clear that there was no DVD player or DVD media in the car.

The report found a laptop, a chromebook, and some SD cards. None of which had a movie on them.

If you want to talk about the theoretical I think it's only fair you add a disclaimer that makes it clear what the known facts are before or in addition to any theory you want to discuss.

Nothing against you or your writing style or your desire to discuss that concept. It's just this well has been tainted already and we need damage control to stop the spread.

 

[chillaban]

It's just after months of people saying he was watching a DVD it should be repeated loud and clear that there was no DVD player or DVD media in the car.

Yeah… I mean, I remember reading through the threads here, and there was a recurring sentiment that he must've been doing something obscenely stupid in his car in order to end up in this situation (e.g. watching a movie, using AP on a totally inappropriate road type, etc etc etc).

Reading through both accident reports, the only thing that I saw was that he barely touched the wheel at all. The steering wheel sensed his hands for all of 25 seconds across 40 minutes of driving, only to respond to nags and clear them. But somehow, I hear a lot of "nag" sentiment around here that implies that many other AP drivers do the same thing.

But at any rate, regardless of how you treat the steering nag, it's worth remembering that you don't have to do something comically stupid in order to end up in a deadly situation with AP. A car with limited automation capabilities piloting itself at highway speeds can get into a lot of trouble in very little time. But as long as you're paying attention, you can avert all of those disasters before they happen.

To date I'm aware of zero AP incidents where an attentive driver wasn't able to avoid a collision caused by an AP input. I am aware of a few near-misses involving over-jerking the steering wheel to disengage and causing loss of control. So far, across millions of miles, it holds true that as long as you supervise your AP, it is a safe feature to use.

 

[SomeJoe7777]

it's worth remembering that you don't have to do something comically stupid in order to end up in a deadly situation with AP.

This. His eyes were off the road for less than 7 seconds.

Every last one of us who has had AP engaged on an inter-city highway has, at one time or another, looked at the media player, the map, the browser, our phone, etc. for longer than 7 seconds. It's easy to do. And because 99.99999% of the time, nothing happens, we grow complacent.

Keep your hand on the wheel, keep your eyes on the road. There are no substitutes for that (yet).

 

[MikeBur]

Did you miss my post? No DVD or movie file was found in the vehicle. He most certainly wasn't watching or listening to a DVD or a ripped version of the movie. See post #18 two posts above this one for details.

Lol, did you miss mine - 9 posts before yours? #9? Or better yet post number 1?

Couldn't resist...

Nett is first witness to get to Tesla, incidentally >300 FEET away from truck, said there was nothing about movie present. Iirc Truck driver apparently stated this, though either has super-human vision (flying in face of noticing oncoming Tesla) or mistaken... I've purposely stayed away from judging, though the DUI on the drug screening is pretty damning if it's interpreted the way I'm reading it "Offense: DUI - Fatality", still at https://dms.ntsb.gov/public/59500-59999/59989/604748.pdf

 

[SMAlset]

I did find it interesting that there was early mention of Harry Potter and maybe he had the music playing through the car speakers and the truck driver heard that briefly and recognized it and suggested it was the movie playing. To me this would make some sense, otherwise I find the mention of Harry Potter in any form like way out there. This was just my thought seeing the account of what was on his equipment.

 

[chillaban]

I did find it interesting that there was early mention of Harry Potter and maybe he had the music playing and the truck driver heard that and suggested it was the movie playing. This was just my thought seeing the account of what was on his equipment.

Every outrageous statement has some grain of truth

But I mean, it's understandable. The truck driver was trying to cope with killing a driver in an at-fault accident while driving under the influence. It's only human nature to attempt to pin the blame on something else, and the media loves a story about a person with an expensive technologically advanced car doing something really stupid like watching a movie.

It just shows to take early witness reports with a grain of salt (someone claimed they were going over 80 and the Tesla passed them, while the black box shows no such speeds). And it also highlights the value and diligence of both the NHTSA and NTSB investigations.

 

[MikeBur]

Findings of THC in the truck drivers bloodstream suggest DUI if that definition applies in Florida ...

It would appear so. From toxicology report of truck driver:

• 0.0662 (ug/ml, ug/g) Tetrahydrocannabinol Carboxylic Acid (Marihuana) detected in Blood
  
  From: CAMI Toxicology Drug Information for: Tetrahydrocannabinol
  
  Therapeutic high is 0.025 ug/ml
  Toxic level is. 0.05 ug/ml
  
  From reports truck driver refused urine sample and this was from blood after compelled.
  
  This may have been the cause, though doesn't change fact that it appears to confirm AP nor AEB took any preventative action. We all, Tesla and anyone using current ADAS systems, have compelling data why these are solely to aid driving, and not a substitute for humans in the loop.

 

[SMAlset]

The truck driver could have also looked right once before making his turn (or not at all) and just pulled out in front of a car coming down the road at a distance at 70mph. Once he committed to making the left turn with that trailer he was pretty much committed to it. I haven't read the report but wonder if he ever saw through the truck's passenger window that Mr. Brown coming. If so the truck driver might have had time to honk his horn. I haven't read too much on this accident and would like to find the time to read through the report but sounds like Mr. Brown was caught off guard. Doesn't absolve him of culpability either. I just hate thinking about this case.

You can have someone come in and rob an establishment and run out and get all kinds of eyewitness statements about the perpetrators and their movements and not get a uniform consensus of it. Stuff happens so fast and when it's as dramatic and tragic as this was I can see why there would be discrepancies and why the investigations are painfully slow as they sift through eyewitness reports, measurement, etc. I've been in a few car accidents as a passenger and even if you are paying attention to the surroundings its hard to recall everything you saw or heard.

 

[mmd]

Yes. I feel like if the average Tesla driver reads this and thinks "well I don't take my hands off the wheel for long periods of time so what happened to him won't happen to me" then it's important that the report be as accurate as possible and people realize that having your hands on the wheel isn't a substitute for paying visual attention. The former is great for sensing the car making movements it shouldn't make but hands on the wheel alone are worthless for sensing the need for evasive maneuvers that the car isn't seeing.

Put a different way: You keep your hands on the wheel to understand what the car is doing. You keep your eyes on the road to understand what the car isn't doing.

I have seen some youtubers using water bottles as hands-on-wheel for their Tesla Model S. IMO, GM's approach of monitoring eye balls is a step in the right direction.

Also. I read somewhere that NTSB couldn't read the data by themselves, as Tesla used a proprietary format/encoding for storing data logs. I'll take any data that is filtered through the manufacturer with a grain of salt. Particularly if a feature in question is a big selling point.

 

[chillaban]

• This may have been the cause, though doesn't change fact that it appears to confirm AP nor AEB took any preventative action. We all, Tesla and anyone using current ADAS systems, have compelling data why these are solely to aid driving, and not a substitute for humans in the loop.

Totally agree. I actually welcome government scrutiny here, particularly from NTSB who has so much expertise with human-automation interactions.

I'm not calling out Tesla in particular, but basically every automaker now has SOME active safety feature they advertise as being a collision avoiding system. Some go out of their way in TV commercials to portray the system preventing all sorts of collisions. But all of them have some fine print basically negating the whole ad, saying that the system cannot stop all crashes. Yet it's obvious what message they're trying to send to buyers to lure them into buying their car.

I think it's good that when a system like this fails to perform an advertised responsibility, it is subject to scrutiny to understand whether or not it functioned properly, regardless of whether or not the driver did something wrong.


FWIW, someone I know just recently got into an accident in his fancy camera-and-radar-equipped Top Safety Pick+ car. It was a textbook sort of rear ending due to multiple factors competing for human attention, fell within all of the restrictions mentioned in the manual, but there was zero warning or AEB. Of course it's still his fault he rear-ended someone, but I wish there was more scrutiny into why these collision avoidance systems work or don't work. Mysteriously, they all seem to work with the IIHS/NCAP blow up dolls

 

[chillaban]

I have seen some youtubers using water bottles as hands-on-wheel for their Tesla Model S. IMO, GM's approach of monitoring eye balls is a step in the right direction.

Also. I read somewhere that NTSB couldn't read the data by themselves, as Tesla used a proprietary format/encoding for storing data logs. I'll take any data that is filtered through the manufacturer with a grain of salt. Particularly if a feature in question is a big selling point.

(1) Monitoring eyeballs has even more limitations. With sunglasses, these systems have a lot more ambiguity of tracking. And that's not even a deliberate attempt to outsmart it. Plus, these systems tend to struggle with situations such as the driver having motor/spine disabilities that change their facial expression / head position relative to what you train your models against. If you're going to pit a computer safety interlock system against a human, I am pretty sure Youtube has proven that humans can outsmart these systems! Adding more complexity just results in a false sense of security and higher costs for everyone involved.

(2) The NTSB also frequently relies on plane manufacturers/suppliers to reconstruct badly damaged black boxes or recover black box like data from other onboard computers. It's a pretty hefty claim to imply that Tesla curated evidence for a NTSB investigation, as I'm sure that comes with a world of hurt beyond even being found at fault for one fatality.

 

[mmd]

(1) Monitoring eyeballs has even more limitations. With sunglasses, these systems have a lot more ambiguity of tracking.
(2) The NTSB also frequently relies on plane manufacturers/suppliers to reconstruct badly damaged black boxes or recover black box like data from other onboard computers. It's a pretty hefty claim to imply that Tesla curated evidence for a NTSB investigation, as I'm sure that comes with a world of hurt beyond even being found at fault for one fatality.

True about the sunglasses I think, it also checks head position, But I agree, this is not a sure shot either.

I've not implied anything; just mentioned the possibility. Why use a proprietary method of recording these? The black boxes you mention, are they also using proprietary formats? If there is an incident, why shouldn't the driver/owner have a right to read his own car's data and download it to his computer to analyze? There were few people who repeatedly asked for their data logs from Tesla after minor incidents, which Tesla has refused.

MobilEye CEO has said, Tesla hyped the abilities of AP beyond its capabilities, and so it ended its relationship with Tesla. Joshua Brown was posting pretty reckless AP driving videos on Youtube prior to this incident, and Elon even re-tweeted one of his "Tesse saved me" video. As if the guy forgot how to drive in a matter of weeks of using AP.

I find it odd that the report mentions about not putting hands on wheel for a long time, as if it is relevant in this case. The accident could have happened within the first few seconds of taking eyes off the road.

 

[chillaban]

True about the sunglasses I think, it also checks head position, But I agree, this is not a sure shot either.

I've not implied anything; just mentioned the possibility. Why use a proprietary method of recording these? The black boxes you mention, are they also using proprietary formats? If there is an incident, why shouldn't the driver/owner have a right to read his own car's data and download it to his computer to analyze? There were few people who repeatedly asked for their data logs from Tesla after minor incidents, which Tesla has refused.

I also believe in the right to read your own data, especially when it's going to implicate your fault in a crash. Don't get me wrong.

But it also seems like this level of raw data provides a lot of information about how the underlying AP technology works. And it has a nonzero engineering cost to document the format and maintain all of that for external consumption.

MobilEye CEO has said, Tesla hyped the abilities of AP beyond its capabilities, and so it ended its relationship with Tesla. Joshua Brown was posting pretty reckless AP driving videos on Youtube prior to this incident, and Elon even re-tweeted one of his "Tesse saved me" video. As if the guy forgot how to drive in a matter of weeks of using AP.

I find it odd that the report mentions about not putting hands on wheel for a long time, as if it is relevant in this case. The accident could have happened within the first few seconds of taking eyes off the road.

FYI, at that point the MobilEye and Tesla relationship was already falling apart, and it's worth taking a competitor's statements with a grain of salt. The truth is usually somewhere in the middle, but if you are a vendor trying to sell Tesla on a long non-exclusive roadmap of incremental technologies when Tesla believes they can meet a quicker timeline and have more competitive advantage, I'm not sure which side of the story I believe.


The AP logs only use whether or not the driver's hands are on the wheel to measure attention. I think there's little other valuable performance data. Heck if it recorded dashcam-like video of the whole drive, the NTSB can draw higher quality conclusions about whether or not the guy was paying attention.

To me, the most damning thing about the whole report was the logs showing the AP steering state. It was literally constantly "hands required, not detected" EXCEPT exactly 1 minute or 3 minute or 5 minutes apart (corresponding to nag events), he wiggled the wheel a bit and went back to not putting his hands on the wheel. That can't be distilled into a snippy headline, but it painted a fairly clear picture of a complacently distracted driver.

 

[azred]

It sounds like this accident got scrutiny on a level only matched by airplane accidents. That surprises me even though I understand that there is a lot of interest in this new technology.

 

[SomeJoe7777]

It sounds like this accident got scrutiny on a level only matched by airplane accidents. That surprises me even though I understand that there is a lot of interest in this new technology.

It's the first automobile fatality in history where the vehicle was under 2-axis automatic control. It warrants a full multi-level investigation, as it has implications for all future automatic vehicle control systems.

I read most of the entire docket. Even though at this point, it is only a factual docket (i.e. no conclusions or recommendations yet), I've read enough NTSB reports to read between the lines. It appears to me that the NTSB wasn't too happy that they had to have Tesla's help to decode the on-board parameters and get other parameters from Tesla's database. They went out of their way to point out that the vehicle did not have (nor was required to have) an Event Data Recorder (EDR) which made their job more difficult compared to aviation investigations where the EDRs (cockpit voice recorder - CVR, and flight data recorder - FDR) are standardized.

I predict that one of the recommendations in the final docket will be that vehicles designed for the higher levels of autonomous driving should be required to have a standardized EDR that needs no manufacturer assistance to retrieve the data from.

 

[stopcrazypp]

True about the sunglasses I think, it also checks head position, But I agree, this is not a sure shot either.

I've not implied anything; just mentioned the possibility. Why use a proprietary method of recording these? The black boxes you mention, are they also using proprietary formats? If there is an incident, why shouldn't the driver/owner have a right to read his own car's data and download it to his computer to analyze? There were few people who repeatedly asked for their data logs from Tesla after minor incidents, which Tesla has refused.

Why proprietary formats? It's quite simple. To save space/bandwidth. If you store everything in a human readable format, you waste a ton of space.

There are 53 different (relevant) variables according to the report.

Let's just take one for example, here's the 8 possible values for the "Autopilot Hands On State" variable:
HANDS NOT REQD = Autopilot not controlling vehicle.
HANDS REQD DETECTED = Steering wheel torque sufficient to detect hands-on.
HANDS REQD NOT DETECTED = Steering wheel torque insufficient to detect hands-on.
HANDS REQD VISUAL = Visual warning to driver.
HANDS REQD CHIME 1 = Stage 1 aural warning to driver.
HANDS REQD CHIME 2 = Stage 2 aural warning to driver.
HANDS REQD SLOWING = Autopilot slowing vehicle.
HANDS SNA = Steering wheel hands-on detection not available

With a proprietary binary format you can store that variable in just 3 bits (which allows 8 values). With human readable 8-bit ascii, it would need at least 23 characters so that would take 23 bytes (184 bits). That is 61x as much space.

As for the part about regulation about car data (summary, automakers are not legally required to equip cars with EDRs, nor provide tools for retrieving such data):
"3.3. Event Data Recorder Regulations
Federal regulation 49 CFR 5637 specifies the data collection, storage, and retrievability requirements for vehicles equipped with event data recorders. The regulation does not require that vehicles be equipped with event data recorders. Equipping a vehicle with an event data recorder is completely voluntary. The regulation also specifies vehicle manufacturer requirements for providing commercially available tools and/or methods for accomplishing data retrieval from an event data recorder in the event of a crash. The Tesla Model S involved in this crash did not, nor was it required by regulation, contain an event data recorder. As a result, the data recorded by the ECU was not recorded in accordance with this regulation. Further, there is no commercially available tool for data retrieval and review of the ECU data. NTSB investigators had to rely on Tesla to provide the data in engineering units using proprietary manufacturer software."

 

[chillaban]

I predict that one of the recommendations in the final docket will be that vehicles designed for the higher levels of autonomous driving should be required to have a standardized EDR that needs no manufacturer assistance to retrieve the data from.

I'm not familiar with how these timelines usually unfold, but will the NTSB eventually release recommendations/conclusions?

I also read strong undertones for both the nag / attention compliance system being lacking and readily exploited by the driver, as well as the use of dual-agreement braking and the rear-only image training of the EyeQ3 as inadequate. It seems like Tesla must've known which way the investigation was heading, before the v8.0 radar enhancement blog. Between that feature and the AP2 vision classification behavior, it seems obvious Tesla made a lot of cautious decisions that led to increased braking false positives, which in turn led to more customer complaints about dissatisfactory TACC/AP behavior.

 

[chillaban]

Why proprietary formats? It's quite simple. To save space/bandwidth. If you store everything in a human readable format, you waste a ton of space.

There are 53 different variables according to the report.

Let's just take one for example, here's the 8 possible values for the "Autopilot Hands On State" variable:
HANDS NOT REQD = Autopilot not controlling vehicle.
HANDS REQD DETECTED = Steering wheel torque sufficient to detect hands-on.
HANDS REQD NOT DETECTED = Steering wheel torque insufficient to detect hands-on.
HANDS REQD VISUAL = Visual warning to driver.
HANDS REQD CHIME 1 = Stage 1 aural warning to driver.
HANDS REQD CHIME 2 = Stage 2 aural warning to driver.
HANDS REQD SLOWING = Autopilot slowing vehicle.
HANDS SNA = Steering wheel hands-on detection not available

With a proprietary binary format you can store that variable in just 3 bits (which allows 8 values). With human readable 8-bit ascii, it would need at least 23 characters so that would take 23 bytes (184 bits). That is 61x as much space.

I think the bigger problem is that:

(1) Tesla gave that interpretation of the data. If they wanted to orchestrate a cover up, they could've given another decoding of the bits that had a more favorable interpretation.
(2) Tesla had to recover the data from the computer. If they wanted to hide data, they could've conveniently curated what to give the NTSB or not. Or say that it was just not recoverable.


With that said, those are pure hypotheticals. Obstructing a NTSB investigation sounds criminal, and I would not imagine Tesla doing that for the sake of covering up one death.