Government regulations of L2 driver assist systems

[diplomat33]

I was watching the NTSB hearing today about the final report on the Tesla crash in 2018.

The chairman was talking about how he feels that government regulations have lagged behind the technology when it comes to advanced driver assist assist systems. Systems like Autopilot are becoming more and more advanced to the point where drivers might confuse them for self-driving when they are not, and regulations have not caught yet. He said L2 systems are prone to causing drivers to become overconfident in the system.

A few points related to AP were discussed that I thought were noteworthy:

1) Many auto makers not just Tesla don't really do a good job of restricting their driver assist system to only operate inside it's specified ODD. For example, AP can be activated on roads that technically are outside it's ODD as specified in the Tesla's owner manual.

2) L2 systems require driver attention but making sure that happens is left to the auto maker and different auto makers use different methods. Tesla's steering wheel torque method is found to be particularly problematic. It's not just possible to fool the system., it's also possible to satisfy the torque without really paying attention.

3) Tesla's camera vision can fail for several reasons with not a lot of redundancy. Poor visibility, camera obstruction, confusing lane markings, sun blinding the cameras etc can all potentially cause an accident if the driver is not paying attention.

I could see the government passing regulations to address those 3 points. They might force auto makers to limit use of an ADAS to only within the ODD. That seems very likely to me. And that would be easy for Tesla to comply with. They could require driver facing cameras for driver monitoring. That also seems very likely. It would require Tesla put additional hardware in future cars. They might also require certain sensors like extra radar or even lidar potentially to add redundancy. This is an open question but it is possible. Obviously, this would require extra hardware as well.

If anything, the crashes with Tesla are probably helping to raise awareness of the problem. I think the NTSB is realizing that as more and more auto makers start releasing advanced driver assist (Cadillac's Supercruise, Lexus's Highway Teammate etc), drivers will get overconfident and that is going to be a bigger and bigger safety issue if the systems don't have good driver attention systems and don't have the proper regulations.

 

[cucubits]

All 3 are valid points but some will be quite hard to address properly. I don't see how Tesla could do any mass hardware retrofit (for example if they end up having to add a driver facing camera to older cars - which would be a good solution).

Also, I do hope they won't have to follow on Cadillac's trail and limit AP to hwy activation only. I'm sure this will drive away potential customers.

 

[diplomat33]

Another point that came up in the hearing is that right now auto makers are left alone to determine the SAE level and ODD of their cars. In the hearing, they also complained that the NHTSA is not proactive enough on safety issues. So I would not be surprised if at some point, the NHTSA or some other third party is given the responsibility of classifying the ODD and SAE level of new cars. That seems like a very common sense change. That way, auto makers cannot fudge the information or mislead. Frankly, it seems kinda crazy to let auto makers just stick whatever level they want to their cars.

 

[S4WRXTTCS]

I don't expect to see any changes made to the existing install base. Aside from optional/expensive retrofits the customer could buy to gain hands free L2 capability.

If I was Tesla I'd highly consider responding to the NTSB this time around and not simply making changes without sending them a letter of what the changes are like they did last time. Last time they greatly increased the nags, and they added the AP jail thing.

This time around there really isn't anything they can do other than migrate new cars to a proper driver monitoring system like Cadillac, BMW, and Subaru use on some of their vehicles. They have to do this anyways to continue to sell L2 vehicles to the European market long term. So might as well add driving monitoring to US builds as well.

Going forwards I do expect Tesla to do a better job restricting the ODD with FSD especially with city NoA. In fact I think it's the best time as you really don't want to allow city NoA without a lot of clarity that where you're at is supported. Plus you disable AP in those areas to force those people to upgrade to FSD.

 

[diplomat33]

All 3 are valid points but some will be quite hard to address properly. I don't see how Tesla could do any mass hardware retrofit (for example if they end up having to add a driver facing camera to older cars - which would be a good solution).

The Model 3 has a driver facing camera so Tesla might only need to add some software to make it work for driver attention. Adding a driver facing camera to Model S and X cars would be more tricky. They could conceivably come up with a small camera that attaches to the dash and connects to the car wireless and then add the software via OTA update.

More likely, Tesla will just add the hardware to future cars.

Also, I do hope they won't have to follow on Cadillac's trail and limit AP to hwy activation only. I'm sure this will drive away potential customers.

Well, this could happen because allowing owners to use a driver assist like AP on roads that it is not designed for, increases the risk of accidents. Restricting the ODD might upset customers but it's an obvious fix that I think regulators will insist on.

But, if I was Tesla I'd highly consider responding to the NTSB this time around and not simply making changes without sending them a letter of what the changes are like they did last time. Last time they greatly increased the nags, and they added the AP jail thing.

Yeah, the NTSB was particularly upset at Tesla in the hearing for ignoring them. It came up several times. And now, that there has been 4 major crashes involving Tesla's AP, the NTSB is getting pretty pissed. Tesla would do well to at least respond and take their recommendations seriously.

Going forwards I do expect Tesla to do a better job restricting the ODD with FSD especially with city NoA. In fact I think it's the best time as you really don't want to allow city NoA without a lot of clarity that where you're at is supported. Plus you disable AP in those areas to force those people to upgrade.

In the hearing, the NTSB referenced the Florida crash with the semi truck where AP was activated outside it's ODD. Restricting the ODD seems like an obvious fix that would be easy to do via OTA update. And it would appease the regulators because then Tesla could say that AP is only being used on the right roads.

 

[MP3Mike]

Yeah, the NTSB was particularly upset at Tesla in the hearing for ignoring them. It came up several times.

As I heard Tesla didn't ignore them, but they felt that the response that they got from Tesla didn't adequately address the concerns that they raised. (i.e. they didn't like what Tesla said.)

 

[diplomat33]

As I heard Tesla didn't ignore them, but they felt that the response that they got from Tesla didn't adequately address the concerns that they raised. (i.e. they didn't like what Tesla said.)

The chairman of NTSB specifically mentioned that they had not gotten any letter from Tesla regarding this particular incident.

 

[MP3Mike]

The chairman of NTSB specifically mentioned that they had not gotten any letter from Tesla regarding this particular incident.

Sure, he said that but when they asked the staff person earlier she said that Tesla did respond but not to their liking.

 

[VT_EE]

I’m still waiting for a system to ensure a driver stays attentive while driving themselves. That is a far worse problem than monitoring the attentiveness of the driver while using a L2 assistant. Driving distracted with L2 is NO different than driving distracted with L1.

 

[diplomat33]

I’m still waiting for a system to ensure a driver stays attentive while driving themselves. That is a far worse problem than monitoring the attentiveness of the driver while using a L2 assistant.

Actually, driver facing cameras can be used for that purpose as well. Some auto makers are already doing it.

Driving distracted with L2 is NO different than driving distracted with L1.

I do think there is a difference. With L1, the car is not steering so the driver has to hold the wheel in order to steer the car. But with L2, the car is steering so the driver does not need to hold the wheel anymore. And when the driver does not need to hold the wheel or press the pedals, they can feel like it is ok to disengage their attention. That's why L2 is that threshold where drivers might be lured into a false sense of security.

 

[Daniel in SD]

I’m still waiting for a system to ensure a driver stays attentive while driving themselves. That is a far worse problem than monitoring the attentiveness of the driver while using a L2 assistant. Driving distracted with L2 is NO different than driving distracted with L1.

Such systems are already on some vehicles. I wouldn't be surprised if they end up being required.
Looks like every major manufacturer has some monitoring system. Driver drowsiness detection - Wikipedia

 

[johnm]

Isn't the real solution to this problem to get rid of L2 systems? Make the cars stop crashing and quit requiring people to babysit them. I guess I hope Tesla's plan is to have a L>2 system before new rules on L2 systems come into effect (years, I assume).

 

[diplomat33]

Isn't the real solution to this problem to get rid of L2 systems? Make the cars stop crashing and quit requiring people to babysit them. I guess I hope Tesla's plan is to have a L>2 system before new rules on L2 systems come into effect (years, I assume).

Ultimately yes, the answer is to get to full autonomous driving where we can remove the human component completely. But that will take time. Tesla is not there yet. Also, even when we get full autonomous driving working, it needs to be safe enough to deploy. Otherwise, you are just putting a lot of unsafe robotaxis on the road that will cause accidents and deaths. That's why companies like Cruise and Waymo that have L4 autonomy now, have not deployed their cars in large numbers yet because they need to make absolutely sure that their systems are safe enough first. Their robotaxis are good but not quite safe enough yet. Getting full autonomy to be safe enough, will also take time. It will be years before we can sell large numbers of L4/L5 autonomous cars to consumers.

So, the bottom line is that we need to do something about L2 in the mean time.

 

[S4WRXTTCS]

I’m still waiting for a system to ensure a driver stays attentive while driving themselves. That is a far worse problem than monitoring the attentiveness of the driver while using a L2 assistant. Driving distracted with L2 is NO different than driving distracted with L1.

The basic problem is that humans constantly evaluate risk when managing their behavior.

Like when they see a bicyclist who isn't wearing a helmet they'll give them more room than a bicyclist wearing a helmet. It's not a conscious behavior, but unconscious behavior. The helmet indicates that there is less danger.

The same kind of risk assessment happens with L0/L1/L2 driving. As an example a driver will see the risk of sending a text as far greater when driving manually than with AP engaged. The more recent Florida crash the accident happened within 10 seconds of engaging AP so it wouldn't be surprising if he did so in order to do something distracting.

I've certainly done that myself.

I've done it despite the fact that I have the knowledge to know better. That AP has all kinds of limitations, and should NEVER be trusted.

I imagine the general population does so a lot more than me whether they acknowledge it or not. They do so because they don't really understand how to access the risk factors with L2 systems are engaged.

I do want to clear one thing up. I'm not the kind of person that wants to put nanny devices in cars. I think we're going a bit too far with this whole "lets eradicate natural selection", and as a result we have devolution.

But, I do support moving from the torque sensor to hands free driver monitor like what Cadillac has as I absolutely hate the truly awful torque sensor.

 

[S4WRXTTCS]

Such systems are already on some vehicles. I wouldn't be surprised if they end up being required.
Looks like every major manufacturer has some monitoring system. Driver drowsiness detection - Wikipedia

What I find funny is how Subaru advertises their driver monitoring system.

They've taken the "we're going to protect your teenage driver" approach versus the "You probably text while you drive so we're gong to nag you" approach.

 

[Tiger]

I've always wondered, why Tesla's autopilot does not attempt to safely slow down and pull over when sensing driver inactivity, instead of just giving beeping signals? This way, if a crash occurred, at least the velocity would be lower, and possibly less injuries. The autopilot should begin safely slowing down and pulling over beginning from the first driver inactivity warning signal (and they could continue perfecting the detection thereof).

 

[Daniel in SD]

What I find funny is how Subaru advertises their driver monitoring system.

They've taken the "we're going to protect your teenage driver" approach versus the "You probably text while you drive so we're gong to nag you" approach.

Yeah. Even Waymo claims the system in their test vehicles is there to detect “fatigue”.

I've always wondered, why Tesla's autopilot does not attempt to safely slow down and pull over when sensing driver inactivity, instead of just giving beeping signals? This way, if a crash occurred, at least the velocity would be lower, and possibly less injuries. The autopilot should begin safely slowing down and pulling over beginning from the first driver inactivity warning signal (and they could continue perfecting the detection thereof).

I’m pretty sure if you ignore the autopilot nags it will put on the hazard lights and stop.

 

[diplomat33]

Here is the archived live stream of the NTSB Board Meeting I was referencing in the opening post of this thread where they discuss the various Tesla crashes, the issues with L2 and their recommendations:
http://ntsb.windrosemedia.com/02252020/

 

[banned-66611]

Tesla may have ruined it for everyone. By releasing a janky system that doesn't make much effort to enforce vital safety requirements like paying attention or only using it on suitable roads they are going to get the regulatory hammer brought down on everyone.

Autopilot needs to be disabled in light of this report, until they can implement a proper attention monitoring system and limit it to highways.

 

[diplomat33]

Tesla may have ruined it for everyone. By releasing a janky system that doesn't make much effort to enforce vital safety requirements like paying attention or only using it on suitable roads they are going to get the regulatory hammer brought down on everyone.

Autopilot needs to be disabled in light of this report, until they can implement a proper attention monitoring system and limit it to highways.

There is no doubt that Tesla took advantage of the lack of regulations. They released a beta driver assist with all kinds of known issues and with a poor driver monitoring system but trusted that they could improve it through OTA updates fast enough. And any accidents could simply be blamed on the driver not paying attention.

I don't think the answer is disabling AP completely because then you go back to no driver assist at all. But I do think limiting AP to strictly operating only in its ODD would be a good start. And I think Tesla should implement a proper attention monitoring system immediately.

I would like to see a more rigorous regulatory system for all automated driving systems regardless of SAE levels since we are clearly entering the era of advanced L2 but also the first autonomous systems (L3, L4 and even L5 at some point) will be coming in a few years. It's not just L2. When automakers start putting L3, L4 or even L5 autonomous systems on their consumer cars, we can't just trust the automakers that the systems are safe enough. There needs to be some oversight.

I would do the following:

1) Have the NHTSA test all automated driving systems for basic standard of safety and competencies. This would not just be hands-on testing but also collecting data and documentation from the auto maker. They need to test for phantom braking, lane keeping, responding to stopped vehicles at highway speeds etc... Issues like frequent phantom braking, getting confused with faded lines and hitting crash barriers, hitting stopped fire trucks, or hitting semi-trucks crossing in front like we saw in the Tesla crashes are serious safety issues whether the driver is paying attention or not. A system with those kinds of issues should not be allowed on roads until the auto maker fixes the issues.

2) The NHTSA would also be responsible for classifying the SAE level and determining the proper ODD.
- Based on the SAE level and ODD, the NHTSA would mandate how the system can be used and when. If the system is L0-L3, it must have a robust driver attention system that includes but not limited to a driver facing camera and it must be able to reliably detect lack of attention, distraction, fatigue, drowsiness and loss of consciousness.
- The NHTSA should also require that the system only be able to be turned on in its designated ODD.

3) The NHTSA should require some basic sensor redundancy based on the SAE level and ODD. To be clear, I am not suggesting that every car be forced to have like 30 cameras, 30 lidar and 30 radar. The sensor suite should match the SAE level and ODD. L2 does not need the same sensor suite as say L4. But I think the NHTSA should mandate some basic sensor redundancies like a rear radar or a forward lidar for example. This would help in those critical safety cases like responding to a stopped vehicle on the highway. Just depending on cameras alone is not good enough. There are too many ways that camera vision can fail from being blinded by the sun, poor visibility, confusing lane markings or even defacing signs.

I think these changes would greatly help with safety but also increase consumer confidence. This way, for example, if a car is labeled as L4 autonomous on the highway by the NHTSA, you would know it was fully validated and tested and it's not just some marketing claim by the automaker.