Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

It's not just Tesla: other ADAS systems can also be fooled!

This site may earn commission on affiliate links.

diplomat33

Average guy who loves autonomous vehicles
Aug 3, 2017
12,559
18,390
USA
Car and Driver tested the L2 ADAS systems from other manufacturers and found that their driver monitoring can be tricked as well. It's not just Tesla that has this problem.

Here is the full article:


Here is the video:


Here is the chart that shows their results:

adas-graphics-chart-v2-1628613228.png
 
  • Like
Reactions: TresLA
Most of them, when saddled with a weight on the helm, would mind the steering and speed for as long as you dare. Riding lawn mowers can detect a missing driver. Why can't cars?
uhm, what?

gee, when we go out of our way to defeat the known seat weight sensor, it detects that there is weight there. amazing. science is wonderful, isn't it?
 
Does anyone have any ideas for a better monitoring system? C&D is pushing for seat sensors which would be easily fooled too.

As I see it you have several requirements that need to be met, not that they are all the car software's responsibility:
  1. Person in the driver's seat is a human
  2. Driver is legally licensed to drive a car. Also legally allowed to drive THIS car, and is not legally impaired
    1. Not really anything the car is likely to be able to monitor
  3. Driver is alert and responds to nudges
  4. Driver is fully monitoring the driving environment and prepared to take over immediately
  5. Failure to obey the rules would be as bad as drinking & driving or dangerous driving
All the systems apparently don't make it past #1, they can all be fooled, and don't seem to care for a period of several seconds to several minutes whether there is a driver there or not. With careful fooling some can drive indefinitely.

Ignore #2, the car doesn't care. Though I still think having to actually insert a physical key, turn it, and operate some of the mechanically-stiff brake/shifter/steering wheel lock functions in some cars does essentially ensure that the driver is at least somewhat trained and likely older than 10 years old. Simply having a key card device somewhere near the car, shutting the door, clicking the belt, and touching the pedals is too easy in my opinion. You should have to do some actions that are difficult and require some strength to get a car moving. But that's not the direction we seem to be going in.

#3: Steering wheel nudges on cue and other wheel sensing, if effective, at least shows there is a driver responding to the nudges. Eye tracking and other sleep sensors, if effective, show that the driver is at least looking forward (even if they are watching a movie on the iPad on the dashboard). These systems are currently ineffective as they can be fooled too easily.

#4: I don't know how you can judge if someone is actually watching what is going on. Perhaps there could be an eye-tracking sensor tied to a projected blip on the screen. You have to look at the blip and follow it briefly. Randomly timed and randomly positioned but no more than a few minutes apart. Periodically the car could announce itself as dropping out of autosteer or autospeed and the driver has to drive for a period of time. Maybe the driver has to press a randomly specified touchpad or control. Mix it up. Do it often, but not during any intersections or driving maneuvers so as not to endanger the drive. Would any of this actually ensure the driver is paying attention though? Perhaps randomly startle the driver by simulating a rumble strip, nothing like a dose of adrenaline to make you pay attention.

#5: Driving while fooling the systems, or allowing unlicensed people/pets to take the car would have harsh penalties akin to the worst driving offenses so people are deterred from doing it.
 
Does anyone have any ideas for a better monitoring system? C&D is pushing for seat sensors which would be easily fooled too.

As I see it you have several requirements that need to be met, not that they are all the car software's responsibility:
  1. Person in the driver's seat is a human
  2. Driver is legally licensed to drive a car. Also legally allowed to drive THIS car, and is not legally impaired
    1. Not really anything the car is likely to be able to monitor
  3. Driver is alert and responds to nudges
  4. Driver is fully monitoring the driving environment and prepared to take over immediately
  5. Failure to obey the rules would be as bad as drinking & driving or dangerous driving
All the systems apparently don't make it past #1, they can all be fooled, and don't seem to care for a period of several seconds to several minutes whether there is a driver there or not. With careful fooling some can drive indefinitely.

Ignore #2, the car doesn't care. Though I still think having to actually insert a physical key, turn it, and operate some of the mechanically-stiff brake/shifter/steering wheel lock functions in some cars does essentially ensure that the driver is at least somewhat trained and likely older than 10 years old. Simply having a key card device somewhere near the car, shutting the door, clicking the belt, and touching the pedals is too easy in my opinion. You should have to do some actions that are difficult and require some strength to get a car moving. But that's not the direction we seem to be going in.

#3: Steering wheel nudges on cue and other wheel sensing, if effective, at least shows there is a driver responding to the nudges. Eye tracking and other sleep sensors, if effective, show that the driver is at least looking forward (even if they are watching a movie on the iPad on the dashboard). These systems are currently ineffective as they can be fooled too easily.

#4: I don't know how you can judge if someone is actually watching what is going on. Perhaps there could be an eye-tracking sensor tied to a projected blip on the screen. You have to look at the blip and follow it briefly. Randomly timed and randomly positioned but no more than a few minutes apart. Periodically the car could announce itself as dropping out of autosteer or autospeed and the driver has to drive for a period of time. Maybe the driver has to press a randomly specified touchpad or control. Mix it up. Do it often, but not during any intersections or driving maneuvers so as not to endanger the drive. Would any of this actually ensure the driver is paying attention though? Perhaps randomly startle the driver by simulating a rumble strip, nothing like a dose of adrenaline to make you pay attention.

#5: Driving while fooling the systems, or allowing unlicensed people/pets to take the car would have harsh penalties akin to the worst driving offenses so people are deterred from doing it.
#1 A camera, with software for human detection. You know, similar to what Tesla has.
#2 No. Just no. Please. No.*
#3 See #1
#4 See #1
#5 That's already true.

* Unless you want to partner with every state in the US + every country that has licensed drivers + every state of every country that issues state licenses instead of federal. And territories. And you also build cars that people would want, with that technology. In short, in case #2 wasn't clear: No.
 
  • Like
Reactions: APotatoGod
uhm, what?

gee, when we go out of our way to defeat the known seat weight sensor, it detects that there is weight there. amazing. science is wonderful, isn't it?
I think you are misunderstanding that part. They are talking about putting a weight on the steering wheel (helm), not the seat. None of the cars cared about having weight on the seat.

C&D mentioned this specifically, they said the cars had passenger airbag sensors, but not drivers side seat weight sensors. However, I'm not sure this is true of all cars, I know my Model 3 definitely has a seat occupancy sensor on the driver's seat given it knows to turn off the screen if I'm off it and parked for a while (and turn it back on if I put some weight on the seat), but apparently it doesn't use it for AP. I guess it's perhaps to eliminate false positives/negatives (like people shifting weight on seat).
 
I don't quite get this test. These systems are designed to support a responsible, normal driver, not to act like some secret service guy who is always looking over someone's shoulder.

There are two issues:
1) Drivers will get complacent over time, especially when the ADAS appears to be "almost self-driving". Without reliable driver monitoring, you risk accidents where the driver fell asleep or was distracted and was not able to intervene in time to prevent an accident when the ADAS screws up. We've seen that happen with Tesla's AP with deadly results.
2) You can't assume that all drivers will be responsible. Some drivers will get annoyed at the nags or the driver monitoring, especially if they think the system is self-driving when it is not, and may try to disable the driver monitoring. We've seen this happen with Tesla where drivers use the steering wheel weight to defeat the torque nag because they want the benefits of "hands-free" and don't want to be annoyed with the nag. This too can be bad because if you disable the driver monitoring, then it won't be able to keep the driver alert. So the driver may get distracted or fall asleep and the driver monitoring won't alert them in time to prevent an accident.

These are all important issues because L2 ADAS requires an attentive driver. They are not able to self-drive without human intervention. So you need to keep the driver attentive and engaged in order for the L2 to be safe and effective.
 
  • Like
Reactions: Dan D.
#1 A camera, with software for human detection. You know, similar to what Tesla has.
#2 No. Just no. Please. No.*
#3 See #1
#4 See #1
#5 That's already true.

* Unless you want to partner with every state in the US + every country that has licensed drivers + every state of every country that issues state licenses instead of federal. And territories. And you also build cars that people would want, with that technology. In short, in case #2 wasn't clear: No.
How would you like the camera to respond to a driver who is wearing dark/mirrored sunglasses and a mask? A driver who is truly ocused but does not appear to move at all in minutes? A fully dressed manikin placed there which fools the camera?

It would be nice if Tesla's camera works, how do you see it working? Are they going to do that?

Level 2 cars may be one of the riskiest products ever launched. Well, really it depends on whether they can successfully ensure the car and driver are able to safely arrive at their destination.
 
How would you like the camera to respond to a driver who is wearing dark/mirrored sunglasses and a mask? A driver who is truly ocused but does not appear to move at all in minutes? A fully dressed manikin placed there which fools the camera?

It would be nice if Tesla's camera works, how do you see it working? Are they going to do that?

Level 2 cars may be one of the riskiest products ever launched. Well, really it depends on whether they can successfully ensure the car and driver are able to safely arrive at their destination.

I think the only part of your message that the video doesn't respond is the manikin. Which in reality is so absurd and long tail that if it was up to me, I'd say "have at it" 🤣 but camera + wheel nag would probably defeat it, too.
 
Fooled by a photograph. Fail.


View attachment 695876
How can you see it's a photograph? With... Vision? What a coincidence. That's exactly what a camera uses. I wonder if software can fix that.

EDIT: Ironically, this was your idea to begin with. Eye sensors are cameras.

#4: I don't know how you can judge if someone is actually watching what is going on. Perhaps there could be an eye-tracking sensor tied to a projected blip on the screen. You have to look at the blip and follow it briefly. Randomly timed and randomly positioned but no more than a few minutes apart. Periodically the car could announce itself as dropping out of autosteer or autospeed and the driver has to drive for a period of time. Maybe the driver has to press a randomly specified touchpad or control. Mix it up. Do it often, but not during any intersections or driving maneuvers so as not to endanger the drive. Would any of this actually ensure the driver is paying attention though? Perhaps randomly startle the driver by simulating a rumble strip, nothing like a dose of adrenaline to make you pay attention.
 
Last edited:
  • Like
Reactions: APotatoGod
I think you are misunderstanding that part. They are talking about putting a weight on the steering wheel (helm), not the seat. None of the cars cared about having weight on the seat.
oh yes they do! haven't you noticed on your car? I first found this out while in a drive thru fast-food and while lifting up to get my wallet, the car saw the seat sensor show 'no weight' and the car went into 'oh the driver is NOT there!' mode.

its very common. when our techs had to do testing, we had ball weights in the garage and it was so common to put weights on seats to keep the car 'happy' ;)

but there is a reason to have some sensor there, but not go nuts about it. sure, you can game systems and fool them, but is that a good use of money, to design an unfoolable car seat? no, I dont think so. good enough is good enough. the point is to have some sanity check features and that's what this is about.

unfair for tests to give a bad rating if they can fool a seat via trickery. that's not a valid use-case. it should be tested and reported, but no number should be higher or lower based on that - its just a description of how one vendor senses occupancy. as long as it does the basics, that's fine.

there are many other things where splitting hairs actually matters. this isn't one of them.
 
  • Like
Reactions: APotatoGod
Ignore #2, the car doesn't care. Though I still think having to actually insert a physical key, turn it, and operate some of the mechanically-stiff brake/shifter/steering wheel lock functions in some cars does essentially ensure that the driver is at least somewhat trained and likely older than 10 years old. Simply having a key card device somewhere near the car, shutting the door, clicking the belt, and touching the pedals is too easy in my opinion. You should have to do some actions that are difficult and require some strength to get a car moving. But that's not the direction we seem to be going in.
9 year old (with her 4 year old sister) knew how to steal keys and drive 10 miles before crashing: 9-year-old Utah driver, her younger sister were headed to the beach when they crashed
 
  • Funny
Reactions: APotatoGod
oh yes they do! haven't you noticed on your car? I first found this out while in a drive thru fast-food and while lifting up to get my wallet, the car saw the seat sensor show 'no weight' and the car went into 'oh the driver is NOT there!' mode.

its very common. when our techs had to do testing, we had ball weights in the garage and it was so common to put weights on seats to keep the car 'happy' ;)

but there is a reason to have some sensor there, but not go nuts about it. sure, you can game systems and fool them, but is that a good use of money, to design an unfoolable car seat? no, I dont think so. good enough is good enough. the point is to have some sanity check features and that's what this is about.

unfair for tests to give a bad rating if they can fool a seat via trickery. that's not a valid use-case. it should be tested and reported, but no number should be higher or lower based on that - its just a description of how one vendor senses occupancy. as long as it does the basics, that's fine.

there are many other things where splitting hairs actually matters. this isn't one of them.
I'm not saying there's no driver's seat sensor, in fact the part you commented out from my post explicitly said my Model 3 seems to have one!
C&D mentioned this specifically, they said the cars had passenger airbag sensors, but not drivers side seat weight sensors. However, I'm not sure this is true of all cars, I know my Model 3 definitely has a seat occupancy sensor on the driver's seat given it knows to turn off the screen if I'm off it and parked for a while (and turn it back on if I put some weight on the seat), but apparently it doesn't use it for AP. I guess it's perhaps to eliminate false positives/negatives (like people shifting weight on seat).

My point is according to the C&D test, none of the ADAS systems cared about having weight on the seat at all, AKA they didn't have to put weight on the seats to defeat them. The part about the weight was talking about putting one on the steering wheel (if you watch the video it's clear). Even if some of the cars use a driver seat sensor for other uses, they apparently are not using it for ADAS driver detection.

C&D said in the video those cars didn't have any driver's weight sensor, only a passenger one (which might be true for some of the cars).
 
Last edited:
How can you see it's a photograph? With... Vision? What a coincidence. That's exactly what a camera uses. I wonder if software can fix that.

EDIT: Ironically, this was your idea to begin with. Eye sensors are cameras.
I agree. If Tesla can use the driver-facing camera and software to determine that there is a real person in the driver's seat, not a photograph, and that the driver is awake and actually paying attention to the driving, not watching a movie on the phone in front of them, or using "internet glasses", in all light situations, and stopping safely otherwise, then that is a pretty good driver monitoring system.

I would like to hope they can.
 
I agree. If Tesla can use the driver-facing camera and software to determine that there is a real person in the driver's seat, not a photograph, and that the driver is awake and actually paying attention to the driving, not watching a movie on the phone in front of them, or using "internet glasses", in all light situations, and stopping safely otherwise, then that is a pretty good driver monitoring system.

I would like to hope they can.
I'm fine with what they have today. I'd even take less. The thing is, for every mechanism, there's always a circumvention.
There are tens (if not hundreds) of millions of VERY creative people in this world. To thing that a one single safety device would beat them all is naïve.

Technically speaking though, what you said can most likely be achieved with high accuracy - or confidence level, in NN lingo.

The question is: are you the kind of person who takes improvements over nothing? Or is it only "all or nothing"?
 
  • Informative
Reactions: Dan D.