Tesla Working on Driver Monitoring System

[verygreen]

Because currently some such systems don't provide the basic functionality needed to detect distraction.

I thought we already determined that that's impossible. Or I guess I don't fully understand what you mean by "basic functionality".

But in all these cases you need a system that can detect driver distraction and a variety of other states. Hence the need for improvement.

We already have such systems. Sure they are not perfect, but they do exist. Different systems have different "blind spots".

So the idea is, if you throw a bunch of nags that, if they keep being ignored, disable the driver aids when someone is heavily distracted, since they don't wish to die, and now no longer have the crutch they THINK makes it safe to be heavily distracted, they will stop being heavily distracted.


That's the idea anyway.

the problem is it does not work like this. While people do not want to die, theey want to get distracted. because they absolutely must take that phone call or reply to that text. they just must. If you disable ADAS - they'd still keep doing that (in fact people are doing it all the time on non-adas enabled cars too!) Some of them specifically bought the best ADAS their money could buy so they don't die. And now you think if you take that adas from them - they are safer? That that's the goal? I disagree with this notion. You are just driving people to drive unassisted when they need the assistance the most.

 

[AlanSubie4Life]

thought we already determined that that's impossible. Or I guess I don't fully understand what you mean by "basic functionality".

For the Tesla system specifically, I'm not aware that it provides all the necessary functionality in the common use cases. The line for "basic functionality" is a little fuzzy. But it's kind of a "I'll know it when I see it" metric. Clearly not there yet.

I don't understand why you say we determined that such a feat is impossible. Everything I've seen, including from the Tesla system (which has highly non-optimal visual capabilities), suggests to me that computer vision may well be at a point where it can detect the most common modes of distraction.

We already have such systems. Sure they are not perfect, but they do exist. Different systems have different "blind spots".

Yes. Of course some systems are better than others. Some may well be close to complete basic functionality (I don't follow them). The idea would be to largely eliminate the blind spots for use cases which are not actively being subjected to defeat attempt.

I'm sure we could make a list of the basic behaviors we'd like to detect, and I don't see too many technical limitations that would prevent that at the current time. But I'm not aware of production systems that do a really good job of that yet (I know Supercruise is ok, but I have no idea of which ways it can be defeated (does it detect a phone held in a hand or in front of the steering wheel? No idea; I don't pay any attention.)).

Seems like in the next few years the systems will be really good though. Would such a system be able to prevent someone from watching a movie on a phone mounted on the windshield? Probably not. But that's probably ok.

(As a specific example, having a system that forces a user (through trial and error) to hold the phone really low, out of view of the system, is not a good system, if it can't detect where the eyes are looking. That probably is forcing less safe behavior (it's better to have eyes up). Part of a solution would be to have a system that detects head tilt and eye movements. Or perhaps you have another camera with a wide-angle view (similar to the Tesla but perhaps better located high on the top of the A-pillar or something - redundancy might be needed?) which can see said phone and checks whether it is being held or manipulated? My understanding is that all of this is quite possible with computer vision, though I'm no expert on that! Just an example of closing a specific common problem. Is it necessary for a driver monitoring system to have this capability? Maybe not...though personally I'd lump that in the "basic functionality" bucket.)

It sounds like you're asking for a crisp answer here, but I don't think there is one. The idea is to make the systems better and resolve the obvious problems, not to make them undefeatable. You also want to avoid false positives and annoying problems (like wearing sunglasses causing problems, etc.), of course, since that will also increase risk of an accident.

 

[cgarnant]

same here. And if you want privacy keep a hand on wheel.

Problem with the 'hand on the wheel' - the torque sensor requires you to 'fight' the steering. For most of you this is not a problem - albeit you have to contend with near constant 'nagging' even if your hand IS on the wheel. I'd rather have some kind of pressure or inductance or ... sensor. Why, you ask? I bought the S for my daughter who has severe hand and wrist RA, solely because it was easier for her to steer (if she had to take over) than the M3 with the smaller diameter 'sport' steering wheel -- forcing her to have to fight the steering wheel every minute means icing her hands every evening for hours!

 

[erb2000]

Problem with the 'hand on the wheel' - the torque sensor requires you to 'fight' the steering. For most of you this is not a problem - albeit you have to contend with near constant 'nagging' even if your hand IS on the wheel. I'd rather have some kind of pressure or inductance or ... sensor. Why, you ask? I bought the S for my daughter who has severe hand and wrist RA, solely because it was easier for her to steer (if she had to take over) than the M3 with the smaller diameter 'sport' steering wheel -- forcing her to have to fight the steering wheel every minute means icing her hands every evening for hours!

Yes, the irony is overwhelming. I often use the Gonex ball, and when I keep my hand on the wheel it starts nagging. Without the ball it is as you describe, constant nagging. The end result is the opposite of what they are trying to achieve. Ironic.

 

[AdamMacDon]

Problem with the 'hand on the wheel' - the torque sensor requires you to 'fight' the steering. For most of you this is not a problem - albeit you have to contend with near constant 'nagging' even if your hand IS on the wheel. I'd rather have some kind of pressure or inductance or ... sensor. Why, you ask? I bought the S for my daughter who has severe hand and wrist RA, solely because it was easier for her to steer (if she had to take over) than the M3 with the smaller diameter 'sport' steering wheel -- forcing her to have to fight the steering wheel every minute means icing her hands every evening for hours!

Pro tip, just scroll the volume or speed scroll wheel up or down, this achieves the same effect as applying torque to the wheel, and far less effort for your wrist.

 

[doug]

I find at times the scroll wheel no longer works, and I'm forced to torque the wheel to satisfy the nag.

 

[GBMaryland]

I think I'll be putting a piece of black tape over that interior camera...


For the people discussing that the FSD has it's issues... I've had it drive into other lanes for an unknown reason related to changes in the lines that look completely normal.

 

[verygreen]

For the Tesla system specifically, I'm not aware that it provides all the necessary functionality in the common use cases. The line for "basic functionality" is a little fuzzy. But it's kind of a "I'll know it when I see it" metric. Clearly not there yet.

well, without clear definitions it's a bit hard to discuss if something meets a certain criteria

I don't understand why you say we determined that such a feat is impossible. Everything I've seen, including from the Tesla system (which has highly non-optimal visual capabilities), suggests to me that computer vision may well be at a point where it can detect the most common modes of distraction.

Unless the driver actively resists. hence the impossible.

Of course some systems are better than others

Well, here's where things get murky. How do we even define "better" in this context? who's going to decide some blindspot is less important than another?

The idea would be to largely eliminate the blind spots for use cases which are not actively being subjected to defeat attempt

why is that a cutoff? just because we know we cannot resist defeat attempts?

The idea is to make the systems better and resolve the obvious problems, not to make them undefeatable. You also want to avoid false positives and annoying problems (like wearing sunglasses causing problems, etc.), of course, since that will also increase risk of an accident.

and here and in the preceding paragraph you again concentrate on on what I perceive as wrong approach.

In my opinion you need to let the system know you are distracted, not hide this fact, because the system is there to help you when you are distracted and to also seek your attention if it needs your help, and that means let you know earlier if you are distracted. By hiding your state from the system everything becomes a lot less safe.

Once you create incentive to hide driver state from the system - you sort of lost.

 

[AlanSubie4Life]

well, without clear definitions it's a bit hard to discuss if something meets a certain criteria

I think you are being a little bit obtuse here. I’ve already provided my approximate bar earlier - have a system that can observe the driver at all times and identify the most problematic behaviors, similar to how a human would. I don’t think it is necessary to list all of those behaviors, for succinctness.

Unless the driver actively resists. hence the impossible.

Sure, but I clearly am not worried about those cases. See prior comments on this. I don’t think that is the main problem driver monitoring is trying to solve.

How do we even define "better" in this context? who's going to decide some blindspot is less important than another?

I guess we can point to specifics. Like if it doesn’t work in the dark that seems like a problem. I assume there would be a set of behaviors to identify under certain conditions and the systems would be judged and compared based on how it does against such metrics. Imperfect way to do it, but it is something.

and here and in the preceding paragraph you again concentrate on on what I perceive as wrong approach.

In my opinion you need to let the system know you are distracted, not hide this fact, because the system is there to help you when you are distracted and to also seek your attention if it needs your help, and that means let you know earlier if you are distracted. By hiding your state from the system everything becomes a lot less safe.

No. I was not talking about actions the system would take as a result of identification. I think you have a good point there about how to respond. There’s the sensing engineering problem, and then there is the social engineering problem and setting up appropriate incentives.

With your proposal, there is still a lot of sensing required - it should not rely on the user to tell the system when the driver is distracted. That can be an option of course. This does assume a certain level of autonomous capability that may not exist currently though. It may not be safe enough to operate with a distracted driver if it can’t identify tricky situations reliably.

 

[hellchris]

I, for one, welcome our new driver monitoring overlords

 

[verygreen]

I guess we can point to specifics. Like if it doesn’t work in the dark that seems like a problem

see, that's not easy. The car has a ton of lights inside. Is using those lights acceptable to remove the darkness? Note it does not mean you need to blast the lighting to 100%, some lower level would likely be enough for the camera to pick up needed info. In fact IR-based solutions already do this but in light that's supposedly invisible. But would a supposedly visible light be unacceptable for the same?

 

[AlanSubie4Life]

Is using those lights acceptable to remove the darkness?

Not in my opinion. I would be super upset even at a low level higher than what currently exists. They have night mode on the display for a reason, and no dash lighting at all is one of the best features of Model 3. I wear scleral contact lenses, and when they get a little dry glare is a problem.

But would a supposedly visible light be unacceptable for the same?

In my opinion that is unacceptable.

In my opinion this is not a difficult call, either.

Excellent imaging in near darkness is a mostly solved problem for stationary subjects, though. It’ll be interesting to see if Tesla uses something like Apple’s long exposure night mode (neural nets!) to overcome this issue, without IR. Obviously quickly moving subjects would not work, but that is not an issue here; would probably work. Apple’s night mode works with low levels of ambient light similar to what is coming off of the display (it does not work well in true darkness of course), so if it works for that I think Tesla can probably solve the issue with darkness (however, I should add that I do not know exactly what imaging hardware Apple needs - I am assuming here that the neural nets do all the work and the image sensor is nothing special - which may be incorrect and I am not going to look it up right now; the point is that it could work). I think it would likely even work with quickly varying levels of light at night.

Exciting! We’ll see.

 

[verygreen]

In my opinion that is unacceptable.

They have night mode on the display for a reason, and no dash lighting at all is one of the best features of Model 3.

There's a no light mode at all? I don't think I've seen that.

Basically when it's dark and car is unsure the can use something similar to current screen edge flashing to ask you to tug the wheel. They already do it anyway so it's not a regression or anything.

As for various exposure-blending tricks to see in the dark, that also might be possible of course.

 

[AlanSubie4Life]

There's a no light mode at all? I don't think I've seen that.

No but you can adjust the brightness level to be pretty low. And I'm assuming that level of light is not enough for the current cameras in pitch-dark conditions, based on what I've seen from your videos, without long exposure.

Basically when it's dark and car is unsure the can use something similar to current screen edge flashing to ask you to tug the wheel. They already do it anyway so it's not a regression or anything.

Sure, nothing wrong with that, but it would be better to just use the camera to do the long exposure (again, assuming the hardware is capable of implementing Night Mode - no idea) with the appropriate neural network backing to create an image of the scene (latency would be introduced in this case of course but that's the price you pay and it's probably acceptable to have 1-3 seconds latency in this application, though not ideal).

In future, I could see users get pretty annoyed when they get used to not torquing the wheel in daylight, and then being confused when they start to get pestered at night when they're paying attention to the road just like they were in the day, and then being forced to apply torque to satisfy the nag. So they would have to interact with the system differently when they're driving cross-country at night...annoying! This, in fact, seems like just the sort of problem that people here would quickly log on to report, and wail about endlessly.

 

[Dolemite]

Couple videos from @verygreen showing the system in action.

General discussion:

https://twitter.com/x/status/1379093474130198529

With detection overlay:

https://twitter.com/x/status/1379928419136339969

Night detection overlay:

https://twitter.com/x/status/1380238569030160400

What say you now, @Knightshade ? I assume (and it's probably correct) you'll say that it's not reliable enough. Though I do wonder whether it is actually pretty good at detecting the most common causes of driver inattention (meaning, an improvement over the torque sensor). The dark performance doesn't seem too good though! And if you slide out the visor you can block the camera...I guess just fallback to torque sensor in that case.

They're using this now, apparently, for driver monitoring, if you agree to enable it. HW3 only though; HW2.5 no support.

That poor girl.

 

[J1mbo]

but what's the end goal? To drive distracted drivers to disable driver assist? Really? is this a worthwhile outcome? Safe?

More than one end goal. One goal would be to make it inconvenient enough to disable/trick the system. Another would be to make sure that the criteria are not too narrow.... this little fella is clearly paying attention and applying torque to the wheel, so all OK, yes?

 

[CyberGus]

You have Q-Tips in your bathroom right now. We all do, and we all use them to clean our ears. But the package says "Do not insert into your ear", because you can be injured if done improperly.

So I would say that most safety-interlock devices exist solely to protect the liability of the providing corporation. With no interlock, accidents could be their fault; but if the interlock is intentionally defeated, then the accident becomes your fault.

 

[Knightshade]

But in this case the system is explicitly not safe without a driver paying active attention.

That's not a "failure" of the system- it's inherent to its design.

L2 system+distracted driver is every single AP death on record last I checked.[/B]

 

[CyberGus]

All vehicles are unsafe when the driver is not attentive.

 

[Knightshade]

All vehicles are unsafe when the driver is not attentive.

Hence why the EU is requiring driver monitoring systems on new cars.

Even when not using driver aid systems.