Driver Monitoring in 10.5

[Ruffles]

I wish I could get some kind of acknowledgement from Tesla. Other prominent YouTubers have expressed the same experience. I just don't know if I should focus on capturing the current issue which will result in 3rd strike and no more FSD or just switch it off until 10.6 comes in hope that its fixed.

 

[Phlier]

What ?!?

Beta users should expect & get white glove support. Thats how it worked in every product I've worked on.

Exactly.

I was a Tier 1 beta tester (unpaid external tester) for Symantec's Mac division (specifically, Norton Utilities for Macintosh, Symantec Anti-Virus for Mac, and for a very short time, Think C) from 1991 to 1996, and I had the phone number of the head of Symantec's QA, and the Lead Test Supervisor, as well as (of course) their email addresses. Phone calls and emails were answered promptly.

Of course, back then the number of testers was a lot lower, but even so, Tesla is doing themselves a huge disservice by not adequately staffing the beta testing department.

I hope the actual software development department is handled much better overall than how they're handling the beta.

Even if they just sent out a weekly email blast to all the testers informing everyone of what's going on, how it's going, what we should focus on, proper usage of the Report button, and maybe a FAQ.

But the silence is truly deafening.

 

[PDX-Y]

I had the phone number of the head of Symantec's QA, and the Lead Test Supervisor

This is an open beta, and we aren't "tier 1" anything. They did the "tier 1" part back in the spring when they handed access toa few hundred influencers. By my estimation there are 23000 cars currently enrolled. I get that's what you want, but that kind of service is entirely unreasonable to expect. No human being is looking at all the reports coming in, there are simply too many of them. This program is for detecting trends and edge cases.

No one is going to hold our hands over individual bugs. It's just not going to happen. Realistically they'll be auditing the AP lockouts and deciding if there's a problem they need to address, and if there is we'll see it appear magically in 10.6.

 

[Phlier]

This is an open beta, and we aren't "tier 1" anything. They did the "tier 1" part back in the spring when they handed access toa few hundred influencers. By my estimation there are 23000 cars currently enrolled. I get that's what you want, but that kind of service is entirely unreasonable to expect. No human being is looking at all the reports coming in, there are simply too many of them. This program is for detecting trends and edge cases.

No one is going to hold our hands over individual bugs. It's just not going to happen. Realistically they'll be auditing the AP lockouts and deciding if there's a problem they need to address, and if there is we'll see it appear magically in 10.6.

All good points, but (there's always a but(t), right?)

There are just way too many testers to have the kind of personal touch I talked about in that post, to be sure.

However, even if they just sent out a weekly email blast (that I suggested), it would go a long way in helping us be better testers.

Right now there's just nothing. They've pretty much proven that the email address they provided to the initial wave or two of testers is pointless to use. New guys don't even get an email at all!

There's a really, really good chance that this beta is going to go on for a long, long time. They could shorten it substantially if their testing program included a useful communication chain and providing us with the tools we need to be good testers, ie, "You have room for xx more snapshots," "You just over wrote your first snapshot", "snapshots currently uploading," etc.

 

[EVNow]

This is an open beta, and we aren't "tier 1" anything. They did the "tier 1" part back in the spring when they handed access to few hundred influencers. By my estimation there are 23000 cars currently enrolled. I get that's what you want, but that kind of service is entirely unreasonable to expect. No human being is looking at all the reports coming in, there are simply too many of them. This program is for detecting trends and edge cases.

No one is going to hold our hands over individual bugs. It's just not going to happen. Realistically they'll be auditing the AP lockouts and deciding if there's a problem they need to address, and if there is we'll see it appear magically in 10.6.

Yes 20k+ beta users is a lot - but I've worked on products with more than a million beta testers.

No - you can't obviously offer individual hand-holding to all - but you estimate the number of reports that are likely to come in and figure out ways to address those, build systems to track issues etc. 30 minutes each day by ~1k labelers would be enough to check out every report that comes in and categorize them.

Anyway a lot of companies have crowdsourced issue system - that are managed by a small number of employees and still effective. I think I've said this many times - Tesla is a better software company than your average auto OEM and better at auto hardware than other tech companies - but their software practices / communication are nowhere near best in the tech world.

 

[PDX-Y]

No - you can't obviously offer individual hand-holding to all - but you estimate the number of reports that are likely to come in and figure out ways to address those, build systems to track issues etc.

Right, but none of that involves reaching out to individual users having troubling and trying to find them individual solutions. They'll fix it when they fix it.

 

[Double-O]

No chance I could even do it. I'd have my 3 strikes in the same day. Left elbow on door handle and left hand on steering wheel. 90% drive with just left hand on steering wheel. Right hand almost never on steering wheel.

 

[EVNow]

Right, but none of that involves reaching out to individual users having troubling and trying to find them individual solutions. They'll fix it when they fix it.

Right - but the problem is there is zero acknowledgement of any reports submitted and what is happening to those. Some kind of issue tracking here would be useful. Knowing what issues have already been acknowledge would help testers focus on other issues.

Yes, they would need to build a new team of 10+ for that - but it would greatly help the beta testers and the fsd team. Tesla is not exactly a struggling strartup now.

 

[EVNow]

No chance I could even do it. I'd have my 3 strikes in the same day. Left elbow on door handle and left hand on steering wheel. 90% drive with just left hand on steering wheel. Right hand almost never on steering wheel.

Whole Mars Catalog never has his hands on the wheel. Just tugs it when there is a nag.

If you keep a hand on the wheel and look at the road ahead and respond to nags - you would have 0 issues.

 

[Phlier]

In 10.5, I've noticed that I get a lot more "Pay Attention" cabin camera induced alerts than previous betas.

Apparently, this is fairly common for taller guys.

There's a popular FSD Beta YouTuber... Chuck Cook. He recently Tweeted to Elon that he's been having a lot of problems with 10.5 giving inappropriate "Pay Attention" cabin camera nags.

It appears that the common denominator for this is that it happens most frequently to tall guys.

I'm 6'1" and get it quite frequently, even if I'm looking straight out the windshield and "ghost steering," as I normally do.

In 10.5, you can tell what set off the nag, the steering wheel or the cabin camera by the type of nag you get. The steering wheel gives you the "apply slight force to steering wheel" nag we've all grown to know and love, while the cabin camera will give you a double aural ding and a "Pay Attention" dialog.

And yeah.. in 10.5, I'm getting the "Pay Attention" ding frequently, even though my eyes are glued to the road.

I'll see if I can find Chuck's Tweet (and the resulting thread) and post it here. I came across it in a Tesla related video, not on Twitter...

Edit: It turns out that Chuck has numerous threads about this on Twitter. There is some wheat to be had here, but also a ton of chaff:

https://twitter.com/x/status/1465644142298607620

Here's another one, but was way back on November 24th, before the tall driver part was figured out. Again, lot's of chaff with some wheat:

https://twitter.com/x/status/1463624612290383877

 

[Ruffles]

Thanks @Phlier . This is exactly what is happening to me even without a hat. I'm tall (6'2") and wear glasses. My two strikes have been when it is dark - early in the morning and late at night. My hope is that since I'm a nobody, Chuck Cook or Whole Mars Catalog or someone else with some clout can get Tesla's attention.

 

[JulienW]

No chance I could even do it. I'd have my 3 strikes in the same day. Left elbow on door handle and left hand on steering wheel. 90% drive with just left hand on steering wheel. Right hand almost never on steering wheel.

The interior camera wasn't designed to monitor steering wheel use and can't even see it. So other than torquing Tesla has no way of knowing if you have your hands on the wheel.

 

[Double-O]

The interior camera wasn't designed to monitor steering wheel use and can't even see it. So other than torquing Tesla has no way of knowing if you have your hands on the wheel.

View attachment 739882

Good to know.

Was only going off of comments about having both hands on the steering wheel, although my right arm usually rests on the center console which looks visible in that photo. Anywho, good info.

 

[top1105]

I just posted this in another thread. Posting here as well.

Something is either broken or changed with my car. Got 10.5 (2021.36.8.8) around 11/24. The first few days were fine.

Then suddenly yesterday, while staring at the road with my hands on the wheel I kept getting the “pay attention to the road” notifications on the freeway. Went to AP jail. Got a strike. Next trip, same thing. Now 2 strikes out of 3.

Extremely likely I will get suspended the next time I drive. No idea what I’m doing wrong. Even started mashing my drive stalk, jiggling the wheel, opening my eyes wider, making sure to blink regularly on my second drive. Just tried a reboot by holding down my steering wheel buttons incase that helps.

 

[PDX-Y]

Even started mashing my drive stalk, jiggling the wheel, opening my eyes wider, making sure to blink regularly on my second drive. Just tried a reboot by holding down my steering wheel buttons incase that helps.

Did you try cleaning the cabin camera?

 

[Ruffles]

I just posted this in another thread. Posting here as well.

My experience as well. I've tried everything I can think of to help the interior camera "see" me paying attention. It looks like 10.6 is coming soon so I'm going to switch off beta to avoid my 3rd strike until I hear how its working for others.

 

[top1105]

Did you try cleaning the cabin camera?

Definitely tried that.

 

[top1105]

I just posted this in another thread. Posting here as well.

I just took a 3 hour trip at night. Did a ton of testing and found out:

• Wearing sunglasses fixes the issue. As long as the car can’t see my eyes. So I wore sunglasses for most of my road trip at night. Funny how this safety feature made me do an unsafe thing so I could get autopilot to work on a long road trip.
• If AP displays the pay attention to the road notification, cancel AP and re-engage. This resets the AP jail counter.
• Came to the conclusion that whatever changes they made, my car is now racist. I’m typical Asian with small eyes. I think the driver monitoring is giving me these issues because it can’t see my eyes. Had my white friend drive for 15 minutes, no issues. I get the notifications within seconds. If I open my eyes abnormally wide, no notification either. Just dry strained eyes.

Will see if 10.6 resolves it.

 

[PDX-Y]

Came to the conclusion that whatever changes they made, my car is now racist. I’m typical Asian with small eyes. I think the driver monitoring is giving me these issues because it can’t see my eyes. Had my white friend drive for 15 minutes, no issues. I get the notifications within seconds. If I open my eyes abnormally wide, no notification either. Just dry strained eyes.

Not that I would exactly recommend that sort of thing, but if you get clear video of this the internet would absolutely mob you with impressions, upvotes and start a giant luddite meme war over this. I'd genuinely be curious to watch, and I'm generally a ML/AI booster. But I could totally believe that in a first cut of a system like this the training set was heavily modal.

For the record: I'm a middle aged, typical height white guy who's had no trouble with the monitoring in 10.5.

 

[reven]

Got nailed twice last week on a road trip with the wife. She was driving and both of us were looking at the road when it happened. Her hands were on the wheel and she wasn't playing with her phone or the screen. Touchscreen started alerting for her to apply force, so she jerked the wheel then the red alert went off that she wasn't paying attention. Best I can think of was that the sun was low so it was some sort of glare. But it did it twice within the hour so I had her leave FSD disabled for the rest of the drive that day, which sucks because even dumb autopilot on a long trip is nice to have.
I've been tempted to email FSD team about it but I'm sure they get so many complaints about being kicked out they wouldn't care. I'm all for making it extra strict but if the system screws up we shouldn't be booted.