A Public Letter to Mr. Musk and Tesla For The Sake Of All Tesla Driver's Safety

[RogerHScott]

This is an example of how "half-cancel" of the automation is bad.

In Pang's specific case, it might not have made a difference, but the more I think about it, the more I believe that while allowing drivers to partially engage the automation is totally reasonable, any transition that can partially disengage it is dangerous. While it might be useful to some drivers in some conditions, nonetheless it is overly likely to cause driver confusion about the current control state of the vehicle.

Even the brief decrease in speed that results from dynamic braking if you cancel-all-the-way-out is immediately recognizable and important feedback that you are 100% in control of the vehicle -- not the automation. And it's feedback through touch, which is the relevant sensory mode -- not hearing, which is largely irrelevant. Feedback which continues (because you need to use the gas to maintain speed) for so long as you must fully manage the vehicle. This is important and prevents screwups. Partial cancel bad.

My wife has a strong background in human factors and is a very careful driver. We had what I would consider almost the ideal introduction to Autopilot, with about 3 months of non-AP operation of our Model S before the feature was turned on. We both carefully experiment with and observe the behavior of AP to ensure we're familiar with its limitations and can operate safely whether it's on or off.

Nonetheless, she's on at least two occasions that I've seen fallen victim to the mental mistake of thinking Autosteer was on when only TACC was on -- not because she'd engaged only TACC -- I've never seen anyone do that and get confused -- but because she'd disengaged only Autosteer, considerable distance had passed on a straight road (the Tesla sure does track nicely when properly aligned!), and she just instinctually thought Autosteer was still on. We have trained ourselves to always use a brake tap to cancel automation, whether TACC or steer. I think Tesla should make it always behave that way, whether a brake tap or wheel yank are the proximate cause of the cancel.

It sounds like you're talking about a somewhat different issue than the accelerate-up-to-TACC-set-speed-after-auto-steer-disengage
issue. I trust you don't love that, either. Some of us have advocated dropping that particular "feature", but what you're suggesting would
make that issue moot.

 

[RogerHScott]

I want to position the car differently in the lane. In this vein, Tesla has talked up AP's greater accuracy in holding to the center of the lane, as if this was a good thing. I think the best position in the lane is situational. It's slightly safer to bias away from the side with moving vehicles that can encroach unexpectedly, and toward the side of the lane that is the road's fixed edge or that only has stationary objects. That AP doesn't do this is a bug (albeit a small one) not a feature.

This seems like yet-another case of "it's just software" and as such a single, one-size-(doesn't)-fits-all choice should not be hard-wired.
Why not provide an option under auto-steer for lane position bias? Such a bias wouldn't override safety issues, of course, but to the
degree there's "choice" of where to (safely) position the car it would let the driver express a preference. Ideal would be to have a setting for no-adjacent-traffic and one for with-adjacent-traffic, though I think if most people could only have one it would probably be the latter.
I assume what people want is: if there is adjacent traffic on only one side, bias away from that side; if there's traffic on both sides (or
neither), thread the needle down the middle.

 

[tls]

You guys can flog me all you want, but I personally vote for geofencing AP and excluding two lane, undivided roads (for now). It shouldn't be used there anyway, so owners won't be losing any supported functionality. And it'll keep this kind of nonsense from occurring.

The problem is that inaccuracies in the map data would lead to sudden AP disengagement on roads where the driver's every reasonable expectation would be that it were engaged.

Don't believe me? Look at how critical correct speed limit data has suddenly become since Tesla implemented the speed restrictions for Autosteer. There is a road near me where Tesla incorrectly (and persistently -- I've reported the error via voice bug report at least 20 times) has the speed limit as 40 despite clear 55MPH markings. It is otherwise as suitable for AP as any two lane road could be: good sight lines, only gentle, constant-radius curves, good markings.

I have repeatedly nearly been rear-ended entering that section of road as AP decelerates from 60MPH to 45MPH due to the persistent bad map data. Not cool.

Now, consider this: there is another persistent speed limit error I know of, on Interstate 95 about 5 miles west of the George Washington Bridge. There, the Tesla insists that the speed limit is 5 MPH. I cannot believe this error persists on such a heavily travelled road, but for months now, it has (again, despite multiple voice reports on my part and, I assume, others' as well). What if this were a road-type error in the database rather than a speed-limit error? You'd be sailing down Interstate 95 at 70MPH, probably with someone following way too closely (because someone always is, there), and suddenly AP would disengage, the car would go into heavy regenerative breaking, and you'd have the full cognitive load of speed management, steering, etc. all while some guy brakes like the Space Shuttle on landing trying not to ram you from behind. I cannot believe this would be safer.

If they're going to geofence like this they need to do it with a far gentler and more progressive series of warnings followed by eventual disengagement and refusal to reengage the automation. It is a much harder problem than many seem to think.

Or so it looks from over here in the "guy who used to build embedded things for a living" seat.

 

[Topher]

First, let me say, "dems some yoouge cones". Thanks for posting the video.

that I shouldn't be using it in a construction zone

Probably not. But if you are paying close attention, and are willing to accept the responsibility and consequences, go ahead. I am going to give you a hard time about driving 75 in a construction zone. Those people have families.

That said, when viewing the video, just watching it, it doesn't seem quite as sudden as it actually was.

Nope, if the video disagrees with your memory, everyone should be believing the video.

The video makes it look like a sedate little drift over, but consider it moves about 4 - 6 feet to the right in the span of less than 100'.

It was not 4-6 feet, it was a foot or so, according to the objective impartial video recording. The cone you hit was in your lane. The slip to the right seems (to me) to be caused by hitting the construction transition between the pavement sections. My car occasionally does this too, and I don't even have auto-steer.

It could have been more serious, sure - but if it was something other than plastic traffic cones, I probably wouldn't have allowed AP to have control at all. But the fact remains, AP failed fairly spectacularly in this instance.

Autosteer is designed to keep you in the lane (which it did). Collision avoidance is designed to keep you from hitting *cars* on either side of you (which it did). It is not designed to notice and miss traffic cones in your lane in construction zones.

but what if there had been a car there or something? It would, at the very least, have scared the bejeebers out of that driver and possibly caused an accident, even without contact.

If circumstances were different, it would have done something else. This like saying, you hit a bug, what if it had been a child.

Conclusion: Auto-pilot is not up to the task of avoiding traffic cones, in the driving lane, at 75 MPH. That doesn't appear to be within the specs as they currently exist, and it shouldn't be asked to do so.

The shift to the right is sudden and coincides with a change in surface, my opinion is, that it was caused by that change in surface, not a decision by auto-steer to move to the right (low confidence in that conclusion).

Thank you kindly.

 

[RogerHScott]

I am going to give you a hard time about driving 75 in a construction zone. Those people have families.

The workers, themselves, being dispensable?

 

[Electroman]

Naonak: I watched your video and I am shaking my head: Why would you not take control when you notice the cones are moving closer and closer and they come right on top of the lane marking itself? In fact one cone seems to be an inch inside the lane itself. If I were to drive manually in that situation, even then I would NOT drive at 75 but reduce the speed to 65 or so and move away from the cones. Essentially this is a condition that would require a lot of attention even when driving manually. I am sorry, driving in AP at 75 mph in those conditions to me is silly and stupid

When I look at all the videos from various folks who have ended in trouble I can only do a face-palm and start wondering what makes these guys try to force and drive on AP on those less than good conditions? Are they trying to flesh out all edge conditions to see where AP will fail? I am not in Tesla's testing team to do all that.

My rules are very simple: I will engage AP only in those conditions where driving manually is also a breeze. If the markings are not good, if there are cones on the lane, if there is construction going on, if there are bots - I am not engaging AP. And you know what, those are only less than say 5% of road conditions give or take, and it is not big deal if the current version AP does not drive me around on those conditions. If it does more than 90% of my highway driving, that is more than enough. I don't need to push the envelope.

 

[tls]

First, let me say, "dems some yoouge cones". Thanks for posting the video.
It was not 4-6 feet, it was a foot or so, according to the objective impartial video recording. The cone you hit was in your lane. The slip to the right seems (to me) to be caused by hitting the construction transition between the pavement sections. My car occasionally does this too, and I don't even have auto-steer.

This has me wondering again about what the software is prone to do when its confidence level is low. Does the slip to the right due to the road surface change introduce enough uncertainty that the system becomes unstable and cannot reliably select the correct steering position to avoid those cones over the incorrect one which actually takes the car into them?

Handwaving wildly to avoid the technical language of automated reasoning, there is always going to be some "noise floor" into which the current state can sink such that the probability of success if you take the right decision D cannot be distinguished from the probability of success if you take the very wrong decision D'. Are there cases where unusual road conditions (like the prior series of hills and curves in the example I posted a few minutes ago, or the pavement change here) can lead Autosteer into this problem and result in a small swerve out of lane rather than a smooth correction to stay in it -- without triggering the bright red beeping indeterminacy warning? Maybe so.

 

[Topher]

This seems like yet-another case of "it's just software" and as such a single, one-size-(doesn't)-fits-all choice should not be hard-wired.

Well I think eventually the 'just software' needs to be, and will be making those decisions. In the mean time, feedback from the human trainer is important.

There is a major road near here that I have been meaning to video, and send to Tesla. The locals all drive 1.5 feet onto the breakdown lane, because the surface in the middle of the lane is so horrible. Perhaps the current auto-pilot shouldn't do that, but eventually that sort of behavior will need to be included. I noticed last night, that DOT is repaving that section, so I missed my chance.

Thank you kindly.

 

[msnow]

I have a very similar view against continuous hands on the wheel with AP on, although in some situations I have one hand on or have both hands hover over the wheel. I admit this may make me sound a little senile, but a key reason is that I never want to become confused, even for a half second, about whether AP is on or not! The way I drive, if both hands are on the wheel in driving position then I know automatically that AP is not on and I have to steer. Another way of putting this is that if you have hands on the wheel in driving position while allowing AP to steer, you in effect are trying to unlearn the behavior of having your muscle memory automatically drive the car whenever you are in driving position in the moving car. That doesn't sound like a good idea to me.

The other reason is that with both hands in driving position I also end up fighting autosteer, usually because I want to position the car differently in the lane. In this vein, Tesla has talked up AP's greater accuracy in holding to the center of the lane, as if this was a good thing. I think the best position in the lane is situational. It's slightly safer to bias away from the side with moving vehicles that can encroach unexpectedly, and toward the side of the lane that is the road's fixed edge or that only has stationary objects. That AP doesn't do this is a bug (albeit a small one) not a feature.

This is why I believe nags, along with the campaign to have both hands on the wheel, are actually a slight negative for safety.

Better get used to it because I would be very surprised if in the near future they don't *require* hand(s) on the wheel at all times or it turns off. Either Tesla will want to get in front of the regulators or will negotiate this with them to keep AP enabled.

 

[tls]

It sounds like you're talking about a somewhat different issue than the accelerate-up-to-TACC-set-speed-after-auto-steer-disengage
issue. I trust you don't love that, either. Some of us have advocated dropping that particular "feature", but what you're suggesting would
make that issue moot.

I know, I'm the one who started that thread. ;-)

Since I originally posted about that, the more I ponder on it and discuss it with my coworkers (I work in a tangentially related area where similar decision problems sometimes arise) or my "household expert" (a.k.a. my wife) the more I conclude that the TACC behavior on Autosteer cancel issue is just an artifact of the larger problem. It should not be possible to "half-cancel" the automation at all, and then that and other problems (like driver confusion, probably more dangerous really) simply could not occur.

Turn the automation halfway on? Fine. No problem there. Turn it halfway off, particularly by an "inferred" command such as a steering or brake input? That, in my personal opinion, is too risky and should not be possible.

 

[sillydriver]

The shift to the right is sudden and coincides with a change in surface, my opinion is, that it was caused by that change in surface, not a decision by auto-steer to move to the right (low confidence in that conclusion).

Watched the clip a few more times. You are right that the short section where the pavement color is lighter also coincides with the car's move to the right. I had only correlated it with the position of the barrier with the three boards.

I doubt the move was "not a decision by auto-steer to move to the right." The reason is that if the texture of the pavement by itself caused cars traveling straight to lunge to the right, then the construction site would be littered by broken cones and mirror parts from other cars.

It is possible that the look of the short section of lighter color pavement could have triggered AP to steer to the right, although my intuition still says the barrier was a more likely trigger. Perhaps a combination of the change in pavement color plus the presence of the barrier was the trigger, which is why incidents of this kind don't happen more often.

Just before it steered right, the car was well-centered by AP in its lane on the interstate. A moment later it's mirror hit the top of a cone that appears to be exactly over the lane line. In "interstate highway standards" in wikipedia, it says the minimum lane width for interstates is 12 feet. With the mirrors, a Tesla is 7 feet 2 inches wide. So the move to the right would have to be 2 1/2 feet or more to hit the mirror against a cone that appears to be centered right on the lane line.

 

[ohmman]

The problem is that inaccuracies in the map data would lead to sudden AP disengagement on roads where the driver's every reasonable expectation would be that it were engaged.

I disagree. You're making an assumption about the implementation - that sudden AP disengagement would occur. I see at the end of your post that you suggest differently, but why wouldn't that be the original implementation? Here's a simpler way to start - don't allow autosteer to be engaged on prohibited roadways. That's what I was suggesting anyway, and it would have kept the original issue in this thread from occurring.

Surely there would be challenges, and I don't think anyone is suggesting there wouldn't be. Most of us here have built a few things in our lives. We know how it goes.

 

[kirkbauer]

Based on the content of Pangs letter I trust Tesla's logs to his memory. The one part that is consistent is that I think Pang said he went back to the car after running away, opened the door and put it in park and the "engine" stopped. That's consistent with Tesla's statement.

Tesla's statement doesn't make sense either. They are saying that he left the vehicle without opening the driver's door (so he exiting through the passenger door perhaps?). They are saying the the vehicle wasn't park so creep mode was turning the motor. Doesn't that mean the car would be moving? So they both exited the vehicle from the passenger side door while it was creeping, and then he ran back and opened the driver door and put it in park?

Additionally, and I know this from experience; at least on my MS, if the vehicle is going very slow and you lift out of the driver's seat, the vehicle shifts into park, even without opening the door. So I'm still trying to figure out how the vehicle did NOT shift to park when the driver left the vehicle. Neither Tesla nor the drivers' accounts make sense on that front.

I guess the only thing that makes sense is the driver never left the vehicle until after he put it into park. I'm curious if Tesla has logs for the driver seat pressure sensor or not.

 

[kirkbauer]

There was only a small portion of faded lines right before impact.

I'd use AP in that situation too.

The only reason I'd use AP in that situation is because the cones likely won't cause any real damage to the vehicle. If those cones were concrete then no way would I use AP. When there are obstacles right on top of one of the lane markings there is a real chance AP will swerve into them quicker than many people could react to stop it. I have had AP slightly go over the lines on numerous occasions (and it is funny because the lane departure warning goes off when AP goes over the line!).

 

[MP3Mike]

Additionally, and I know this from experience; at least on my MS, if the vehicle is going very slow and you lift out of the driver's seat, the vehicle shifts into park, even without opening the door. So I'm still trying to figure out how the vehicle did NOT shift to park when the driver left the vehicle. Neither Tesla nor the drivers' accounts make sense on that front.

But it doesn't shift to park if the seat belt is buckled does it? Maybe Pang wasn't actually in the drivers seat?

 

[Max*]

(and it is funny because the lane departure warning goes off when AP goes over the line!).

I've had that happen too.

 

[kirkbauer]

This seems like yet-another case of "it's just software" and as such a single, one-size-(doesn't)-fits-all choice should not be hard-wired.
Why not provide an option under auto-steer for lane position bias? Such a bias wouldn't override safety issues, of course, but to the
degree there's "choice" of where to (safely) position the car it would let the driver express a preference. Ideal would be to have a setting for no-adjacent-traffic and one for with-adjacent-traffic, though I think if most people could only have one it would probably be the latter.
I assume what people want is: if there is adjacent traffic on only one side, bias away from that side; if there's traffic on both sides (or
neither), thread the needle down the middle.

Obviously the MS can sense torque on the steering wheel. I wonder if it can detect the direction of that torque? Since my hands are supposed to be on the steering wheel anyways while using AP, it would be cool if I could coerce it towards one side of the lane or the other without disabling AP just by putting a little force on the steering wheel.

 

[rscott0]

As many have pointed out, this boils down to "he said she said." But it doesn't have to be.

If the driver feels that the fault lies with the Tesla, he should request the log files from Tesla. It's (almost) as simple as that.

The log files can show many details that Tesla knows, but neither we nor the driver know. They could be irrelevant, or very important. For example, at what point did the car use the steering wheel to determine the direction of the wheels (rather than AP)? Did the AP actually swerve, or was the car going straight but the road curved? When were the brakes applied? Where exactly did this occur? How much time lapsed from the time the brakes were applied until the car stopped? Was the motor running at tranny creep or faster? Were the wheels moving after the car stopped? Was the A/C on (possibly causing the noise that was heard)? Those questions can all likely be answered with the log files.

To get an idea of what information the log files could generate, you can take a look at http://evtools.info/display3.htm . That is an old page I made with data from a 2011 Chevy Volt, but lets you visualize the raw OBD2 data that the car generates. You can see at any point in the ~5 minute trips whether the accelerator and/or brakes were being pressed, the speed, location,
and SOC. There is a massive amount of other data it does not show. A similar page for a Tesla can be found at Moc Prototype CANBus data visualisation .

This is data that Tesla users can record if they wish (with a piece of hardware such as CANDue connecting to the car), so there should be no reason that Tesla cannot give a copy of the log files to the driver if he requests it (possibly excluding proprietary data, if it is explained what was removed).

If Tesla is confident that it was user error, and the user is willing to authorize the release of the log files, we could determine with much more detail exactly what happened.

 

[Topher]

I doubt the move was "not a decision by auto-steer to move to the right." The reason is that if the texture of the pavement by itself caused cars traveling straight to lunge to the right, then the construction site would be littered by broken cones and mirror parts from other cars.

That is not the only reasonable conclusion. The same result would occur if the auto-steer just corrected slower than humans do.

Thank you kindly.

 

[RogerHScott]

I know, I'm the one who started that thread. ;-)

It has come up in other threads as well. I hadn't even seen that thread until you mentioned it here.