Autopilot appears to turn car against incoming car

[Matias]

https://www.youtube.com/watch?v=MrwxEX8qOxA

Looks scary.

Apparently AP tries to evade tree shadow.

 

[thebishop]

Well, it is made for highway driving with hands on the wheel from on-ramp to off-ramp. Quite explicit. The use case in that video was with oncoming traffic, not a highway... I guess the temptation to use it for all kinds of situations is large, just as TACC - as it mostly works - not the best idea, more Darwin Award territory.

 

[Matias]

If you look at the release notes, they don't explicitly forbid using AP in that road.

 

[thebishop]

I guess they should write controlled-access highway to be crystal clear. I will definitely not use a beta feature with incoming traffic whenever it is actually release in Europe.

I wouldnt expect tesla to forbid anything, they can only inform what will work - the driver need to take responsibility - same thing with TACC, or even which speed to use. It is in the drivers hands to behave sensibly.

that video was anything but.

If if there are problems on a controlled-access highway (motorway) - then there would be cause for concern.

 

[Matias]

I haven't got access to AP yet (as I live in Europe), but I have to admit, if I haven't seen that video I would have definitely tried it with that kind of road.

Ps. I tweeted link that video to Elon.

Pps. I searched a definition for highway and apparently road in that video can be described as highway. But I think definition of highways is not the problem here.

 

[S4WRXTTCS]

From the video the lane-steering indicator goes from blue to off. So it likely lost tracking well before that point.

The user was just completely ignorant of the warnings the car was giving, and completely ignorant of the previous history just a few seconds before than where it was obvious the car wasn't tracking the lanes.

Furthermore I can't imagine trusting any kind of lane-steering system at this point on a roadway without dividers between the two directions of traffic. Sure I've tested it on similar roads with both hands on the wheel to the point where I could feel it glitch and correct it before anything happened.

The bottom line is it simply doesn't work reliably on those roads.

It's meant for hwy/freeway use.

Sure Tesla doesn't FORBID using it elsewhere just like Tesla doesn't programmatically require both hands on the steering wheel.

But, they do give plenty of guidance as what to expect.

Apparently though a certain percentage of the user base are complete morons who are going to ruin it for the rest of us.

Heck I thought the first issue with lane-steering was going to be someone falling asleep, but apparently it's going to be some clueless idiot using it in a way that it's not ready for. Or putting too much faith into it.

I can only imagine the lawyers at Tesla just cringing, and telling Elon Musk "I told you so".

 

[schonelucht]

Looks very scary. If that had caused a real accident and footage from inside the car would've been uploaded there is no doubt the public would see this as "autopilot caused the accident". Impossible then to get the message out that this guy was using AP in a way that is not supported by the manufacturer.

 

[Matias]

S4WRXTTCS, To my understanding AP thinks that tree shadow in road is a structure and tries to avoid that, you hear collision warning. Problem is not about indentifying lanes.

Where in that video you see warnings before that sharp avoiding maneuver?

 

[S4WRXTTCS]

S4WRXTTCS, To my understanding AP thinks that tree shadow in road is a structure and tries to avoid that, you hear collision warning. Problem is not about indentifying lanes.

Where in that video you see warnings before that sharp avoiding maneuver?

At the 9-10 second mark it does the red warning. I don't see any warning before than because it's covered by the steering wheel. But, I also don't see the lines on the UI to indicate it's tracking the lines. The video doesn't have any audio (at least not on my system) so I don't have any audible indication as to whether any indicator sounded.

 

[NoMoGas]

S4WRXTTCS, To my understanding AP thinks that tree shadow in road is a structure and tries to avoid that, you hear collision warning. Problem is not about indentifying lanes.

Where in that video you see warnings before that sharp avoiding maneuver?

the sun blocked the camera. It wasn't the collision warning but the I can't see take the wheel now alarm.

 

[S4WRXTTCS]

S4WRXTTCS, To my understanding AP thinks that tree shadow in road is a structure and tries to avoid that, you hear collision warning. Problem is not about indentifying lanes.

Where in that video you see warnings before that sharp avoiding maneuver?

You are correct though that it's likely the mobileeye system got tricked by the tree shadow. So it assumed there was something in the road. But, at the same time the system wasn't tracking lines visually or at least not that I can tell.

There have also been reports of TACC being tricked by shadows, and trying to slow down the car when nothing was there.

My friends Subaru has the same issue occasionally with shadows causing the adaptive cruise control to brake, but that system is ENTIRELY reliant on visual.

I'm not exactly sure what Tesla should do in the case where the lines are identified, and it knows where to go but it visually see's an object in the road.

With the current sensor set the options are pretty limited.

The Radar is for tracking moving objects.
Ultrasonic is too short range
The Camera is the only visual thing, but the mobileeye system can be tricked by shadows.

With TACC they made some changes that I believe improved the handling of situations with shadows. I'm not sure what they did exactly.

 

[Matias]

There is audio on that video.

At ten seconds mark car does show "take control immediately" sign and warning sound. It is possible there is a visual warning before that, true. We can't tell because steering wheel is covering that area in video.

I agree with you, that it is possible, that car showed a visual warning also earlier.

 

[Firewired]

Unfortunately this video is already getting seen by a wide audience. I came into work this morning and was told by a coworker who is a non-Tesla owner, to not use Autopilot. My co-worker asked if I had seen where Autopilot tried to steer into oncoming traffic.

 

[hobbes]

Probably didn´t help either that he was going over a hill with all sensors looking up. Definitely out of spec from what conditions Tesla said it was made for. Maybe they should actually limit the auto steering to those roads they specified (multi lane with divider), car can get the info from nav. Or not have it available on a narrow, curvy, up and down road with those lighting conditions. I think the Mobileeyle system should give some live feedback on its reliability at any time? Does anyone have technical insight into that?

 

[BerTX]

Best case -- the driver was warned and ignored the warning, then the system disengaged at the wrong moment. Worst case -- we can't see the driver's hands, or what action was taken, either intentional or not.

 

[Electricfan]

I think Tesla clearly intends autopilot be used on divided highway only. Definitely not in this case. Never use autopilot with oncoming traffic in the next lane over. Driver made a horrible mistake and would have been 100% responsible if he had caused head-on collision. But press would have blamed Tesla and autopilot. This makes me worry that somebody out there will not be lucky and survive his/her mistake as this guy did.

 

[Matias]

I think Tesla clearly intends autopilot be used on divided highway only..

See release notes which I posted first page. They don't make such disclaimer.

 

[muleferg]

Apparently though a certain percentage of the user base are complete morons who ruin are going to ruin it for the rest of us.

You got that right.

 

[Hodginator]

Apparently though a certain percentage of the user base are complete morons who ruin are going to ruin it for the rest of us.

You got that right.

This... Unfortunately the media also falls into the moron category.

 

[Matias]

I have to repeat my question. Where in release notes you see, that AP is not intended to be used in that kind of road?