Consumer Reports: Latest Autopilot “far less competent than a human”

[diplomat33]

Just wanted to share my most recent NOA experience I had today.

Auto Lane Change without confirmation worked smoothly to pass a semi truck. Both the lane change into the left lane and then the lane change back into the right lane after I passed the truck were smooth. The driving part on the highway was great.

When I got to my exit, NOA did suggest 2 incorrect lane changes on the screen. First, about a second before taking the exit, NOA suggested a left lane change (?) which would have moved into the left passing lane not the exit lane on my right. Then about a second after NOA steered me into the exit lane, NOA again suggested a left lane change (?) which would have taken me back on the highway instead of staying in my exit. Now, NOA did steer me correctly into the exit lane and correctly steered me in the off ramp, it just suggested bad lane changes but did not do them.

It is worth noting that this exit had some serious construction work not too long ago. In fact Google Maps still shows the old exit. So I suspect this is a maps issue, not a NOA issue per se. If the map is off, it would explain why NOA was recommending incorrect lane changes. I definitely have not had this experience with other exits.

Hopefully, if Tesla is able to move to a more vision based NN with AP3, that will solve this issue.

 

[gnuarm]

It is worth noting that this exit had some serious construction work not too long ago. In fact Google Maps still shows the old exit. So I suspect this is a maps issue, not a NOA issue per se. If the map is off, it would explain why NOA was recommending incorrect lane changes. I definitely have not had this experience with other exits.

Hopefully, if Tesla is able to move to a more vision based NN with AP3, that will solve this issue.

That is possible. I've had the experience of the car changing the speed limit from 55 to 25 at a new overpass for months after it was complete because during the construction they used the new ramp lanes as by-passes at 25 MPH. I still don't know if the maps have caught up, but it was a bit scary when the car would do a hard speed reduction for no apparent reason on a highway. There are some other, similar locations where it was finally resolved after a couple/three months.

 

[EVNow]

Hopefully, if Tesla is able to move to a more vision based NN with AP3, that will solve this issue.

You mean exit by actually reading the signs like we did in the 90s ?

 

[dgatwood]

Except in countries where you drive on the left side ;-).

Meh. They're not part of Europe anyway.

Too soon?

 

[diplomat33]

Good video that points out how CR's video is "fake news":

 

[dhrivnak]

We just returned from a 800 mile road trip and my wife asks that I do not use the AutoPilot and she never did. She says I am a noticeably better driver and I have to agree. Thrice the car did a brake check when none was warranted. When lanes merge onto the Interstate the car wanders far to the right causing a later correction. While I use it and I do think it helps I would never use it unsupervised , closely supervised.

 

[Watts_Up]

We just returned from a 800 mile road trip and my wife asks that I do not use the AutoPilot and she never did. She says I am a noticeably better driver and I have to agree. Thrice the car did a brake check when none was warranted. When lanes merge onto the Interstate the car wanders far to the right causing a later correction. While I use it and I do think it helps I would never use it unsupervised , closely supervised.

I certainly had to intervene at some point when using EAP, however I also received warnings,
like steering wheel vibrating and felt more secured as a result.

I try to review my dash cam to understand the issue and learn from it.
Recently a car following me decided to pass me while I had already indicated that I was going to change lane,
or a car cut me in front of me while zigzagging between lanes to pass other cars.

 

[acoste]

Good video that points out how CR's video is "fake news":

Good?

not so much....

CR says the car cuts in front of others but they didn't show a proof of it on a video. This guy says because they didn't show it, it must be fake. Now that's not an acceptable argument for me.

2nd. They come up with Tesla's report that Autopilot is safer than non AP. Tesla's report has no merit as they don't disclose any data how they come up with that assessment. Given Elon's history I don't give any faith to anything they report.

3rd. CR says Autopilot makes trips not relaxing. Well. You may be relaxed when using it but this guy won't be any more:
Tesla Model 3 Brake Checks Semi Truck on Autopilot: Video

Also Teslas are not new to CR. They bought the first iteration of 85D. Later on they bought the P85D as soon as it came out. And so on.

 

[diplomat33]

CR says the car cuts in front of others but they didn't show a proof of it on a video. This guy says because they didn't show it, it must be fake. Now that's not an acceptable argument for me.

What is an acceptable argument is that there are plenty of youtube videos that show this not to be true. Perhaps CR did not provide any video evidence precisely because there is none, because it is not true? And we know that when a car is in your blind spot, NOA will always indicate that a lane change is not permitted by showing a red lane line and showing the car as red, and NOA will NOT perform the auto lane change in this instance. That's further evidence that NOA does not cut in front of other cars.

What I think happened, is that NOA performed auto lane changes that CR clearly did not like and they are exaggerating a bit.

2nd. They come up with Tesla's report that Autopilot is safer than non AP. Tesla's report has no merit as they don't disclose any data how they come up with that assessment. Given Elon's history I don't give any faith to anything they report.

Tesla's safety report may lack some detail and context but the numbers are accurate.

3rd. CR says Autopilot makes trips not relaxing. Well. You may be relaxed when using it but this guy won't be any more:
Tesla Model 3 Brake Checks Semi Truck on Autopilot: Video

Relaxation is really a matter of confidence and knowledge in the system. There are plenty of owners who know how NOA works and are used to using it, so NOA is more relaxing to them. Obviously, if you are inexperienced with using NOA or don't know how NOA works or NOA does not drive the way you like, I can see how NOA might be less relaxing under those circumstances. CR is giving their opinion. There are plenty of Tesla owners who disagree.

And being relaxed and being attentive are not mutually exclusive. There are plenty of Tesla owners who are relaxed and attentive at the same time.

 

[acoste]

What is an acceptable argument is that there are plenty of youtube videos that show this not to be true. Perhaps CR did not provide any video evidence precisely because there is none, because it is not true? And we know that when a car is in your blind spot, NOA will always indicate that a lane change is not permitted by showing a red lane line and showing the car as red, and NOA will NOT perform the auto lane change in this instance. That's further evidence that NOA does not cut in front of other cars..

How about here? 16.2 Near Collision w/ Auto Lane Change
Car didn't see a faster approaching car until later. This guy is relaxed as well.

 

[acoste]

Tesla's safety report may lack some detail and context but the numbers are accurate.

Here are two numbers : 5 and 6. Super accurate.
But what are they representing? Who knows.

 

[diplomat33]

Here are two numbers : 5 and 6. Super accurate.
But what are they representing? Who knows.

You are being purposely facetious. Here is the Q1 2019 report again:

In the 1st quarter, we registered one accident for every 2.87 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot, we registered one accident for every 1.76 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 436,000 miles.

I think it is very clear what the numbers represent!! That's a bit more than just telling us 5 and 6.

 

[acoste]

You are being purposely facetious. Here is the Q1 2019 report again:

In the 1st quarter, we registered one accident for every 2.87 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot, we registered one accident for every 1.76 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 436,000 miles.

I think it is very clear what the numbers represent!! That's a bit more than just telling us 5 and 6.

Were those nonAP miles driven under the same conditions as the AP miles?

 

[brianman]

I know what you mean, but sometimes I wonder if we all wouldn't be much better off just driving the speed limit. Everyone would be driving the same speed and I can't believe there wouldn't be a low fewer accidents, not to mention the end of road rage. No?

A does not imply B, nor C or D.

 

[diplomat33]

Were those nonAP miles driven under the same conditions as the AP miles?

Like I said, we don't know. That is why I said that the numbers lack detail and context. And yes, we can find fault with the numbers. But to equate "1 accident for every 1.76 million miles driven without AP" with "5" and "6" is incorrect. 5 and 6 are just isolated numbers whereas "1 accident for every 1.76 million miles" is a ratio with units (ie we know that we are talking about accidents per miles driven not apples per tree or cats per house). They are not at all the same thing!!

 

[Az_Rael]

How about here? 16.2 Near Collision w/ Auto Lane Change
Car didn't see a faster approaching car until later. This guy is relaxed as well.

Yeah, that is pretty bad. I agree with the CR report and documented behavior like the one in the thread just confirms my opinion of NoA.

 

[acoste]

Yeah, that is pretty bad. I agree with the CR report and documented behavior like the one in the thread just confirms my opinion of NoA.

Almost had a serious crash driving on AP

I wonder if Tesla will turn off the dashcam feature at some point.

 

[diplomat33]

Nav on AP handles construction zone with no right lane line:
Tesla Navigate on Autopilot seamlessly handles construction zone with no lane lines

 

[iTech]

Nav on AP handles construction zone with no right lane line:
Tesla Navigate on Autopilot seamlessly handles construction zone with no lane lines

What happens without the lead car?

 

[Az_Rael]

Nav on AP handles construction zone with no right lane line:
Tesla Navigate on Autopilot seamlessly handles construction zone with no lane lines

What happens without the lead car?

Looking at the AP screen, the car is in follow mode I believe (highlighted lead car plus no blue lane lines). Without the lead car, I am not sure it would have done that. AP1 also was excellent at no lane driving with a car to follow, so not sure this really shows anything NoA unique.

Now if he did it without the car ahead, then that would be super impressive. It is interesting the car recognized it was in a construction zone but did not disengage AP.