Navigate on Autopilot get worse on 2020.20.5

[diplomat33]

On Twitter, Green reported several big regressions on NOA on the latest 2020.20.5 version:
- Just finished a sudden unplanned 1475 miles roadtrip in my X (to NY and back). Installed 2020.20.5 and decided to give NoA a try to see how it improved since March when I last used it. TL;DR; it's even worse today, not sure when it got this worse.
- Now it super frequently performs a lane change away from exits "to stay on route", huh? conveniently this hides the annoying subsequent "wave in lane as it widens due to merge lane on the other side (shown at the end of this clip)
- New annoyance to lane changes when overtaking "to maintain speed" - now it randomly decides that "nah, I did not really want to do it anyway" and aborts, might repeat it a few times until you manually ask it to overtake.
- Auto high beam is pathetic nowadays somehow, it flips it on and off based on various reflections from signs and whatnot. Also tends to blink many times at trucks. No idea what those drivers think of me.
- at a start of one of 2.5 hour segments it decided it hates me completely so I had to drive it 100% manually, not even a cruise control... Why is not there a button "reset everything" for cases like this?

See full thread here:
https://twitter.com/greentheonly/status/1269967821297065985

Yikes! I wonder if this is related to the FSD rewrite. Like with previous rewrites, can we expect things to get worse before they get better again?

 

[powertoold]

As long as there's no safety issues involved, I don't see it as a "regression."

Certain driving behaviors are annoying to one person but acceptable for another. For FSD, it's not fair to say something is a regression simply based on your driving, lane, or speed preference.

Also, I don't see anything in these that are indicative of the rewrite.

 

[thewishmaster]

Sounds pretty par for the course to me. Granted, I haven't tried it much in the last few months, so I'm not sure if there was some leap in ability in the prior few versions

 

[mtndrew1]

Green, while clearly an incredibly gifted contributor, is consistently pessimistic in his Twitter feed.

I always take his crankiness with a grain of salt but really enjoy his technical contributions which give such great insight.

 

[diplomat33]

As long as there's no safety issues involved, I don't see it as a "regression."

Certain driving behaviors are annoying to one person but acceptable for another. For FSD, it's not fair to say something is a regression simply based on your driving, lane, or speed preference.

They are still regressions if the new behaviors are considered worse than the previous behaviors. And I grant you that they are not safety issues but I think they are certainly undesirable behaviors.

The car making unnecessary lane changes into the passing lane because it is confused by exits, is not a good thing. Although it does appear that maybe Tesla has not solved the issue with widening lanes at on ramps and also is not sure about merging traffic from on ramps, so to play it safe, Tesla changed NOA to move over to the passing lanes for exits by default. That's certainly one way to handle the problem.

And deciding to change lanes into the passing lane to pass a slow car and then aborting repeatedly, is not good behavior either.

And Green's last complaint was when AP suddenly stopped working completely and gave him this series of notifications.

He had to drive manually because he could not use AP at all. Surely, that is a bad thing, right?

On my current version, 2020.16.2.1, I've definitely observed what I consider to be regressions because NOA used to be able to handle 2 exits near my house and now, NOA does not take the exits at all anymore.

Also, I don't see anything in these that are indicative of the rewrite.

I merely asked the question about whether the rewrite might have caused some temporary regressions. Also, to be clear, I am not suggestions that these regressions are permanent. I am sure Tesla will fix them at some point. I am merely reporting what Green observed on his road trip. He clearly observed behavior that he calls regressions on this particular sw version 2020.20.5.

 

[mtndrew1]

They are still regressions if the new behaviors are considered worse than the previous behaviors. And I grant you that they are not safety issues but I think they are certainly undesirable behaviors.

The car making unnecessary lane changes into the passing lane because it is confused by exits, is not a good thing. Although it does appear that maybe Tesla has not solved the issue with widening lanes at on ramps and also is not sure about merging traffic from on ramps, so to play it safe, Tesla changed NOA to move over to the passing lanes for exits by default. That's certainly one way to handle the problem.

And deciding to change lanes into the passing lane to pass a slow car and then aborting repeatedly, is not good behavior either.

And Green's last complaint was when AP suddenly stopped working completely and gave him this series of notifications.

He had to drive manually because he could not use AP at all. Surely, that is a bad thing, right?



I merely asked the question about whether the rewrite might have caused some temporary regressions. Also, to be clear, I am not suggestions that these regressions are permanent. I am sure Tesla will fix them at some point. I am merely reporting what Green observed on his road trip. He clearly observed behavior that he calls regressions on this particular sw version 2020.20.5.

Based on this photo he’s in developer mode and part of it is in Chinese. Grain of salt and all that.

 

[LowlyOilBurner]

He mentioned auto high beam is worse. I think it’s gotten better since I got my Y in March.

I agree with the NoA comments, I see the same stuff too, basically, since 2020.12.1.

 

[Eno Deb]

NoA sucked before, and it sucks now --> no regression.

Seriously, I sometimes think they move a team to a new feature, get it half done, check a box on their todo list, then move on to something else. Later they change other parts of the system that affect the orphaned feature, but never go back and regression-test it. The same thing seems to be happening with Smart Summon.

 

[diplomat33]

NoA sucked before, and it sucks now --> no regression.

Seriously, I sometimes think they move a team to a new feature, get it half done, check a box on their todo list, then move on to something else. Later they change other parts of the system that affect the orphaned feature, but never go back and regression-test it. The same thing seems to be happening with Smart Summon.

Yeah, I get that feeling too. Tesla released NOA without confirmation and seemed to consider it good enough. Tesla basically dropped NOA and moved on to city driving situations. Tesla released the traffic light & stop sign feature and are apparently working on "autosteer on city streets" now. Maybe, once "autosteer on city streets" without confirmation is released, Tesla will go back and tie NOA and "autosteer on city streets" together and fix everything?

 

[murc585]

Perhaps they've dropped the ball on NoA in the past months. Not really any improvements to talk about other than the better lane changes end of 2019.
I also do agree that NoA is not the most refined experience and my gut feeling tells me this is not part of the rewrite.
I assume NoA is more the "code 1.0" with procedural logic based on what the "2.0 code" perception engine outputs.
They could be starting to use NNs for NoA though, as Elon has said in the past with their goal to replace classical code by NNs as much as possible.

My car has been in the body shop for the last 3 weeks and it now has 2020.16.2.1.
I've noticed NoA works in places it used to last summer and had not for the past few months. Still not a comforting experience, I can't really use it in Quebec with our short on-ramps and half of the painted lane lines missing.
It normally doesn't have time to apply the blinker to transition onto the highway and if I let it run on the end of the on-ramp, it will swerve into onto the highway, kiss the inside lane and then stabilize. (aka drunk driver style)

My hopes are like yours, that they will divert focus on refining the existing features after "feature complete".
In 2-3 years I would assume autopilot will be smooth like butter.

*A side note about what @greentheonly observed. Couldn't this be seen as a good feature?
An off-ramp is normally followed by an on-ramp and the polite thing to do is move over to the fast lane (if there are no approaching vehicles) when cars are trying to enter the highway.
I really despise when my car stays in the right lane when cars are coming in the on-ramp. It slows down immensely and other drivers are confused because no one is that polite here.
I personally prefer this new behaviour.
If I'm not exiting and an on-ramp is coming up and there are no oncoming vehicles in the fast lane, getting into the fast lane is my desired outcome.

 

[diplomat33]

I assume NoA is more the "code 1.0" with procedural logic based on what the "2.0 code" perception engine outputs.
They could be starting to use NNs for NoA though, as Elon has said in the past with their goal to replace classical code by NNs as much as possible.

I think you are probably right that NOA is still working on mostly "code 1.0" logic. And yes, I am hoping that at some point, with either the rewrite or new driving policy, that NOA gets a redo. I'm still waiting for NOA on the highway to be FSD and it's far from it.

*A side note about what @greentheonly observed. Couldn't this be seen as a good feature?
An off-ramp is normally followed by an on-ramp and the polite thing to do is move over to the fast lane (if there are no approaching vehicles) when cars are trying to enter the highway.
I really despise when my car stays in the right lane when cars are coming in the on-ramp. It slows down immensely and other drivers are confused because no one is that polite here.
I personally prefer this new behaviour.
If I'm not exiting and an on-ramp is coming up and there are no oncoming vehicles in the fast lane, getting into the fast lane is my desired outcome.

Yes, it is desirable to move over to allow cars in the on ramp to merge safely onto the highway. But if there are no vehicles on the on ramp trying to merge, then there is no need to move over. So, I think you would want the car to have the perception and the driving policy to know the different. And you probably don't need to move over for an off ramp. Lastly, you would not want to move over into the passing if it means cutting in front of a car already in the passing lane, especially if there was nobody in the on ramp trying to merge onto the highway. That would be a real douche move. I am not sure NOA has the driving policy to know these nuances. It just follows "dumb logic" of always trying to move over to the passing lane when certain basic criteria are met without exceptions. But that means good behavior sometimes and really annoying behavior at other times.

 

[SilverString]

I noticed the regressions immediately in 2020.5. Particularly in intersections with no painted lines and in widening lane situations. I do believe there is a bit of improvement staying in the center of the lane on Autopilot. The biggest irritation for me is its refusal to change lanes in Autopilot, I have to manually do it most of the time. When you combine this with the zero tolerance speed limit I pretty much have stopped using Autopilot. Have not had a chance to try NOA, or really any extended highway trips..

Hopefully these are just re-write growing pains.

 

[murc585]

I think you are probably right that NOA is still working on mostly "code 1.0" logic. And yes, I am hoping that at some point, with either the rewrite or new driving policy, that NOA gets a redo. I'm still waiting for NOA on the highway to be FSD and it's far from it.



Yes, it is desirable to move over to allow cars in the on ramp to merge safely onto the highway. But if there are no vehicles on the on ramp trying to merge, then there is no need to move over. So, I think you would want the car to have the perception and the driving policy to know the different. And you probably don't need to move over for an off ramp. Lastly, you would not want to move over into the passing if it means cutting in front of a car already in the passing lane, especially if there was nobody in the on ramp trying to merge onto the highway. That would be a real douche move. I am not sure NOA has the driving policy to know these nuances. It just follows "dumb logic" of always trying to move over to the passing lane when certain basic criteria are met without exceptions. But that means good behavior sometimes and really annoying behavior at other times.

It's probably just a bug, since I agree that it would be counterproductive to do this at an off-ramp and of course the car should never do this if an oncoming car in in the passing lane (as I noted in my previous post).

I noticed the regressions immediately in 2020.5. Particularly in intersections with no painted lines and in widening lane situations. I do believe there is a bit of improvement staying in the center of the lane on Autopilot. The biggest irritation for me is its refusal to change lanes in Autopilot, I have to manually do it most of the time. When you combine this with the zero tolerance speed limit I pretty much have stopped using Autopilot. Have not had a chance to try NOA, or really any extended highway trips..

Hopefully these are just re-write growing pains.

IMO, we will really "feel" the re-write, it would be really disapointing if this was just a step change/tuning to autopilot.
All Elon's indications make obvious that this will be a dramatic change to autopilot's behavior, not just minor NoA changes. We would hear about it.

https://twitter.com/elonmusk/status/1270049671046508544?s=20
Autopilot prime directive is: don’t crash. What seems fast to humans is slow to a computer. 360 degree low light vision & sonar, plus forward radar enable to be superhuman. Upcoming software upgrades will increasingly show the potential.

 

[cybergates]

its apalling to me how poorly lanes that widen are STILL (2020.16.3 but I have no reason to believe this has improved in 20.5) handled with Autopilot 2.xx vs. Autopilot 1. Nothing has improved in the last few years with this behavior. Autopilot 1 handled a lane widening with grace (smooth steady course correction) whereas AP2.xx still likes to thrash the steering wheel wildly about whenever the lanes widen or there's a merge - who drives like that - not sure what they were thinking whoever wrote that part of the code. Even with HW3 on the model 3 I see no improvement yet. Just yesterday I was driving HW3 and a left turn lane begins and opens up adn the car immediately thrashes the wheel and tries to skip into the left turn lane for unknown reasons (yes clearly it thinks the newly opened up left turn lane is the current lane), I have to take over in a panic....jarring the passengers....not exactly comforting. Either we're going to need to paint dotted lines in all exiting lanes to prevent the car from hopping lanes (I think arizona freeways and some other states have this) or they are going to need to really get this right. I did a bug report as usual, as mentioned here, not sure who really sees that - or at next service.

Here's hoping FSD re-write fixes this and many more issues esp with city driving

 

[diplomat33]

https://twitter.com/elonmusk/status/1270049671046508544?s=20
Autopilot prime directive is: don’t crash. What seems fast to humans is slow to a computer. 360 degree low light vision & sonar, plus forward radar enable to be superhuman. Upcoming software upgrades will increasingly show the potential.

I think the AP prime directive still needs some work:

 

[am_dmd]

NoA still does not have any situational awareness. I recently got FSD computer upgrade with MCU1 and the only difference between my old AP2/MCU1 is AP does not stay planted in express lane/passing lane. Other than that there is no improvement at all. Depending on what car length setting (1, 2, 3...7) you have, AP still gets all the way behind the vehicle in local lane, slows down and then ask to change lane (manual or auto) even though the passing lane is wide open. Also swinging within the lane when lane widens/shortens like others have mentioned above, no difference between AP2 vs FSD computer. Still barges out of exit lane or slows down way too much risking getting rear ended.
While getting FSD I got an AP1 loaner and it still feels so smooth when changing lanes, no swinging with lane widening, just autosteer is also smooth and autopark works great too.

 

[murc585]

I think the AP prime directive still needs some work:

The lack of creativity of that response was underwhelming.

FACTS:
1) Tesla does not recommend you stop paying attention while using their Beta FSD features, they state that you must always keep your eyes on the road and be alert.
2) What does it say about the human driving? Like many on the road right now, I would revoke their license unless in medical distress.
3) We don’t know if Autopilot was on or not. My suspicion is it was since the driver was obviously not paying attention or they would notice a huge truck flipped over on the road.
4) Will Autopilot be 100% perfect? No.
5) Will any other type of FSD be perfect? No.
6) Does Autopilot save lives today? Yes, and will reduce chances of accidents.
7) Will Autopilot be safer than a human one day on the road in 99% of situations? Yes.
8) Does Autopilot have the potential of saving more lives than any other assistance system available on the market today? Yes.
9) Does Tesla learn from situations like this? Yes.

 

[diplomat33]

The lack of creativity of that response was underwhelming.

FACTS:
1) Tesla does not recommend you stop paying attention while using their Beta FSD features, they state that you must always keep your eyes on the road and be alert.
2) What does it say about the human driving? Like many on the road right now, I would revoke their license unless in medical distress.
3) We don’t know if Autopilot wasn’t on or not. My suspicion is it was since the driver was obviously not paying attention or they would notice a huge truck flipped over on the road.
4) Will Autopilot be 100% perfect? No.
5) Will any other type of FSD be perfect? No.
6) Does Autopilot save lives today? Yes, and will reduce chances of accidents.
7) Will Autopilot be safer than a human one day on the road in 99% of situations? Yes.
8) Does Autopilot have the potential of saving more lives than any other assistance system available on the market today? Yes.
9) Does Tesla learn from situations like this? Yes.

Well, I was going for the quippy remark in response to Elon's tweet. Elon is the one that tweeted that AP is "superhuman" and it's prime directive, ie number 1 rule above all other rules, is to not crash. So I think bringing up when AP crashes is fair game.

But yes, I am well aware that AP is beta and the driver needs to pay attention. Thank you.

 

[Soda Popinski]

Well, I was going for the quippy remark in response to Elon's tweet. Elon is the one that tweeted that AP is "superhuman" and it's prime directive, ie number 1 rule above all other rules, is to not crash. So I think bringing up when AP crashes is fair game.

But yes, I am well aware that AP is beta and the driver needs to pay attention. Thank you.

One of my questions is, for every time a Tesla crashes into a truck (2 semi trailers, 1 overturned, several firetrucks), how many rear end crashes or other less edge case crashes were avoided? Just keeping things in perspective. How likely am I to come across a turned over truck on my commute?

 

[diplomat33]

One of my questions is, for every time a Tesla crashes into a truck (2 semi trailers, 1 overturned, several firetrucks), how many rear end crashes or other less edge case crashes were avoided? Just keeping things in perspective. How likely am I to come across a turned over truck on my commute?

Fair point. I am sure AP does save drivers more often than it crashes them into overturned trucks. An overturned truck is definitely one those pesky edge cases. It is rare but when it does happen, ouch.

And yes, perspective is good. I do think it is a bit too easy to excuse AP's crashes on the grounds that it is saves more lives than it kills. Sure, but just because AP might be very safe on average, does not mean that it does not have room to improve. I think we all want AP to get even better and I trust it will. Certainly, hitting an overturned truck is an example of where AP still needs to improve.

I just thought it was a bit ironic for Elon to tweet that AP's prime directive, ie most important rule, is not to crash, given that AP has crashed numerous times.