Navigate On Autopilot: automatic lane change results

[Fernand]

I'm not sure. Nicki is a bit mysterious, but I'm OK with that. My impression in riding in Mad Max in mid-dense traffic with people doing their usual, either sitting in a lane and clogging flow, or doing all sorts of shifts and a few last minute acrobatics, is that if the car has incentive, i.e. say 15 MPH of extra strive range, it launches into the dance with pretty optimal and decisive lane seeking. It SEEMS that with a small, say 5 MPH "opening", it's much less ambitious. It's not easy to be sure, but there seem to be a lot of subtle interactions, and given the Neural Network programming, why wouldn't there be?
.

 

[Fernand]

You mention "the car you are following"? Haven't done that a lot lately. I'm more used to NOA in 4 lanes of wall to wall zombie apocalypse, where sometimes the best approach seems to be to close your eyes and let the car figure it out.
.

 

[Fernand]

And to be perfectly honest, a rare and unpopular thing these days, I sometimes do just about close my eyes. In the scariest multi-lane merges, rather than try to keep track of all the kamikazes coming at all the blind spots, I fiddle with my seat belt, and only retain a vague view of the situation until we're out of the maelstrom. It's a lot less stressful.
.

 

[Enginerd]

if the car has incentive, i.e. say 15 MPH of extra strive range, it launches into the dance with pretty optimal and decisive lane seeking. It SEEMS that with a small, say 5 MPH "opening", it's much less ambitious. It's not easy to be sure, but there seem to be a lot of subtle interactions, and given the Neural Network programming, why wouldn't there be?

My impression is that the lane change behavior is much more binary than y'all'r hypothesizing. I envision that there is a chart of speed reduction vs. time, with different lines/curves for Mad Max, Average, Mild. With Mild, it will tolerate being more below the target speed for a longer duration. With Mad Max, it only tolerates a couple of mph below the target speed for a couple seconds before making the decision to change lanes. In either case, my impression is that once it decides to change lanes (yes/no binary), the lane change maneuver itself is going to be the same regardless of the aggressiveness setting... in terms of spacing, steering rate, acceleration rate and everything else.

 

[DopeGhoti]

Well, one other difference is that your car will start accelerating to either the new leader's speed or your set speed as you begin to pass, so if you have more headroom in terms of speed, that may make a lane change seem more aggressive when doing a "standard" lane change from 30 MPH to 70 MPH then doing a "Mad Max" name change from 30 MPH to 35 MPH.

 

[Fernand]

I should be clearer. I don't know if there's a difference in micro-aggression, i.e. timing of the lane change, that could well be perfectly binary/standardized. What I mean is that with a lot of speed headroom the car sure seems to make more lane changes, aiming for a target speed that it can't reach.

With little headroom, i.e. with a small delta between current and target speed, the car can't even properly accelerate when passing, as @DopeGhoti says, because it tops out during the maneuver. And it must have some sort of deadband, differences that aren't worth chasing. The net result is that a small increment doesn't do a lot, but setting to a much higher speed than what you really want (and that can't cause crazy speeding when there's no open space to speed in), sure seems to translate to a more energetic ride.
.

 

[Enginerd]

It's been a while since my last data update. Life's been busy. This update logs 5 more drives with FW 2020.4.1, and the results are largely the same. The failure to exit the passing lane is still the dominant negative feature. I'm looking forward to my next update around March 16, which will include the HW3.0 upgrade and probably FW 2020.8.1. Specific notes from recent drives:

• It bothers me that Tesla appears to be leaving significant shortcomings (hogging passing lane) in the current firmware stack when mated to HW 2.5. @DirtyT3sla and others have demonstrated that HW 3.0 cars with the upgraded FSD computer don't exhibit this problem. While I resent the idea that I should have to pay for the FSD upgrade to remove a recently-introduced bug in their firmware, my car is eligible for the hardware upgrade and I'm scheduled for a service visit this week, to include HW3 and a few other lesser items. To the extent that I have any influence at the service center, I will advocate for those who have Enhanced Autopilot and will not be upgrading to HW3.0 for FSD. The rumor mill has it that an AP core driving path rewrite / re-factor is coming. Hopefully that will benefit both HW2.5 and HW3.0 cars while removing the bug.
• Ping-pong or poor steering control loop stability is also purportedly fixed with HW3.0. I'm very much looking forward to that. It's embarrassing when any perturbation starts an underdamped steering oscillation.
• Both driver aborts on Drive 58 occurred on interstate onramps as previously observed, when the car did not remain within the first lane while merging.
• Refining a prior observation about on-ramps. The probability of NOA lane change success depends on the length of the merging section, expressed in terms of time (# of turn signal blinks).
  
  • Short (< 2 blinks): the lane change kind of disintegrates before the car has a chance to really maneuver, and the car centers up on the merged lane and ends up looking OK.
    
  • Long (> 5.5 blinks), the car has sufficient time to complete a normal lane change.
    
  • Medium (2 to 5.5 blinks), the car blunders past the merging zone and either drives on the shoulder or quickly switches to the nearest lane after the merging lane has vanished.

Full results:

Weighted score:

Cumulative results with FW 2020.4.1:

 

[Fernand]

Well, I'm not sure if this is an engineering tally or a bit of a social experiment The overriding impression you are still conveying to anyone who jumps in (evidently in an effort to "advocate for those who have Enhanced Autopilot and will not be upgrading to HW3.0 for FSD"), that lane change (and hence NOA) has gradually gone to hell, does not match my experience, which I would describe as "not perfect, but purdy damned impressive, and very useful". Can't wait to hear your impressions of NOA on HW3.
.

 

[DopeGhoti]

The overriding impression you are still conveying to anyone who jumps in (evidently in an effort to "advocate for those who have Enhanced Autopilot and will not be upgrading to HW3.0 for FSD"), that lane change (and hence NOA) has gradually gone to hell

And this is the problem, such as it is, with an otherwise impressive and well-regimented body of statistical performance data with a sample size of one. It's exceedingly unlikely we'll ever see the like, but it would be absolutely fascinating to see an equally rigorous set of data for (say) a fleet of 100 cars over the same time period.

 

[Enginerd]

I'm a happy camper after getting the HW3 upgrade. The most recent drive included the latest firmware, 2020.12. As expected, HW3 fixed the failure-to-exit-passing-lane problem (at last!). This makes my long drive much more enjoyable and less stressful. I haven't noticed any obvious differences or improvements specific to the latest firmware, relative to the performance prior to 2019.40.2.1. Specific notes based on Drive 62:

• As expected, steering stability is improved and ping-ponging is essentially eliminated with HW3. Although I had requested it with the HW3 upgrade based on a forum recommendation for improved steering stability, "ultrasonic sensor calibration" is not a thing, per the Raleigh NC Service Center.
• The steering input for lane changes seems less aggressive with HW3. With HW2.5, it was slightly jerky on turn-in… something that would make me nervous if I lived somewhere with icy roads. I theorize that HW3 runs at a high enough frame rate (e.g. 100 times per second) that the initial steering input comes in more gradually instead of in larger chunks (e.g. 10 times per second).
• Even with HW3 and 2020.12, NOA failed to exit the passing lane, but only once. Traffic was essentially non-existent, and I couldn't identify any reason for the oversight. Immediately after the car in the right lane had passed, NOA realized the faux pas, and returned to the right hand lane. Huge improvement, but still not perfect.
• One marginal lane change: NOA switched into the passing lane ahead of faster traffic coming from the left. The timing was somewhere between "rude" and "cut-off".
• The AP-abort and driver-abort both occurred during a single lane change attempt. After committing to the lane change, the car waited too long (6 signal blinks), and steered into the passing lane as a faster car was approaching from the rear, at an uncomfortable (but not unsafe) distance. NOA car swerved away (AP abort), then swerved back toward the passing lane. It may have attempted to swerve away again, but my grip on the wheel prevented it (driver abort).
• One driver decline: after passing slower traffic, NOA attempted to exit the passing lane. The problem was that it was tucking in behind a truck going about 15 mph slower. And it was also near a major interchange with potential for slower merging traffic from the right. It seemed a better choice just to stay in the passing lane a bit longer. I rarely decline a suggested lane change, but this seemed appropriate.

Full results:

Weighted score:

Category breakdown:

 

[Didgie]

I'm a happy camper after getting the HW3 upgrade. The most recent drive included the latest firmware, 2020.12. As expected, HW3 fixed the failure-to-exit-passing-lane problem (at last!). This makes my long drive much more enjoyable and less stressful. I haven't noticed any obvious differences or improvements specific to the latest firmware, relative to the performance prior to 2019.40.2.1. Specific notes based on Drive 62:

• As expected, steering stability is improved and ping-ponging is essentially eliminated with HW3. Although I had requested it with the HW3 upgrade based on a forum recommendation for improved steering stability, "ultrasonic sensor calibration" is not a thing, per the Raleigh NC Service Center.
• The steering input for lane changes seems less aggressive with HW3. With HW2.5, it was slightly jerky on turn-in… something that would make me nervous if I lived somewhere with icy roads. I theorize that HW3 runs at a high enough frame rate (e.g. 100 times per second) that the initial steering input comes in more gradually instead of in larger chunks (e.g. 10 times per second).
• Even with HW3 and 2020.12, NOA failed to exit the passing lane, but only once. Traffic was essentially non-existent, and I couldn't identify any reason for the oversight. Immediately after the car in the right lane had passed, NOA realized the faux pas, and returned to the right hand lane. Huge improvement, but still not perfect.
• One marginal lane change: NOA switched into the passing lane ahead of faster traffic coming from the left. The timing was somewhere between "rude" and "cut-off".
• The AP-abort and driver-abort both occurred during a single lane change attempt. After committing to the lane change, the car waited too long (6 signal blinks), and steered into the passing lane as a faster car was approaching from the rear, at an uncomfortable (but not unsafe) distance. NOA car swerved away (AP abort), then swerved back toward the passing lane. It may have attempted to swerve away again, but my grip on the wheel prevented it (driver abort).
• One driver decline: after passing slower traffic, NOA attempted to exit the passing lane. The problem was that it was tucking in behind a truck going about 15 mph slower. And it was also near a major interchange with potential for slower merging traffic from the right. It seemed a better choice just to stay in the passing lane a bit longer. I rarely decline a suggested lane change, but this seemed appropriate.

Full results:
View attachment 522957

Weighted score:
View attachment 522958

Category breakdown:
View attachment 522959

Back on track then! How are you enjoying the new visualisations?

 

[BridgeMojo]

@Enginerd I've seen this regression since about December regarding HOV lanes:

Before then, NoA was good about NOT changing into an HOV lane unless the option was enabled. Since then I've had several attempted lane changes into an HOV lane. One time I let it change into the lane, and it very shortly changed back out "to follow route."

Here in Southern California, the HOV lanes have limited entrance and exit areas, and autopilot is good about respecting a double-yellow lane divider. When it changes to dashed, I often get an attempted lane change into the faster moving lane, with "Use HOV lanes" disabled.

I'm guessing you don't have an opportunity to measure this behavior in your area. Anyone with HW3 notice an improvement?

Model S, July 2018, HW 2.5

 

[Jeff Hudson]

@Enginerd, do you attribute the big dip in Success with 2019.36.2.4 to the time period Tesla adopted their rewrite of the basic AP codebase? I think this link may explain my question in much more detail than my simple query. Tesla Autopilot Mystery Solved — HW3 Full Potential Soon To Be Unlocked | CleanTechnica

 

[Enginerd]

I've seen this regression since about December regarding HOV lanes

Yeah, sorry, no HOV lanes in my area. I wonder if it's a map data coding issue, or whether they're trying to read signs and road markings to identify HOV lanes. Me and my car have no experience with this.

do you attribute the big dip in Success with 2019.36.2.4 to the time period Tesla adopted their rewrite of the basic AP codebase?

I think the core AP rewrite is still in work and has not released yet (certainly not as far back as December with 2019.36.2.4). I'm guessing that when this comes out, we'll have a lot to talk about. I forgot to mention that while in for my HW3 upgrade, the Tesla service advisor independently volunteered (without being prompted) that a core AP rewrite was rumored to be in the works. Maybe he reads the same forums that we do?

 

[Jeff Hudson]

core AP rewrite was rumored to be in the works

Yeah, that's good to know. It is my understanding that the core AP rewrite was necessary for FSD to be brought to fruition - at least insofar as making the most efficient use of HW3 as possible.

 

[Silicon Desert]

The most recent drive included the latest firmware, 2020.12. As expected, HW3 fixed the failure-to-exit-passing-lane problem (at last!).

Uh, I am thinking that the improvement isn't due to HW3, but instead due to 2020.12. Reason I say that is I also noticed this improvement when upgrading to 2012.12 and I don't have HW3.

 

[sewnsurg]

I'm a happy camper after getting the HW3 upgrade. The most recent drive included the latest firmware, 2020.12. As expected, HW3 fixed the failure-to-exit-passing-lane problem (at last!). This makes my long drive much more enjoyable and less stressful. I haven't noticed any obvious differences or improvements specific to the latest firmware, relative to the performance prior to 2019.40.2.1. Specific notes based on Drive 62:

• As expected, steering stability is improved and ping-ponging is essentially eliminated with HW3. Although I had requested it with the HW3 upgrade based on a forum recommendation for improved steering stability, "ultrasonic sensor calibration" is not a thing, per the Raleigh NC Service Center.
• The steering input for lane changes seems less aggressive with HW3. With HW2.5, it was slightly jerky on turn-in… something that would make me nervous if I lived somewhere with icy roads. I theorize that HW3 runs at a high enough frame rate (e.g. 100 times per second) that the initial steering input comes in more gradually instead of in larger chunks (e.g. 10 times per second).
• Even with HW3 and 2020.12, NOA failed to exit the passing lane, but only once. Traffic was essentially non-existent, and I couldn't identify any reason for the oversight. Immediately after the car in the right lane had passed, NOA realized the faux pas, and returned to the right hand lane. Huge improvement, but still not perfect.
• One marginal lane change: NOA switched into the passing lane ahead of faster traffic coming from the left. The timing was somewhere between "rude" and "cut-off".
• The AP-abort and driver-abort both occurred during a single lane change attempt. After committing to the lane change, the car waited too long (6 signal blinks), and steered into the passing lane as a faster car was approaching from the rear, at an uncomfortable (but not unsafe) distance. NOA car swerved away (AP abort), then swerved back toward the passing lane. It may have attempted to swerve away again, but my grip on the wheel prevented it (driver abort).
• One driver decline: after passing slower traffic, NOA attempted to exit the passing lane. The problem was that it was tucking in behind a truck going about 15 mph slower. And it was also near a major interchange with potential for slower merging traffic from the right. It seemed a better choice just to stay in the passing lane a bit longer. I rarely decline a suggested lane change, but this seemed appropriate.

Full results:
View attachment 522957

Weighted score:
View attachment 522958

Category breakdown:
View attachment 522959

Sorry to be ignorant, but what does NOA mean?

 

[navguy12]

Sorry to be ignorant, but what does NOA mean?

"Navigate on Autopilot".

In a nutshell, the car will decide when and where it should change lanes (on a freeway) to get to the intended destination in the most time efficient manner available for the current traffic conditions.

 

[tinkererguy]

I'm a happy camper after getting the HW3 upgrade. The most recent drive included the latest firmware, 2020.12. As expected, HW3 fixed the failure-to-exit-passing-lane problem (at last!). This makes my long drive much more enjoyable and less stressful.
...

Fantastic post, @Enginerd! I agree with you on your careful observations, on your earlier post too. My VIN # is similar to @DirtyT3sla , and I also recently went from HW2.5 to HW3 staying at 2020.4.1 for a while, and yes, ping-ponging was gone, and staying in the left lane too long on NOA almost gone!. About the same with HW3 2020.8.1, or maybe a little worse actually, with some lane change aborts and sudden unexplained braking, things I have't experienced in months. It sure sounds like I'll be happy with 2020.12!

I've pulled together my observations, tips, and demonstration video at:

• Model 3 HW3 brain replacement caused temporary amnesia but Tesla quickly restored settings, Autopilot improvement demonstrated with new FSD Computer

 

[Didgie]

Very excited, picking up my 2018 MS tomorrow, 2 year checks, 4 new tires and HW3 retrofit upgrade (from 2.5). I know it doesn’t give me much over 2.5 but still excited. I feel a video upgrade coming on.

Tesla Autopilot Test - Take me to Shakespeare 100%