Navigate On Autopilot: automatic lane change results

[DopeGhoti]

Manual lane changes for me seem to both occur faster in terms of how soon the maneuver starts, and happen faster in terms of how quickly the lane change is executed. A manual change (on otherwise empty roads) will blink twice and then cut into the land; a NoA change will blink four times and then move into the lane at a rate I would quantify as slower than a "cut" but faster than a "drift".

I've only had the "camera blinded" message appear in one scenario so far, and it was not rain; it was sunlight shining into the camera combined with being on a car-height concrete wall; the car complained that the driver side B pillar camera was blinded or obstructed. I suspect it was just having trouble resolving flat vertical concrete at 68 miles per hour when the camera was partially overwhelmed by sunlight.

 

[navguy12]

I'm pretty sure that the settings (mild / average / mad max) only have to do with the decision to change lanes, not the character of the maneuver itself. I think the metric is how much below your target speed NOA will tolerate, perhaps with a duration component as well. Once the logic determines that a lane change should be made, I think the steering command is likely the same in any mode. Most of my experience is in Mad Max, so if I'm wrong about this, let me know. I do agree, though, that the aggressiveness of the lateral acceleration should at least consider road conditions (wet, ice), and perhaps leisurely vs. immediate need.

Both my passenger and I thought the same thing, that whatever setting was selected, the only thing that should change is how soon the car would want to pass slower traffic to keep up with my intended set speed.

The actual lateral g force of the lane change itself should not, theoretically, change between Mad Max and the other settings.

However, in my car, both my passenger and I agreed that there was an effect on the aggressiveness of the lane changes such that for the first time since running NOA, I no longer use Mad Max mode.

 

[Enginerd]

• There was not a drop of rain in drive #54, which I conducted after cleaning the already-clean camera lenses. FW 2019.40.2.1 will not change lanes to the right in order to exit the passing lane. NOA will, however, change lanes to the right to navigate an interchange (follow route). Therefore, there's nothing wrong with the cameras or ultrasonics. The lane change request is not apparently being generated by NOA. This is a huge disappointment, and is reflected in the score (worst ever). I submitted several bug reports, for whatever that's worth. DirtyTesla has videos showing I'm not alone with this complaint.
• Overshot merging into the #2 lane from a highway onramp, jumping two lanes straight into lane #1. I logged this as an Uncomfortable Success, simply because I was lucky enough to have no conflicting traffic at that moment.
• Another similar merging failure that I logged as a Driver Abort due to conflicting traffic.
• The car is definitely moving away from disabled vehicles on the side of the road, hugging the inside lane line. I've seen this in prior FW, but have never been 100% sure it wasn't a fluke until now. Again, DirtyTesla has video of this.
• Snaky ping pong bobbing and weaving is still present. Others have postulated that this anomaly may improve after a camera calibration period with the new FW. However, observing the same characteristic 600 miles into testing, I'm rejecting that hypothesis.
• Due to planned personal and business travel, this will be my last test until late January. Hopefully by then we'll have a FW update addressing failure to exit passing lane and bobbing & weaving. Happy holidays, y'all.

Full results:

Weighted score:

So far with this FW:

 

[Bet TSLA]

Yeah, I'm definitely seeing this release as more aggressive and pretty rough. I wonder if they'll give up on their changes and go back to what they had before, or if they'll polish this one until it's as good as before only retaining the faster and more aggressive aspects. My guess is the latter, meaning that to some degree I think this is a deliberate way to make progress.

Of course as soon as we move to a native HW3 version, everything changes. Hope that happens soon.

 

[Fernand]

I'm looking at this a bit differently, I see it as a bold improvement. The car feels faster and nervier too. The regen slowdown -> hold isn't working well, I'm having to use the brakes at times. It's clearly a different branch of the software tree.

This revision is able to hold its own and quite aggressively hunt for the best lane. It seems to do well on crowded "conveyor belt" California freeways, where the passing lane concept isn't respected much anyway. The effect is to leave no slower lanes to "move over" into. I'll have to find some bona fide passing lane scenarios. Californians (unfortunately) don't signal with blinkers, and they don't waste much space on "passing lanes". Maybe that's the priority demographic initially with this new release.

I remember how Consumer Reports rather stupidly berated the system (back when?) for "passing on the right". They even got some small state cops to pontificate at length and concur. On e.g. 10 lane California freeways if you don't go/pass wherever you can, you'll bog things down and you won't get very far.

I haven't tested slow bumper to bumper, but I haven't seen any jerking or snaky ping ponging yet.

 

[Fernand]

Rather than double posting, y'all, take a look at THIS!!! I might have found a way to stabilize the beast.

MASTER THREAD: 2019.40.2 - FSD AutoSteer Stop Sign Warning and Adjacent Lane Speeds

 

[Didgie]

My first drive with FW 2019.40.2.1 was almost entirely in light to moderate rain. While this is certainly a variation on my normal test conditions, it is representative of real world conditions.

• Due to rain, NOA was unavailable for a substantial part of the drive: the number of NOA lane change events was only 59% of the average to date. To state the obvious, rain is not an abnormal condition, and an FSD car must be able to handle it.
  • If the problem is software for image processing, Tesla may work on this in the future.
    
  • If the problem is purely optical, i.e. can't digest images from rear-facing cameras due to water/mist/road spray, then there is a fundamental hardware problem with all Tesla cars.
• Failure to exit passing lane was a big problem with drive 53 in the rain. Interestingly, the car:
  • would not change lanes to the right to exit the passing lane
  • was able to change lanes to the right when traffic in that lane was moving faster
  • was able to change lanes to the right when that lane was required to navigate an interchange.
• After about 5 of failures to exit the passing lane, I exited the highway, shut the car down, and started fresh, with no apparent effect.
• Late in the drive, after several failures to exit the passing lane, a warning message posted periodically: "Right front fender camera blocked or blinded. Clean camera or wait for it to regain visibility." Interestingly, while this message posted a couple times, NOA remained engaged, although it did not make any lane changes to the right in this condition. Based on the photo below, NOA could not see the far lane line to the right. Perhaps this may have been a problem earlier in the drive, resulting in degraded performance, but not quite reaching the threshold for the warning message? I clean my car weekly, so the only schmutz on the camera was due to driving in the rain that day. The camera appeared clean upon reaching my destination.

View attachment 488083​

• Speed matching: this time, it latched on to the rear of a semi trailer instead of latching onto the cab of the truck. The car was driving at about 15 mph below the target speed until I manually accelerated past the truck.
• Ironically, while the drive 53 score was the worst I've recorded, no driver abort was required for safety.
• Unrelated to NOA:
  • Driver-initiated lane changes are much more assertive with reduced delay between signal and steering. Unfortunately, this improvement was not apparently applied to lane changes initiated by NOA (yet?).
  • Ping pong bobbing and weaving is much worse than previous FW.
  • The Deep Rain neural network wipers worked well. In 330 miles of light to moderate rain, I never had to manually wipe or take the wipers off AUTO mode.
  • Adjacent Lane Speed activated a few times, and it seemed entirely appropriate… a welcome human-like change.

Full results:
View attachment 488079

Weighted score:
View attachment 488080

So far with 2019.40.2.1:
View attachment 488081

Wow, that is a massive drop in performance (albeit for fewer events than your usual extensive tests), not what I was seeing in my initial tests, however longer term testing is showing increased ping ponging and starting to cause some concern, although not quite on the scale your charts suggest.

 

[DopeGhoti]

40.2.1 for me is showing markedly better performance at handling the on-ramp and off-ramp transitions, anecdotally.

Prior versions would frequently:

• Signal for the merging zone and then try to cut two lanes over, for both left- and right-on-ramps
• At one specific HOV onramp, not so spectrally 'phantom-brake' just after the merge quite reliably; seemingly cued by a sign on an overpass ahead.
• Try to merge into a new lane and give up after surrounding cars had given ample room

Improvements in 40.2.1 (again, for me anecdotally) have included:

• Beginning to signal before an of- or off-ramp merge lane fully "melds" into the road, better signalling intent
• No longer brake for that specific overpass sign
• No longer hop two lanes over after an onramp
• Significantly better merging into occupied lanes

Granted, I have not been keeping absolute records, nor have I had a lot of opportunities to drive in the rain with NoA, but at least in my use-case, 40.2.1 has been yet another incremental improvement. I should note, though, that I have not seen the "ping-ponging" that others have been reporting with this version.

 

[Fernand]

@DopeGhoti I wonder, how do you hold the wheel? Since I can reproduce suppression of "ping-ponging" with a small weight on right side of wheel (that doesn't defeat nags), and some people, like you, never see it, it suggests it could be because of how they hold the wheel. Try completely letting go as long as it lets you, and see if it hunts more side to side.

ps. but it could be your AO center console wrap too ;-)

Oh, and BTW, with that little weight, I have zero complaints in bumper to bumper too. I haven't seen many clear passing + adjacent lanes of late, can't comment.
.

 

[DopeGhoti]

@DopeGhoti I wonder, how do you hold the wheel?

Generally, I hold the wheel with my hand as dead weight at roughly the 8 O'Clock position, with my elbow resting on my leg. I will sometimes, but not often get a nag with this configuration; when I do I just have to flex my grip a wee bit and the car is happy. Sometimes I switch to my right hand in a symmetrical arrangement.

 

[Fernand]

So it sounds like you ARE pulling the wheel. Could you try just letting go entirely for a while and see if the car starts wandering in its lane?

@Enginerd if you take out the "fail to exit passing lane" events, that aren't really failing lane changes, what does the plot look like?

 

[DopeGhoti]

@Fernand - I actually thought of that and tested "hands-free-but-near" driving on my commute home today. No ping-ponging at all that I could discern. Dead center other than when it was giving semis a wide berth.

 

[Fernand]

So it persists as a mystery.

 

[DopeGhoti]

Alas, yes.

 

[Falkirk]

I find I'm hoping for more improvements as well but when stopped on autopilot behind another car. I find it starts to move still too slowly and at least now guns it to catch up but it could just start earlier and be less heavy on the accelerator in most cases.

 

[DopeGhoti]

Personally I'd rather it sit tight for a second rather than rabbit along and rear-end or panic-stop to avoid rear-ending someone who changed their mind.

 

[Didgie]

So it sounds like you ARE pulling the wheel. Could you try just letting go entirely for a while and see if the car starts wandering in its lane?

@Enginerd if you take out the "fail to exit passing lane" events, that aren't really failing lane changes, what does the plot look like?

You make a very good point, the data shows a huge drop in total number of fails if you include the 'Fail to exit' issue, however if you take that outlier out the data is pretty stable (2 or 3% change on a couple of the actions, which statistically is not enough of a change to identify an issue), in comparison to earlier releases (notwithstanding the ping ponging ).

 

[Fernand]

I went back to an older counterweight that's better sealed, smaller, sits right in the "hollow" (so a little further out on the cross-arm towards the wheel itself), and only weights 100 grams ~ 3.5 Oz. Seems as effective as the bigger one. No significant hunting, nothing I'd call "ping-ponging".

As I mentioned, with 2019.40 I think the car drives more decidedly, more dynamically, much more like a human, and isn't as oriented to always keeping dead center in the lane, but that doesn't bother me at all.
.

 

[drtimhill]

I went back to an older counterweight that's better sealed, smaller, sits right in the "hollow" (so a little further out on the cross-arm towards the wheel itself), and only weights 100 grams ~ 3.5 Oz. Seems as effective as the bigger one. No significant hunting, nothing I'd call "ping-ponging".

I'm going to regret asking this, but what is the purpose of this counterweight ?

 

[Fernand]

It seems to stabilize how my car drives in AS. I originally started using it to make responding to "nags" in NOA require less motion/torque, as I was often jerking the car out of AP. A small weight like this doesn't defeat the nags themselves.

With 2019.40 I never saw any of the reported pingpong hunting in the lane until I took it off. It was noticeable and 100% reproducible, so I put the little packet of BBs back on. It's quite common for a small change to stabilize or destabilize a complex feedback loop like an AS system.

Not clear why some cars never exhibit the pingponging. I expect a newer release to get rid of the issue.
.