Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Navigate On Autopilot: automatic lane change results

This site may earn commission on affiliate links.
My wife rates Autopilot incidents with a 1 to 10 verbal, 10 being bad. Yesterday I managed a 12 and had to buy her a glass of wine at our destination restaurant. It was an exit where two lanes peeled off, I was in the right one, which would later be an exit, and a new exit was coming up on the right. I thought we'd be needing the left lane so I hit the turn signal to the left and an auto-lane change started. As it turns out, Navigate on Autopilot actually wanted to take the new exit to the right and so mid-left-lane-change somehow thought that was being confirmed and so went from doing a left lane change to doing a right exit into the new lane, so effectively cut across the middle lane I was still half in (I wonder if I still have the dashcam footage, I should check). It surprised us both.
 
  • Like
Reactions: Enginerd
The latest iteration is a step back. NOA now tries to make me go to toll lanes, despite earlier versions and AP correctly sending thru the 'pass' lanes. Yesterday, it swerved so hard to the left, I had to grab and tug the wheel heavily to regain control. My wife is now not willing to use NOA, despite being a real devotee of earlier versions.
 
I am impressed with the brilliant owners on this and other forums who are really understanding, questioning, logging, analyzing, describing, and publishing their work and making things better for all of us.
 
  • Like
Reactions: Enginerd and Richt
My usual non scientific observation is that 28.3.1 is a step back. I'm experiencing a lot of aborted lane changes (not including the damn EU 5sec abort rule). Interestingly Tesla told me there is no update due for my car despite 32.2.x being on 7% of Stats App Model S cars
 
This week's update wraps it up for 2019.28.3.1, as I just downloaded the latest this evening. Notes:
  • Drive 31: The only driver aborted lane change was in order to avoid road debris.
  • Drive 32: At the lane split noted in my prior post, where the car typically exhibits indecisive swervy behavior, I did something slightly different. Approaching the decision point, I disabled NOA, leaving AP (only) in the exact same situation. AP (without NOA) exhibited no indecision or swervy DUI-type behavior. It's just one data point, but the result was unexpectedly better, and I'm not sure I understand why.
Full dataset:
upload_2019-9-18_0-44-43.png


Weighted scoring:
upload_2019-9-18_0-45-11.png


Cumulative results for 2019.28.3.1:
upload_2019-9-18_0-46-28.png
 
The first two drives with 2019.32.2 revealed no surprises. Notes this week:
  • Drive 33: This was the first time I've actually been honked at while on NOA. Granted, "honked at" on I-95 is better than being "shot at" on I-5. I was in the process of aborting a lane change that was uncomfortably close to cutting off a big pickup. He apparently took umbrage at the signal and initial movement towards his lane.
  • Drive 34: Exactly 2 driver aborts were recorded on this drive, and they happened to be back-to back. I aborted a lane change that was too close to cutting off car #1. Car #1 passed. Then I aborted a lane change that was too close to car #2. Needless to say, car #2 saw both aborted attempts, and probably didn't have a good impression of NOA's skills based on his brief encounter.
Since May, I've recorded 2081 successful NOA lane changes out of 2304 total events (90.3% success rate). Recent drives on the latest firmware are generally above that average. The most common failure mode (3.8%) is "no offer" (failure to merge out of passing lane before being passed on the right). Other threads contain hints of discrete improvements coming in V10, so I'm looking forward to that.

upload_2019-9-23_23-23-30.png

upload_2019-9-23_23-24-0.png
 
Got V10, it's all good except we don't get Smart Summon in Europe :mad:

AP & NoAP seem better but that is subjective, looking forward to @Enginerd more scientific test :cool:


Thanks for the vid @Didgie , this is the most complete footage I've seen of V10 Autopilot/NOA on a Model S.

I noticed in the video the reason the car didn't auto signal when merging is that it treats that as a lane change and was asking for your confirmation. I'm really curious if lane merges have improved as that was a really weak point of prior NOA version. Sometimes it would just fail to realize it needs to merge until the lane disappears like in your case (without confirmation) and other times it would change lanes 2 lanes over instead of 1.
 
My biggest problem with NoA is that it wants to change into the slow lane miles before the exit it wants to take. In my own commutes, this is a terrible decision.
I knew from your post that you live in SoCal, and I don't doubt this NOA characteristic is an issue there. Just to balance things out, here on the east coast, I've had a few times where NOA doesn't get into the right lane early enough to exit, and I have to "rescue" it. Obviously neither situation is ideal, and I hope they address it.

AP & NoAP seem better but that is subjective, looking forward to @Enginerd more scientific test
Me too. I'm on travel, and the car sits alone in a garage, eager to test 2019.32.11. I only had enough time to briefly play with Smart Summon before I left. I've read that AP/NOA gives semi trucks a wide berth now :), so I'll be sure to keep an eye on that during my first full eval on Sunday 10/6.
 
  • Like
Reactions: navguy12
I knew from your post that you live in SoCal, and I don't doubt this NOA characteristic is an issue there. Just to balance things out, here on the east coast, I've had a few times where NOA doesn't get into the right lane early enough to exit, and I have to "rescue" it. Obviously neither situation is ideal, and I hope they address it.


Me too. I'm on travel, and the car sits alone in a garage, eager to test 2019.32.11. I only had enough time to briefly play with Smart Summon before I left. I've read that AP/NOA gives semi trucks a wide berth now :), so I'll be sure to keep an eye on that during my first full eval on Sunday 10/6.

I can confirm the car will hug the left side of the lane while passing a semi. Doesn't do the same for smaller trucks.
 
I was out of town for a bit, and I've logged only one drive on V10. Some things are different. Some things aren't. Here are my observations:
  • Lane change visualization. Most likely the reason for the visualization update was to facilitate illustration of various types of FSD maneuvers in the future. The key difference in my point of view is that the V9 visualization was essentially static: car was shown with stationary lane lines, with intended path curving into the adjacent lane ahead (sometimes through a semi trailer, although there was never any collision conflict). The V10 visualization is dynamic: camera is fixed to the moving car, as it passes stationary dashed lane dividers. In this view, it's more appropriate to show the car moving purely left or right into the open space between cars.
  • Lane change maneuver. The time between turn signal and lane change completion seems about the same (11-12 blinks). However, the car blinks the turn signal 4 or 5 times to communicate intentions before steering out of the lane. The steering input is a little more aggressive/definitive, and the resulting time between steering input and completion is slightly shorter (by about 2 blinks).
  • Hesitancy/uncertainty. There were a couple times when the lane change process seemed tentative or less confident than V9. The statistics don't illustrate a statistically significant difference in marginal/uncomfortable lane changes in V10, but the character seemed a little different. Watch this space.
  • Wide berth for semi trucks. Someone reported that (currently) the off-center lane offset only occurs when the car is in the middle lane of 3 (or more) when passing a big truck. Most of my route is only 2 lanes in each direction, so if the assumption is correct, this explains why I didn't observe the new behavior. I hope after Tesla establishes some confidence, they'll extend "wide berth" to more driving situations... especially 2-lane undivided highways with opposing truck traffic.
  • On-ramps. NOA finally handled the SC-340 to I-20E on-ramp properly, where V9 consistently overshot the left lane line. My only other notable on-ramp was Santee SC to I-95S. It seemed like that was going to go well, but NOA decided to overtake a slow truck before completing with the initial merge. While the resulting double lane change was an unconventional success, it precluded evaluation of the single lane change on-ramp behavior.
  • Map issues. Previously I noted that I would not allow what appear to be repeatable map issues to negatively impact the lane change results. V10 did not properly handle the I-20E to I-95S, which is a slightly non-standard interchange (but only slightly). V9 didn't handle it either, although the lane positioning error was a little less dumbfounding. No penalty logged.
  • Acceleration from a stop in traffic. In V9, I always helped AP keep up with surrounding traffic, adding some accelerator off the line. Now AP keeps pace just fine, thank you. Chill mode seems to take the edge off this new characteristic, if desired.

Overall, V10 includes some discrete changes and improvements :). However, limited results so far seem to indicate that the decisions about when to change lanes or not... seem to be about the same as V9. V10 certainly wasn't an improvement from 90% to 99% success rate, as some had hoped :(. We'll see how this trends out in future drives.

upload_2019-10-9_22-0-50.png


upload_2019-10-9_22-1-50.png
 
Great stats. Over how many miles?

Is your "driver abort" category for all reasons to abort? how often do you get situations where failure to abort would likely have resulted in collision?

How do you write down the data? Do you have a sex dwarf, I mean, erh, a height-challenged erotic worker, who takes notes as you drive?
.
 
Is your "driver abort" category for all reasons to abort? how often do you get situations where failure to abort would likely have resulted in collision?
There's a more complete description in post #1, but here's a quick recap. I've updated my criteria only slightly since the beginning. Each drive covers approximately 263 NOA miles (so this test is approaching 10000 miles at this point).
  • Success (1 point): Successful lane change, no negative interaction with other motorists
  • Uncomfortable success (0.5 points): Successful lane change, but negative interaction with other motorists or unexpected swervines.
  • No offer (-0.7 points): NOA failed to merge out of fast lane before being passed on the right. ---or--- NOA failed to take intended exit.
  • AP abort (-0.8 points): Autopilot initiated but aborted lane change (did not complete.
  • Driver decline (-0.9 points): NOA indicated intent to change lanes, driver declined on screen in order to avoid: (1) negative interaction with other motorists, or (2) a poor tactical choice.
  • Driver abort (-1 point): Driver aborted an autopilot-initiated lane change in progress, either for safety or to avoid a negative interaction.
Since drive #5, I haven't used the "driver decline" category, which forces those attempts into one of the other categories.

How do you write down the data?
I count beads.