What NOA needs to get to FSD on the highway

[diplomat33]

I just got back yesterday from a 680 mile round trip, almost all of it on NOA. So here's my quick report and what I think NOA needs to get to true Highway FSD.

TL;DR SUMMARY Overall, NOA is really good. It handles the open highway extremely well, pretty much hands-free. It still struggles with some exits, especially on city highways, and it needs better driving policy in a few areas to really make it FSD.

GOOD
- Lane Keeping was perfect, even in construction zones.
- Auto Lane changes were very smooth.
- NOA did a great job of executing a lane change only when it was safe. Every time, it would turn the blinker on, but if it saw a car in my blind spot, it waited for the car to pass and then finished the lane change behind them. Or when the car slowed down, NOA would finish the lane change when space opened up.
- NOA also canceled lane change suggestions when it saw that traffic in the passing lane was not any faster.
- NOA executed lane changes into the passing lane and back into the slow lane.
- NOA handled merging well. It slowed down for cars cutting in or merging into my lane and cars merging in from on ramp.

The long stretches of open highway were excellent. Definitely a solid highway ADAS.

BAD
- NOA had some issues when I got to city highways and clover leaf ramps. Auto Steer handled the clover leaf turn perfectly but NOA almost missed the highway transition at the end of the clover leaf. At another point, I was in a two lane exit, and NOA saw a semi cutting in front of me and suggested a lane change into the right exit lane which was the wrong lane for me.
- Most exits were fine. But there were a couple exits, NOA would have missed had I not intervened.
- Congested traffic is an issue because NOA only tries to make the lane change to take an exit at 1 mile from the exit. That does not leave a lot of buffer to cut in if traffic is congested, especially with how timid NOA is at making lane changes.
- NOA likes to move over the passing lane even when it should stick to the right lane because your next exit is coming up soon and traffic ahead is congested.
- NOA did not move over for stopped vehicles on the side of the highway.

While lane keeping was excellent, I definitely needed to monitor things a lot more on city highways to make sure I would take the right exit etc...

So what does NOA need to get to true FSD? I think it needs two things:

1) Better vision NN & higher reliability
NOA needs much higher reliability so that it can handle all exits and all highway transition with 99.9999% reliability without driver intervention. Also, I think NOA should be able to read highway signs and also avoid objects on the road. This is key for true FSD. I think reading highway signs would be very helpful in cases where the Navigation Maps are outdated. I think it would help NOA not miss exits or highway transitions. And of course, avoiding objects on the highway is a no-brainer.

2) Better driving policy.
Specifically, NOA needs 3 main driving policies IMO:
- "if the designated lane I need for my route is clear say 5 miles from my exit or highway transition, make the lane change now and stay in that lane so that I stay on my nav route. Don't move over the passing lane on a 4 lane highway when I need to be in the right most lane soon and traffic is congested"

This policy would make missing exits less likely and also make for a smoother drive in city highways.

- "If there is stopped vehicle or person on the side of the highway, move over to the passing lane."

I believe that is an actual driving rule so NOA should follow it.

- "If a car is merging from an on ramp, move over to the passing lane".

That is just courteous and promotes safe driving.

Those 3 driving policies alone would dramatically bring NOA closer to FSD IMO.

 

[RDoc]

I think people need to define what they mean by "FSD". My definition is L5, even if regulations require a driver as backup, that is, the car will handle all conditions that the average human driver would at least as well. The "Full" in "FSD" seems to be getting pretty soft, morphing into a list of what the car can handle with everything else being excluded.

For example, sticking to highways, about once a year or so I come up on an accident with highway traffic being directed by a person. I'd say that a FSD car should handle this safely and reasonably. Clearly there are lots of lesser capabilities that are quite useful, but couldn't handle this case. However, they wouldn't be "Full" IMHO. Other wise we're going to have to start talking about Full, Chock a Block Full, Full to the Tippy Top, etc.

 

[diplomat33]

I think people need to define what they mean by "FSD". My definition is L5, even if regulations require a driver as backup, that is, the car will handle all conditions that the average human driver would at least as well. The "Full" in "FSD" seems to be getting pretty soft, morphing into a list of what the car can handle with everything else being excluded.

For example, sticking to highways, about once a year or so I come up on an accident with highway traffic being directed by a person. I'd say that a FSD car should handle this safely and reasonably. Clearly there are lots of lesser capabilities that are quite useful, but couldn't handle this case. However, they wouldn't be "Full" IMHO. Other wise we're going to have to start talking about Full, Chock a Block Full, Full to the Tippy Top, etc.

In the context of this thread, since I am talking about full self-driving limited to highways, it would be L4 autonomy.

 

[Sporty]

I'm new to NOA, but I would not call Lane Keeping as "perfect". It works very well at low speeds. But at speeds > 60 (65) mph it starts to reach some limitations. The other day (405 N Bellevue) at around 70 mph taking a normal highway road curve the car about left my lane into a neighboring car. The driver mirror was on the line. Given my limited experience, I suspect at 55 mph it would have been fine. It's a difficulty feature to use on a crowded road unless you want to drive slow.

The lane following feels "in the moment". Like it doesn't project/plan what it needs to do.

I have even less experience with lane changing, but it started moving right 5 miles before my exit. And stayed in the slow lane (to get onto 520). It was fine so long as you don't mind drive 40 - 45 mph why everyone else is going 60+.

It put itself in the wrong lane to go 520 w. Looked to be a map bug. Should have been second from right and it put itself in the right lane.. Traffic was heavy so I took over as it would have required non-seattle driving skills to get back into the proper lane.


All and all I find it useful. It's nice in bumper to bumper slower traffic and in light traffic. And I look forward to experience it while it improves.

 

[SanDiegoM3]

I just got back yesterday from a 680 mile round trip, almost all of it on NOA. So here's my quick report and what I think NOA needs to get to true Highway FSD.

TL;DR SUMMARY Overall, NOA is really good. It handles the open highway extremely well, pretty much hands-free. It still struggles with some exits, especially on city highways, and it needs better driving policy in a few areas to really make it FSD.

GOOD
- Lane Keeping was perfect, even in construction zones.
- Auto Lane changes were very smooth.
- NOA did a great job of executing a lane change only when it was safe. Every time, it would turn the blinker on, but if it saw a car in my blind spot, it waited for the car to pass and then finished the lane change behind them. Or when the car slowed down, NOA would finish the lane change when space opened up.
- NOA also canceled lane change suggestions when it saw that traffic in the passing lane was not any faster.
- NOA executed lane changes into the passing lane and back into the slow lane.
- NOA handled merging well. It slowed down for cars cutting in or merging into my lane and cars merging in from on ramp.

The long stretches of open highway were excellent. Definitely a solid highway ADAS.

BAD
- NOA had some issues when I got to city highways and clover leaf ramps. Auto Steer handled the clover leaf turn perfectly but NOA almost missed the highway transition at the end of the clover leaf. At another point, I was in a two lane exit, and NOA saw a semi cutting in front of me and suggested a lane change into the right exit lane which was the wrong lane for me.
- Most exits were fine. But there were a couple exits, NOA would have missed had I not intervened.
- Congested traffic is an issue because NOA only tries to make the lane change to take an exit at 1 mile from the exit. That does not leave a lot of buffer to cut in if traffic is congested, especially with how timid NOA is at making lane changes.
- NOA likes to move over the passing lane even when it should stick to the right lane because your next exit is coming up soon and traffic ahead is congested.
- NOA did not move over for stopped vehicles on the side of the highway.

While lane keeping was excellent, I definitely needed to monitor things a lot more on city highways to make sure I would take the right exit etc...

So what does NOA need to get to true FSD? I think it needs two things:

1) Better vision NN & higher reliability
NOA needs much higher reliability so that it can handle all exits and all highway transition with 99.9999% reliability without driver intervention. Also, I think NOA should be able to read highway signs and also avoid objects on the road. This is key for true FSD. I think reading highway signs would be very helpful in cases where the Navigation Maps are outdated. I think it would help NOA not miss exits or highway transitions. And of course, avoiding objects on the highway is a no-brainer.

2) Better driving policy.
Specifically, NOA needs 3 main driving policies IMO:
- "if the designated lane I need for my route is clear say 5 miles from my exit or highway transition, make the lane change now and stay in that lane so that I stay on my nav route. Don't move over the passing lane on a 4 lane highway when I need to be in the right most lane soon and traffic is congested"

This policy would make missing exits less likely and also make for a smoother drive in city highways.

- "If there is stopped vehicle or person on the side of the highway, move over to the passing lane."

I believe that is an actual driving rule so NOA should follow it.

- "If a car is merging from an on ramp, move over to the passing lane".

That is just courteous and promotes safe driving.

Those 3 driving policies alone would dramatically bring NOA closer to FSD IMO.

Have to disagree on the lane changing. Moving to the right lane early is a horrible decision. There are many right lanes on freeways that are jammed for a popular exit, but clear up afterward and what if you’re the afterward exit? You get stuck in a jam for absolutely no reason.

Each situation is different and the car simply needs to learn exits or use information better.

For example, it should use traffic data from google maps to understand where the jams start and relieves. If the jam relieves prior to your exit it should not move over to the exit lane early, but stay at most within 2 lanes of the exit lane (no fast lane on 4 lane highway).

Your post is fundamentally backwards as it operates within limits of technology as you see it. NoA should always operate optimally, which means it shouldn’t be timid in all situations and driving is a dance with other cars.

 

[diplomat33]

Have to disagree on the lane changing. Moving to the right lane early is a horrible decision. There are many right lanes on freeways that are jammed for a popular exit, but clear up afterward and what if you’re the afterward exit? You get stuck in a jam for absolutely no reason.

Each situation is different and the car simply needs to learn exits or use information better.

For example, it should use traffic data from google maps to understand where the jams start and relieves. If the jam relieves prior to your exit it should not move over to the exit lane early, but stay at most within 2 lanes of the exit lane (no fast lane on 4 lane highway).

Your post is fundamentally backwards as it operates within limits of technology as you see it. NoA should always operate optimally, which means it shouldn’t be timid in all situations and driving is a dance with other cars.

I agree that NOA should be more flexible and check traffic data. But right now, it does not do that. As far as I know, NOA will always wait to about 1 mile from your exit and then attempt to take the exit. That's fine if traffic is light but if there is a traffic jam at your exit and you are in the left lane, you will most likely miss your exit because you won't be able to move over in time. So I was suggesting an easy fix within the limits of how NOA currently works. Basically, if NOA moves over to the right lane sooner then it won't miss the exit.

 

[EVNow]

Basically, if NOA moves over to the right lane sooner then it won't miss the exit.

Yes - currently I don't use NOA because of this. The interchange I need to take is perennially backed up - Tesla will always miss it.

Two simple choices
- If the traffic is heavy move to the exit lane as early as possible or
- Have a configuration option that if you check, NOA would move the car to the exit lane as early as possible

 

[happyzod]

I think people need to define what they mean by "FSD". My definition is L5, even if regulations require a driver as backup, that is, the car will handle all conditions that the average human driver would at least as well.

It really depends on your definition of FSD. My personal definition for FSD (and it seems to follow Tesla's definition), is having the feature set required to get from point A to point B at a good enough quality to ship. Sort of like a minimum viable product you put out there and iterate on.This is separate from the L1-L5 definitions for full autonomy.

I consider NOA to be an FSD system for highways already. All the building blocks are in place and the quality is high enough but not L3+. I expect NOA to improve as more teslas are out on the road allowing the team to iterate quickly via their data engine.

I expect something similar for city driving. It will take many billions of miles to get to the same quality as NOA, then many more billions of miles to reach level 3,4,5. In the meantime, the system is far from useless and is very functional and quite a large QOL boost. I know that I no longer fear traffic!

 

[RDoc]

It really depends on your definition of FSD. My personal definition for FSD (and it seems to follow Tesla's definition), is having the feature set required to get from point A to point B at a good enough quality to ship. Sort of like a minimum viable product you put out there and iterate on.This is separate from the L1-L5 definitions for full autonomy.

If that's your definition, then they've already got it. Actually all cars since the late 1800's have been able to get from A to B. However, Musk has stated that their cars will be autonomous enough to build Robotaxis next year, so I think they have a pretty high level in mind.

In my view, if the "Full" in "Full Self Driving" has the standard English meaning, the car, without human assistance or intervention, should be capable of driving itself, reasonably safely, in the full set of conditions a human would be expected to handle. Limiting that to highway driving is fine if that's the stated capability, but it has to handle all the difficult edge cases there, e.g. flag men, officers directing traffic at an accident, lane crossovers at construction sites, animals crossing the highway, etc.

Now if we're talking about "Kind of Self Driving" (KoSD), or you view the "F" in "FSD" as standing for "Faux", then that's a different matter.

 

[Routybouty]

I'm wondering how they are going to handle traffic that doesn't really follow the rules and mimic humans instead, which is probably safer to be consistent than correct. The problem in my case is the road itself is poorly laid out. One example is an exit that creates two lanes only during the curve of the exit. No one will use the right lane except for passing...kinda the opposite of the rules. The right lane didn't used to exist. Of course, in the google picture there is someone in the right lane, but honestly, rarely does anyone get into the right lane, because it ends only an 1/8 mile away past the merge.

Just before that is the actual exit lane which is very long. This exit lane causes NOA to very aggressively follow that right lane line to the point that it overshoots the lane to either on the line or slightly past it, very annoying.

The other place I get un-human like behavior is when a two lane curve is getting ready to merge to the next highway, it steps down to one lane from two. Since the right lane ends, everybody anticipates and gets into the left lane much much earlier. NOA will hang out in the right lane until it ends. While the right lane is technically the better lane as it's not the passing lane, it's only used for passing in this instance...

Since they are trying to shy away from mapping things, I wonder how they will handle this behavior? Should there be some navigation data for biasing it to mimic human behavior?

 

[diplomat33]

I'm wondering how they are going to handle traffic that doesn't really follow the rules and mimic humans instead, which is probably safer to be consistent than correct. The problem in my case is the road itself is poorly laid out. One example is an exit that creates two lanes only during the curve of the exit. No one will use the right lane except for passing...kinda the opposite of the rules. The right lane didn't used to exist. Of course, in the google picture there is someone in the right lane, but honestly, rarely does anyone get into the right lane, because it ends only an 1/8 mile away past the merge.

View attachment 435588

Just before that is the actual exit lane which is very long. This exit lane causes NOA to very aggressively follow that right lane line to the point that it overshoots the lane to either on the line or slightly past it, very annoying.

View attachment 435597

The other place I get un-human like behavior is when a two lane curve is getting ready to merge to the next highway, it steps down to one lane from two. Since the right lane ends, everybody anticipates and gets into the left lane much much earlier. NOA will hang out in the right lane until it ends. While the right lane is technically the better lane as it's not the passing lane, it's only used for passing in this instance...

View attachment 435593

Since they are trying to shy away from mapping things, I wonder how they will handle this behavior? Should there be some navigation data for biasing it to mimic human behavior?

Thanks for sharing. I suspect there are different ways of handling this situation. Vision would be one: teach the NN to understand what it is seeing when it looks at the lane lines to see that it should stay in the left lane in the exit for example. After all, humans do that. We can look at those lane lines and deduce what we should do, and remember for the next time. But another method would be navigation data. Just have a navigation "reminder" to AP about what it should do in this particular scenario.

 

[RDoc]

The thing about humans vs Neural Nets is that humans have no problem at all with single trial learning. When presented with a novel situation we can trivially extend other "similar" situations to handle it.

A straight NN in many cases has a terrible time doing the same thing.

 

[North75]

I tend to agree with diplomat33's assessment of where things are at with the latest software revisions. I took a trip this weekend and was generally surprised how much better behaved the NOA was over the older versions. The lane changes seemed much more assured and less likely to annoyingly cutoff fast approaching cars.
Sure there are situations where it has it's problems but overall I would rather drive with it on than off, which definitely wasn't the case in the past.

 

[happyzod]

If that's your definition, then they've already got it. Actually all cars since the late 1800's have been able to get from A to B.

Not really since I mentioned minimum viable product and having a good enough quality to ship. With Autopilot and NavOnAutopilot, it is obviously good enough to ship since people were willing to pay lots of money for those features. There are many benchmarks you can use but with Navigate on Autopilot, I can tell it to get from Point A to Point B and it will get there without intervention more than half the times. So I expect a similar level after 1 billion miles of city driving for FSD.

 

[dgatwood]

Yes - currently I don't use NOA because of this. The interchange I need to take is perennially backed up - Tesla will always miss it.

Two simple choices
- If the traffic is heavy move to the exit lane as early as possible or
- Have a configuration option that if you check, NOA would move the car to the exit lane as early as possible

I would lean towards adding two rules to the existing approach:

• Do not ever change into the right (or left) lane until after you have passed the exit prior to where you're getting off unless it is a paired exit.
• Move out of the carpool lane as late as possible, if the carpool lane is in force.

Most of the time, the previous exit results in cars getting off, thus making room for new cars, and the cars getting on are invariably trying to get quickly out of that right lane, again making room for new cars. And the carpool lane is almost always faster, which means you should stay in it until as late as possible before moving over as quickly as possible.

 

[EVNow]

I would lean towards adding two rules to the existing approach:

• Do not ever change into the right (or left) lane until after you have passed the exit prior to where you're getting off unless it is a paired exit.
• Move out of the carpool lane as late as possible, if the carpool lane is in force.

Most of the time, the previous exit results in cars getting off, thus making room for new cars, and the cars getting on are invariably trying to get quickly out of that right lane, again making room for new cars. And the carpool lane is almost always faster, which means you should stay in it until as late as possible before moving over as quickly as possible.

Yes - when I say as soon as possible I meant soon after the previous exit. Not sure about the second part - if there is an interchange or very heavily backed up exit involved.

The truth is - there are a myriad cases, that's what makes FSD challenging.

 

[diplomat33]

Got a chance to drive 150 miles on NOA on the latest 2019.28.2 version.

Here are some thoughts:

-Auto lane changes seem a bit quicker and more confident than previous versions.
- NOA did too many auto lane changes for my taste but they were all well executed. NOA was able to safely wait until several cars had passed before finishing a lane change. The lane keeping and auto lane changes were excellent. Once I was out of the city, NOA was pretty much FSD. No issues.
- I did have one weird situation where NOA failed to merge from on ramp to highway. There was no traffic. For some reason, it started to merge over and then jerked me back into the exit lane so I had to take over. Not sure what spooked NOA. It's never done that before.
- I did take over when I got close to the city because traffic was very congested (backed up stop and go traffic for about 2 miles) and I was not sure if NOA could merge across several lanes of traffic to catch my exit in time.
- I did have one situation where I think I did not satisfy the hands on wheel nag correctly because NOA terminated a lane change, jerked me back in to right lane, gave me the red hands on wheel warning, beeped loudly at me, and disengaged AP. Or maybe NOA saw a car approaching fast in the left lane and jerked me back into the right lane to save me. Not sure.
- I still had to "manually" move over when there were parked cars or parked trucks on the side of the highway.

 

[EVNow]

- I still had to "manually" move over when there were parked cars or parked trucks on the side of the highway.

Does the car stop or crash if you don't ? I guess it's like that Russian accident.

 

[diplomat33]

Does the car stop or crash if you don't ? I guess it's like that Russian accident.

It would depend on whether the parked vehicles were sticking out into the travel lane. If they were sticking out into the lane, AP would probably hit them, I imagine. It does not stop for them, no. But I move over even when they were not sticking out into the lane because it's the right thing to do.

 

[AlanSubie4Life]

NOA likes to move over the passing lane even when it should stick to the right lane because your next exit is coming up soon and traffic ahead is congested.

Most exits were fine. But there were a couple exits, NOA would have missed had I not intervened.

I did take over when I got close to the city because traffic was very congested (backed up stop and go traffic for about 2 miles) and I was not sure if NOA could merge across several lanes of traffic to catch my exit in time.

I find these complaints to be odd. Maybe I need to try ULC NoA again (I think maybe I last used ULC in March or April). I had to turn off the ULC and now always request lane changes with my own timing, because I found that NoA was trying to get over for the exit from the fast lane at least three miles before my exit, regardless of traffic conditions! Really was aggressive about getting over.

I'd much prefer that it have a late exit mode, where it mostly camps in the number two lane (assuming an appropriate set speed for that lane of travel), then it tries to make the switch across 3-4 lanes in less than a mile (like a normal human!). That way there's minimal hanging out behind and amongst the gravel trucks, behind work trucks disgorging ladders & nails, and the nice folks watching videos & chatting on their cell phones in the slow lanes.