FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[S100D]

Like what? Flagship Model S makes turns to random streets when GPS clearly indicates to go straight

I am currently on 11.4.9 and I have had this same problem on multiple prior firmware releases. My Model S with FSD on but no destination, it wants to turn on the same streets (left and right turns) and even wants to turn into the parking lot of the neighborhood pool.

If i use Navigate Autopilot instead of FSD, my Model S follows the lane lines and does not make any turns.

 

[Ramphex]

I am currently on 11.4.9 and I have had this same problem on multiple prior firmware releases. My Model S with FSD on but no destination, it wants to turn on the same streets (left and right turns) and even wants to turn into the parking lot of the neighborhood pool.

If i use Navigate Autopilot instead of FSD, my Model S follows the lane lines and does not make any turns.

Random turns with no destination is annoying, but random turns with a destination is even more annoying especially when you’re trying to get somewhere and don’t want to do unnecessary uturns or turns across 2-3 lanes.

Just a good reminder that the junk is still literally a pile of junk, and considering the fact the improvements have been so minimal over the past couple of years of being in beta, it’ll likely not allow me to ever drive without it supervising me supervising it, within my ownership time of this specific car.

 

[manole]

For literally every single update I’ve tested, not a single one performed adequately WRT unprotected left or right turns. Has anyone ever experienced this work safely? I know this topic comes up over and over again, but IMHO this is still by far the biggest issue with FSD and I’m not sure why it’s not covered even more.

Some questions / theories:
-I’m too chicken and I don’t give FSD enough of a chance; instead, I should cross my fingers, let it creep forward even more and hope I don’t get t boned.
-Could it be location specific and roads in CT haven’t been adequately trained (e.g. San Francisco is magical)?
-Is there something wrong with my specific vehicle?
-Are there so many other issues people are facing no one cares about ULT and URT anymore since it’s such a lost cause?

What am I missing? If it really is as bad as I’ve experienced for everyone, the solution is simple - program the vehicle to simple avoid ULT and URT at all cost. For example, just wait for the light to turn green on URT (yes, super annoying for the cars behind but at least safer) and plan route to avoid ULT even if it means adding significant time to the journey. For unprotected left or rights with a stop sign instead of a light where rerouting isn’t possible, just prompt the driver to intervene.

 

[cbdream99]

I am currently on 11.4.9 and I have had this same problem on multiple prior firmware releases. My Model S with FSD on but no destination, it wants to turn on the same streets (left and right turns) and even wants to turn into the parking lot of the neighborhood pool.

If i use Navigate Autopilot instead of FSD, my Model S follows the lane lines and does not make any turns.

It used to work quite well in 11.3.6, but now I think they are trying something new and it is biased on vision, so if it sees a turn lane open up and if it is close to the turn location, it will turn into it, or sometimes like a new driver who just focus on the immediate road, sees a lane - must follow it. Be very careful though, if you are traveling at 50 mph, an early turn into business turn lane if not stop immediately can kill you. I experienced this quite often. Imagine this is 30 years ago, you got your new garmin GPS, driving to a new location and not sure the exact location to turn, so you turn early …. FSDb city behaves like that very often. It doesn't fail on very turns but enough to be annoying and I either make it drives on the next lane or turn it off to avoid accident.

 

[sleepydoc]

I am currently on 11.4.9 and I have had this same problem on multiple prior firmware releases. My Model S with FSD on but no destination, it wants to turn on the same streets (left and right turns) and even wants to turn into the parking lot of the neighborhood pool.

If i use Navigate Autopilot instead of FSD, my Model S follows the lane lines and does not make any turns.

Why are you using FSD with no destination? That’s the equivalent of getting into a cab and saying ‘just drive’ then getting annoyed when the driver makes a turn you don’t like. This isn’t a fault in FSD, it‘s a fault with how you’re using it.

For literally every single update I’ve tested, not a single one performed adequately WRT unprotected left or right turns. Has anyone ever experienced this work safely? I know this topic comes up over and over again, but IMHO this is still by far the biggest issue with FSD and I’m not sure why it’s not covered even more.

Some questions / theories:
-I’m too chicken and I don’t give FSD enough of a chance; instead, I should cross my fingers, let it creep forward even more and hope I don’t get t boned.
-Could it be location specific and roads in CT haven’t been adequately trained (e.g. San Francisco is magical)?
-Is there something wrong with my specific vehicle?
-Are there so many other issues people are facing no one cares about ULT and URT anymore since it’s such a lost cause?

What am I missing? If it really is as bad as I’ve experienced for everyone, the solution is simple - program the vehicle to simple avoid ULT and URT at all cost. For example, just wait for the light to turn green on URT (yes, super annoying for the cars behind but at least safer) and plan route to avoid ULT even if it means adding significant time to the journey. For unprotected left or rights with a stop sign instead of a light where rerouting isn’t possible, just prompt the driver to intervene.

I’ve used it for both. There’s a lot of regional variation in roads and driving styles so I’m not sure my experience in the suburbs of Minneapolis would necessarily translate to you, though.

One problem is you can’t watch the screen to see what the car’s intent is while simultaneously watching traffic so there are definitely times I disengage when the car may be intending to do the correct thing.

 

[JHCCAZ]

Why are you using FSD with no destination? That’s the equivalent of getting into a cab and saying ‘just drive’ then getting annoyed when the driver makes a turn you don’t like. This isn’t a fault in FSD, it‘s a fault with how you’re using it.
...

I have found, and posted here, that FSDb actually drives quite well with no destination, and it solves certain problems, particularly the sometimes false " Changing Lanes to Follow Route". It also seems to reduce, but not eliminate, the noted tendency to interpret a turn lane as a new through lane that it should follow.

I actually don't use this mode a lot but it behaves well when I do.

See my comments in this recent post, 3rd paragraph on:

FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN. From AI day and Lex Fridman interview we have a good sense of what might be included. - Object permanence both...

teslamotorsclub.com

 

[cbdream99]

For literally every single update I’ve tested, not a single one performed adequately WRT unprotected left or right turns. Has anyone ever experienced this work safely? I know this topic comes up over and over again, but IMHO this is still by far the biggest issue with FSD and I’m not sure why it’s not covered even more.

Some questions / theories:
-I’m too chicken and I don’t give FSD enough of a chance; instead, I should cross my fingers, let it creep forward even more and hope I don’t get t boned.
-Could it be location specific and roads in CT haven’t been adequately trained (e.g. San Francisco is magical)?
-Is there something wrong with my specific vehicle?
-Are there so many other issues people are facing no one cares about ULT and URT anymore since it’s such a lost cause?

What am I missing? If it really is as bad as I’ve experienced for everyone, the solution is simple - program the vehicle to simple avoid ULT and URT at all cost. For example, just wait for the light to turn green on URT (yes, super annoying for the cars behind but at least safer) and plan route to avoid ULT even if it means adding significant time to the journey. For unprotected left or rights with a stop sign instead of a light where rerouting isn’t possible, just prompt the driver to intervene.

I believe there are some bias on bay area settings and they also have more Tesla than other areas, so the trainings definitely benefit them more. Each area has some degree of variations and whether the road is level, angle, any obstruction will affect the behavior. Even between releases there are some differences. I have seen quite different behaviors in UPT, flashing yellow lights on a busy 4 lanes junction, double turn lanes. Usually if the traffic is sparse, it handles well, but there may be surprises every now and then, it made it but phew that was close.

 

[Ramphex]

I believe there are some bias on bay area settings and they also have more Tesla than other areas, so the trainings definitely benefit them more. Each area has some degree of variations and whether the road is level, angle, any obstruction will affect the behavior. Even between releases there are some differences. I have seen quite different behaviors in UPT, flashing yellow lights on a busy 4 lanes junction, double turn lanes. Usually if the traffic is sparse, it handles well, but there may be surprises every now and then, it made it but phew that was close.

Well then I guess there’s no training at all in my area because the junk behaves like a drunk 15 year old

 

[jebinc]

Well then I guess there’s no training at all in my area because the junk behaves like a drunk 15 year old

 

[TK211X]

M3 RWD 2023,

After an update to the holiday version 44.30.2 then to 44.30.5.1 I noticed while FSD is active in some situations my visualization will lag horribly and the surroundings will disappear. After 5 seconds they will come back after some stutter. FSD remains engaged.

I think this has to do with the complications some people have had but obviously theirs won’t even enable. Mine works.


After reading horror stories on Reddit about their HW3 randomly bricking, I’m tempted to go out to preemptively reset calibration and do a soft reset just in case it helps.

If you’re driving around with FSD on the holiday update, keep an eye on your visualization from time to time. You might be able to spot this behavior.

 

[Dewg]

We've been getting the red wheel of death a lot. I installed 2023.44.30.6 today so we'll see if this version is any better at not giving us the red wheel of death. We don't usually get it at the time of a difficult maneuver, but rather when driving straight on the freeway or on a neighborhood street. We also have frequent issues with our FSD display not functioning & an error message appearing that the display is degraded & may be restored on a future drive. It usually comes back after a minute or 2. During this time we can use FSD Beta even though the display doesn't show any lane lines or other vehicles, just our car.

Go into Service Mode and check your hardware. This could be a camera failure or FSD computer failure.

When it happens on a drive, press the voice button and say "bug report". Open a service ticket and reference the date/time of the report. This will give telemetry to the techs.

 

[Therealm80]

The latest version is actually working really well for me. There is one really bad regression though on a double left turn off of a freeway onto a normal road if there is a car in the left turn lane as I am making the left in the left lane, if it sees a car there, it will swerve into the right left turn lane and almost hit someone. It used to do that perfectly. First time it truly freaked me out.

 

[Ramphex]

The latest version is actually working really well for me. There is one really bad regression though on a double left turn off of a freeway onto a normal road if there is a car in the left turn lane as I am making the left in the left lane, if it sees a car there, it will swerve into the right left turn lane and almost hit someone. It used to do that perfectly. First time it truly freaked me out.

What model?

 

[PACEMD]

Ok, the new light shows are pretty cool, being Christmas and all, but what about those walk-away noises/actions? Elon Twitter-Promised we would get live effects, actions with those noises, like smells with the goats, farts, and ducks, at the least..........maybe a little head butting, wing flapping, brown stains? Sky's the limit you know.........

 

[Therealm80]

What model?

2018 M3PLR

 

[cbdream99]

View attachment 1003627

LOL. FSD is trying very hard. I have seen many improvements through the years. From trying to dead stop in the middle of the interstate announcing arrived at the destination, dart out two lanes from the ramp to now handling quite well on highway.

On the city street, I can see the same path, but I think it will be many years away. There are some noticeable improvements like steering turns at corners are smoother instead of jerky move like 2 years ago. Human has experience, intuition, defensive driving skills, courtesy, and common sense, FSD today just barely making it. What considered smooth by FSD machine itself may not be the same definitions for everyone.

For example, this is the opposite of Chuck's ULT. Main road's speed limit is 40mph (It's Texas, people drives above speed limit). When turning into side street, if there is no traffic, FSD made is smoothly.

If there is traffic then it is aggressive and somewhat a jerk. It will slow down in the median and tries to creep into the other side traffic, wait for the traffic to pass and turn. The problem is there may be one car on the inner lane initially, but another car may speed up coming from the outer lane just 1-2 second later, the road is slight curved and the angle of the camera cannot see it clearly, basically making a risky decision to make a turn. By experience, I know I can wait a few seconds for traffic to clear and it will and then make the turns. It doesn't have to handle every situation like Chuck's ULT.

I also have when there is another car already moving out of the side street to the median, FSD will just drove by him before stopping in the median, I usually don't mind stopping at the turn lane and let the other driver go first in this situation as a courtesy and I don't like to compete with him in the path, worse if my car prevented the other car from stopping completely within the median.

City accident can be as deadly as highway accident if not more, having true self driving is good but we are certainly not a year from out of beta.

 

[JulienW]

.....I have seen many improvements through the years. From trying to dead stop in the middle of the interstate announcing arrived at the destination.....

....but FSD Beta hasn't worked on the Interstate for even 1 year yet.

 

[cbdream99]

....but FSD Beta hasn't worked on the Interstate for even 1 year yet.

Haha, you got me, technically it is the EAP. I lumped them into the full self driving category at least conceptually

 

[Twiglett]

Haha, you got me, technically it is the EAP. I lumped them into the full self driving category at least conceptually

I'm hoping they keep selling EAP, I'll be getting that with next purchase. Its been pretty good even though it chooses lanes like the worlds most impatient driver.

 

[PACEMD]

Haha, you got me, technically it is the EAP. I lumped them into the full self driving category at least conceptually

If concepts were mirrors we would all be at least half illegitimate twins.........