FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[aronth5]

For those who down play obstructed views with the B-pillar camera today was an example why you may not understand the problem. Last night we got 18" of snow and the plowed snow banks at some secondary road intersections completely blocked the B-pillar camera view. Banks are higher than the car.

Disengaged FSD and slowly crept into the intersection since it was even hard for me to see when leaning forward. If there was a wide camera view from the fender/headlight or windshield camera FSD could probably be ok but as is I have no idea how a robotaxi with L4/L5 would have ever made it's destination. Definitely an edge case but it happens.

 

[AlanSubie4Life]

So close to being pretty good.

Just needs to stop trying to drive into fields, and then it will be so good! Seriously though: I agree less of the terrifying (to passengers - as a driver it is NBD) behavior would make usage rates go up.

It should make a wider turn into the next lane rather than a tight turn from a right lane to a right lane especially on 3+ lane roads

If you gun it at the turn, you can usually push it into the far lane. Obviously only do this if there is one right-turn lane and it is legal to do so!

Pushing the accelerator is a convenient way to push the car wide when needed. It can be fun. If there are no passengers.

It’s also often required since it is so slow to make turns when there is no traffic - can be a hazard. If you really step on it, it will often go wide.

 

[enemji]

MN law states you need to turn in to the closest lane. If there are two lanes then the first lane turns into the closest lane and the next lane turns into the next closets lane, etc. Making a wider turn is technically illegal.

This should be applicable Only if there are more than two lanes available for turning right.

 

[Skotty]

Typical speed bump with markings that FSD always ignores.
View attachment 1006646

Hm...if they could make the car learn to understand road incline changes, it could adjust speed based on sharpness of road incline changes. Then it should always handle speed bumps whether marked or not (slowing more for the monster short ones and less for the longer ramp-style ones). Not to mention just handling things like intersections where you have to slow down due to one road ramping over the other (which can be fun to take fast if done intentionally, but the car should handle it with more caution and slow down a bit). I wonder if that's possible.

 

[AlanSubie4Life]

This should be applicable Only if there are more than two lanes available for turning right.

You might think that it should be that way but it is not the case; in some states you must always turn into the closest lane.

 

[cbdream99]

A new behavior I've seen with the holiday build is single-lane protected left turns. There is one intersection by my house that navigation always takes me through. It's a single left turn lane, protected, turning onto a 3-lane (technically 4 lanes, but the #4 lane almost immediately ends and merges into #3). My car used to always turn into the #1 lane, and then immediately start changing lanes to the right to make the right turn at the next intersection. Now, with the holiday build, it will turn wide, all the way to the #4 lane, which then has to merge into #3. Most people turn wide and into #2 or #3 lanes on this turn, so it's more human-like. I worry about people turning right in the opposite direction, but technically, since it's a protected left, and the opposite direction has a red light, they shouldn't be turning until the left-turners are done.

Yes, I have seen this behavior for left turn, but for the right turn it is not doing that. I have also seen it move more forward before making the left turn instead of wide turn in the same junction, I don't feel comfortable for this and take over.

 

[sleepydoc]

This should be applicable Only if there are more than two lanes available for turning right.

No - if there is only one turn lane then you still turn into the nearest lane.

 

[sleepydoc]

For those who down play obstructed views with the B-pillar camera today was an example why you may not understand the problem. Last night we got 18" of snow and the plowed snow banks at some secondary road intersections completely blocked the B-pillar camera view. Banks are higher than the car.

Disengaged FSD and slowly crept into the intersection since it was even hard for me to see when leaning forward. If there was a wide camera view from the fender/headlight or windshield camera FSD could probably be ok but as is I have no idea how a robotaxi with L4/L5 would have ever made it's destination. Definitely an edge case but it happens.

Below are two pictures I took from my MY at an obstructed intersection. The first is at the pillar, giving the camera view, the second is as far forward as I could lean looking myself. Is there a difference? Yes. Is it a big difference? No.

 

[rlsd]

Yes, I have seen this behavior for left turn, but for the right turn it is not doing that. I have also seen it move more forward before making the left turn instead of wide turn in the same junction, I don't feel comfortable for this and take over.

It's ok to do that although it's not wise for human driver. I guess FSD does that because it "thinks" it's safe to do so (no coming car to the target lane).

For muti-lane right turn, I wish FSD does that way in some cases. On my way to work there are 2 right turn lanes from a freeway to a 4 lane street. A lot people move to lane 1 or 2 right away after turning from lane 1 (leftmost turn lane) to go faster. Some people move to lane 3 right away instead of staying in lane 4 after turning from lane 2. I wish FSD move to lane 1 or 2 like humans in this case. If turning from lane 1 to lane 3 without moving to lane 2 or 1 right away there is potential accident when people switch from lane 4 to 3. It's illegal but that is the reality because lane 4 is a dead end.

 

[aronth5]

Below are two pictures I took from my MY at an obstructed intersection. The first is at the pillar, giving the camera view, the second is as far forward as I could lean looking myself. Is there a difference? Yes. Is it a big difference? No.

View attachment 1007361View attachment 1007360

I don't consider that much of an obstructed view. Now put some trees and bushes on this side of the building and you have an obstructed view.

Here us a real obstructed view intersection. My car is already at the edge of the intersection. Note no shoulder. Right Pillar Canera view.

 

[cbdream99]

It's ok to do that although it's not wise for human driver. I guess FSD does that because it "thinks" it's safe to do so (no coming car to the target lane).

It's a very odd position. I am referring to a single protected left turn lane. If FSD goes forward making a 90 degree slow turn, and the car(s) behind thinks granny in the front is too slow, decided to speed ahead and making a wide turn, then we will be competing for the target lane.

 

[sleepydoc]

Hm...if they could make the car learn to understand road incline changes, it could adjust speed based on sharpness of road incline changes. Then it should always handle speed bumps whether marked or not (slowing more for the monster short ones and less for the longer ramp-style ones). Not to mention just handling things like intersections where you have to slow down due to one road ramping over the other (which can be fun to take fast if done intentionally, but the car should handle it with more caution and slow down a bit). I wonder if that's possible.

that would be good to do. In practice I'm not sure how feasible it would be - I often times can't see unmarked speed bumps, especially if the lighting is poor.

 

[jabloomf1230]

Below are two pictures I took from my MY at an obstructed intersection. The first is at the pillar, giving the camera view, the second is as far forward as I could lean looking myself. Is there a difference? Yes. Is it a big difference? No.

View attachment 1007361View attachment 1007360

I think you've made a case that the difference is significant. Leaning forward, one can see well down the road. If the speed limit there was 30 mph, then the difference in visibility is small. But if the speed limit is 55 mph on the crossroad, then the difference is substantial. No amount of NN elegance will substitute for having a better view of cross traffic.

 

[kabin]

Below are two pictures I took from my MY at an obstructed intersection. The first is at the pillar, giving the camera view, the second is as far forward as I could lean looking myself. Is there a difference? Yes. Is it a big difference? No.

View attachment 1007361View attachment 1007360

That's no problemo with the nice roadway shoulder and relatively distant obstruction.

 

[kabin]

I think you've made a case that the difference is significant. Leaning forward, one can see well down the road. If the speed limit there was 30 mph, then the difference in visibility is small. But if the speed limit is 55 mph on the crossroad, then the difference is substantial. No amount of NN elegance will substitute for having a better view of cross traffic.

Most of us feel a lack of NN elegance when FSD approaches stop sign intersections. There's the funky braking. It starts to go then stops. Then stomps on the pedal all the while we know the B pillar can't see around the obstruction cause the driver can't see around the obstruction.

 

[nikolajz1]

11.4.9 really has gotten worse in DC traffic for me. I have Progressive and my phone monitors all the driving stats since you save like 30 percent on insurance with it turned on. I'm getting hard braking dings on my insurance almost every drive to work now with this new version. Really likes to accelerate quickly when someone leaves a lane and then slam on brakes even on Chill.

Also has been making some bad decisions lately too, trying to go around stopped traffic on a one lane on ramp. Happens almost every time for me now where it tries to jerk to the right to go around stopped cars, even with minimal lane changes on.

Also the traffic flow speed add on sometimes makes bad decisions coming up to speed camera that are marked on the map. It will ignore the speed change and just quickly go through areas because everyone else is too.

Overall not happy with this version. 4.4 was pretty great.

 

[rlsd]

11.4.9 really has gotten worse in DC traffic for me. I have Progressive and my phone monitors all the driving stats since you save like 30 percent on insurance with it turned on. I'm getting hard braking dings on my insurance almost every drive to work now with this new version. Really likes to accelerate quickly when someone leaves a lane and then slam on brakes even on Chill.

Also has been making some bad decisions lately too, trying to go around stopped traffic on a one lane on ramp. Happens almost every time for me now where it tries to jerk to the right to go around stopped cars, even with minimal lane changes on.

Also the traffic flow speed add on sometimes makes bad decisions coming up to speed camera that are marked on the map. It will ignore the speed change and just quickly go through areas because everyone else is too.

Overall not happy with this version. 4.4 was pretty great.

How does Progressive get Tesla Safety scores information?

 

[JulienW]

How does Progressive get Tesla Safety scores information?

Likely uses a Progressive app that works in ALL cars and nothing to do with Tesla.

 

[Twiglett]

11.4.9 really has gotten worse in DC traffic for me. I have Progressive and my phone monitors all the driving stats since you save like 30 percent on insurance with it turned on. I'm getting hard braking dings on my insurance almost every drive to work now with this new version. Really likes to accelerate quickly when someone leaves a lane and then slam on brakes even on Chill.

Also has been making some bad decisions lately too, trying to go around stopped traffic on a one lane on ramp. Happens almost every time for me now where it tries to jerk to the right to go around stopped cars, even with minimal lane changes on.

Also the traffic flow speed add on sometimes makes bad decisions coming up to speed camera that are marked on the map. It will ignore the speed change and just quickly go through areas because everyone else is too.

Overall not happy with this version. 4.4 was pretty great.

lesson to be learned - don't use FSD and expect good insurance driving scores with any company, that only happens with Tesla insurance.
Several companies use their own app now, so when FSD does something stupid you will get dinged for it.

 

[archae86]

Regarding the eternal debate as to whether checking for updates makes any difference, just now I hopped in my MY 2023 model 3 RWD and tapped the software item. After a few seconds with the "checking for updates" indications, it stated that 2023.44.30.8 was initiating download. Car is currently on 2023.44.30.5.1.

It appears that 2023.44.30.8 is going "pretty wide" and is already the second highest number of cars in the TeslaFi subscriber base for installations.

My personal opinion is that the "does checking matter" answer changes from time to time, and that probably this time for me it advanced my installation by at least a little.