FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[emmz0r]

He does seem to be quite particular about the words he uses, so while unknown unknowns could cause delays to the provided timelines, the specific intent still is there. As you point out, the rollout would only be starting but the intent is to be wider, and as others point out, this weekend deadline could be missed.

For reference, he has referred to earlier rollouts as "limited beta" different from "wide release" maybe to differentiate internal/employee releases from general public:

https://twitter.com/x/status/1624693067143069702

Yes, it's confusing that people are probably generally asking about themselves, but it does provide some indication of progress even if only really noticeable internally.

"Then we adapt for EU roads & submit to regulators. 90% of what we’ve done so far for NA applies WW."

Oh boy, my car is gonna be on the junk yard before I get V11

 

[Supcom]

He does seem to be quite particular about the words he uses, so while unknown unknowns could cause delays to the provided timelines, the specific intent still is there. As you point out, the rollout would only be starting but the intent is to be wider, and as others point out, this weekend deadline could be missed.

For reference, he has referred to earlier rollouts as "limited beta" different from "wide release" maybe to differentiate internal/employee releases from general public:

https://twitter.com/x/status/1624693067143069702

Yes, it's confusing that people are probably generally asking about themselves, but it does provide some indication of progress even if only really noticeable internally.

We've all seen the staged rollouts here, so this should be nothing new. Just hope that you you get a good number in the lottery and are in the early group. I've received new software early and I've received it late.

 

[Mardak]

Not sure if others have noticed the new speed limit behavior in 11.3.1 working both ways? Unclear if it's an intentional change or accidental behavior where before Autopilot would apply the speed limit only when passing the sign even though it visualized it earlier whereas now it seems to update the speed limit almost when it is first visualized.

Here's a couple examples from Dirty Tesla going from 30mph to 35mph and Chuck Cook nearing a 30mph to replace bad 15mph map data:

By the time the former passed the sign, the car gets up to 33mph from the previous 30mph set speed, so the new behavior actually results in speeding up sooner too.

 

[Mardak]

I see no real improvements over 10.69.25.2, city or highway

Maybe this is too subtle for many to notice right away, but Black Tesla has a nice comparison of the same highway merges using 11.3.1's Autosteer/Navigate on Autopilot setting as well as FSD Beta setting:

In these two situations, the NoA visualizations for the path do jerk to the right indicating it wants to suddenly center along with jitter of the imagined lane lines to avoid even worse centering while FSD Beta 11.3.1 understands what lines are actually there and plans a smooth path.

 

[JB47394]

In these two situations, the NoA visualizations for the path do jerk to the right indicating it wants to suddenly center along with jitter of the imagined lane lines to avoid even worse centering while FSD Beta 11.3.1 understands what lines are actually there and plans a smooth path.

Criminy. It's about time. I just went on a long road trip, and I'd occasionally hit this with such a wide combined lane at a merge that the car would wander back and forth multiple times. People behind me must have thought I was losing my mind. I was, but not in the way they imagined.

The choice of focusing on centering in a lane really made me wonder if any of the FSD guys own a car.

Now I really want this update.

 

[AlanSubie4Life]

Maybe this is too subtle for many to notice right away, but Black Tesla has a nice comparison of the same highway merges using 11.3.1's Autosteer/Navigate on Autopilot setting as well as FSD Beta setting:
View attachment 918234

In these two situations, the NoA visualizations for the path do jerk to the right indicating it wants to suddenly center along with jitter of the imagined lane lines to avoid even worse centering while FSD Beta 11.3.1 understands what lines are actually there and plans a smooth path.

Finally somebody tested it! (Rather than highlighting great performance in non-problematic scenarios.)

I wonder if this means they fixed this in FSDb 11.3 from FSDb 10.69 (where it doesn’t work on surface streets). That would be an undocumented change.

Would be interesting to see that tested, since merges and lane centering with intermittent/non-existent right lane lines on surface streets are poorly handled by 10.69.25.2 (car wanders all over the place rather than sticking to the correct place in the partially unmarked lane).

He cannot do it though since he can’t do the comparison of course.

Could also be a consequence of range extension of the next-gen (which we already have) planner (though that could be a visualization thing only with no actual change, they do mention something about it in the release notes specifically for the freeway - “Extended maximum trajectory optimization horizon”).

 

[jeremymc7]

Would be interesting to see that tested, since merges and lane centering with intermittent/non-existent right lane lines on surface streets are poorly handled by 10.69.25.2 (car wanders all over the place rather than sticking to the correct place in the partially unmarked lane).

Beta hates bike lanes near me. One lane streets but with a small bike lane. Car goes back and forth between them and eventually just throws a red screen and forces me to take over.

I guess it’s better than before where it would insist on driving in the bike line, which is half the size of the car, and ignore the actual car lane.

 

[fobo]

After watching a couple of DirtyTesla’s v11.X videos, I see no real improvements over 10.69.25.2, city or highway.

Lol quality trolling at least

 

[aronth5]

That's right. Randomized rollouts don't bias towards high Safety Scores other than the fact that for earlier 10.x rollouts, only those with high enough scores got added to FSD Beta. The exception is for those in the actual early access program before the Safety Score requirement, and they still have the video snapshot button. Now that the FSD Beta population is around 400k, a 10% rollout for 11.x will reach much more people than the original group that got FSD Beta 10.2. However, this also means only roughly 10% / ~400 of that original group from October 2021 will be in the earlier phase of the rollout. So it's good news for those who joined FSD Beta late as they aren't required to wait for others to get 11.x first, and there's a lot of them as the original public beta group was around 4,000 vehicles vs now 400,000.

I guess I've been lucky then since I've gotten every new release right away after the YouTube influencers got it and I was part of the October 2021 group. Of course with V11 likely a significant improvement this time I'll get the update later.

 

[kabin]

Finally somebody tested it! (Rather than highlighting great performance in non-problematic scenarios.)

I wonder if this means they fixed this in FSDb 11.3 from FSDb 10.69 (where it doesn’t work on surface streets). That would be an undocumented change.

Would be interesting to see that tested, since merges and lane centering with intermittent/non-existent right lane lines on surface streets are poorly handled by 10.69.25.2 (car wanders all over the place rather than sticking to the correct place in the partially unmarked lane).

He cannot do it though since he can’t do the comparison of course.

Could also be a consequence of range extension of the next-gen (which we already have) planner (though that could be a visualization thing only with no actual change, they do mention something about it in the release notes specifically for the freeway - “Extended maximum trajectory optimization horizon”).

It seems improved but I've seen cases where it's still suboptimal. Merging on the freeway it hangs onto the right side line until the last moment robotically locked as if a Disney Autopedia ride even though the merging lane is clear. And another case on city streets where a right lane appeared before an intersection the ego momentarily began to hug the right side in error.

 

[AlanSubie4Life]

It seems improved but I've seen cases where it's still suboptimal.

I assume you mean in videos of 11.3.x.

Guess at some point we’ll all find out. I’d be shocked if it were just fixed.

Hanging on the right side on a merge doesn’t seem that bad if you’re on the right to start with (getting on to the road) and a lane change is not being issued. I do that myself and it definitely is a driver preference. And of course it depends on traffic and a ton of other factors. As long as there is no veering/hunting.

 

[PianoAl]

I'm hoping that from 11.3.1 to 11.3.2 they add the following:

Slow down faster
Turn blinker on sooner

despite how difficult those mods are.

 

[Pdubs]

I’ll be impressed when I can text @FSDtester#1 while the car drives, without any strikes or nags

I got you.

I do this all the time. Email, Text....while in NoAP or FSDb, hold your phone in both hands extended past your yoke, in-between the yoke and the dash. Cameras can't see it and you can do it for as long as you need to text FSDtester#1, your wife, your kids or even me.

I'd keep an eye out for Five-0 though.

 

[gsmith123]

Hanging on the right side on a merge doesn’t seem that bad if you’re on the right to start with (getting on to the road)

I wonder if this is a regional preference and/or depends on road design? Around here people just do not do this.

You can but the normal way to merge is to smoothly move left as soon as possible. If you just hang out in the merge lane as long as possible until the right lane line bumps you in, it's harsher merge than necessary. There's usually bit of loose dirt and debris at the end, showing how people avoid that area.

 

[Pdubs]

That’s not impressive to me. I want both of my hands on deck typing borderline inappropriate text messages to @FSDtester#1 while driving

why borderline?

 

[aronth5]

I got you.

I do this all the time. Email, Text....while in NoAP or FSDb, hold your phone in both hands extended past your yoke, in-between the yoke and the dash. Cameras can't see it and you can do it for as long as you need to text FSDtester#1, your wife, your kids or even me.

I'd keep an eye out for Five-0 though.

I hope you're not serious. I've never typed a text message in 5 years in my Tesla's while driving. Voice only.

 

[Pdubs]

I hope you're not serious. I've never typed a text message in 5 years in my Tesla's while driving. Voice only.

grats

 

[Ramphex]

why borderline?

I don’t want to get him too excited, you know

 

[LowlyOilBurner]

Lol quality trolling at least

Trolling? Ok…. I’ve never posted any sort of “trolling” posts on this board. This is just my take. If you’re upset about the $15,000 you paid for FSD, I feel your pain, only to the tune of $6,000.00, than again at $12,000.00. IMHO, until I try the new hot jam for myself, I’m a bit disappointed in what I see. That’s all. No need to get upset. Just an opinion. It will be OK. I’m sure HW4 will remedy all, in a few months.

 

[EVNow]

You need more freinds

Not the kind that encourage texting while driving...