FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[cbdream99]

Got the 30.6 now, was 30.4 couple of days ago.

 

[manole]

Finally getting the update now (2021 Refreshed Model S with USS).

 

[bhakan]

Seems like finally the refresh Model S (2021-2022) cars are getting the 5.1 updates. Feeling unloved and ignored. Hopefully, it is a not a lump of coal and a good update.

 

[Rikster]

Seems like finally the refresh Model S (2021-2022) cars are getting the 5.1 updates. Feeling unloved and ignored. Hopefully, it is a not a lump of coal and a good update.

Yep, I have a refresh Model S (2022) and I just got the push overnight also, so it seems you're right. I'm looking forward to some of the new features, and it's always "interesting" to see how FSDb changes with these updates. I use FSDb pretty much every drive, and it has a bunch of issues that I encounter every time (but have gotten used to). I'll have to get used to the new hiccups again with this update...

 

[boonedocks]

Is 30.6 for HW4 and 30.5.1 for HW3?

Nope. Our 2017 Model X HW3 FSD MCU2 has .6 and my 2021 Model S HW3 is installing .5.1

 

[EVNow]

FWIW, On at least some versions of MacOS, the auto reboot at the end of an update resets less than a normal reboot. So, I’ll continue my superstitious extra reboot after an install.

I remember the days when any and every change needed a reboot on Windows

 

[EVNow]

Maybe this was the cause?

View attachment 1002327

So the question is - does Tesla have a robust automated test suit .. they need it to avoid problems like these.

 

[sleepydoc]

I’ve been using the high fidelity park assist on a rental Model 3, and it’s actually pretty cool. Dunno how well this would work in the rain, but I’m pleased so far. I think it can be improved, but no doubt it could be very useful for those pesky parking concrete things low down that USS can’t deal with etc.

I have USS so I don’t get to play with it But I suspect it will improve. This is really v1.0 of the feature. They should have had it up and running before they removed the USS, but at least it’s coming out now.

There’s a post in another thread from someone who says he got curb rash because it wasn’t accurate so I wouldn’t trust it for distances less than 6”, but I wouldn’t trust USS for distances less than 6” either and USS doesn’t do a thing for curbs anyway.

 

[sleepydoc]

I remember the days when any and every change needed a reboot on Windows

Any you’d have to reinstall windows every 6 months or so regardless.

 

[GWord]

So the question is - does Tesla have a robust automated test suit .. they need it to avoid problems like these.

They almost certainly do, but they also have an approximately 1 metric buttload of different vehicle configurations in the wild, resultant of their policy to integrate design changes at any time. I would hypothesize that this results in a more challenging QA environment than is typical (unless you are an Android developer )

 

[sleepydoc]

So the question is - does Tesla have a robust automated test suit .. they need it to avoid problems like these.

Agreed - it seems like this roll out was royally messed up. I wonder if they suddenly had a bunch of extra work to do for the recall ‘fix’ and didn’t have time to do the normal testing.

Sandy Munro has also commented how he likes Tesla’s method of adding hardware changes to cars as soon as they’re ready rather than waiting for a new model year. The problem with this approach is you have far more combinations out on the road. Rather than just a 2023 M3 you have a 2023, 2023.1, 2023.2, 2023.3, etc. This potentially makes software testing much more difficult. (Edit: @GWord just posted my exact thoughts!)

 

[hybridbear]

Agreed - it seems like this roll out was royally messed up. I wonder if they suddenly had a bunch of extra work to do for the recall ‘fix’ and didn’t have time to do the normal testing.

Sandy Munro has also commented how he likes Tesla’s method of adding hardware changes to cars as soon as they’re ready rather than waiting for a new model year. The problem with this approach is you have far more combinations out on the road. Rather than just a 2023 M3 you have a 2023, 2023.1, 2023.2, 2023.3, etc. This potentially makes software testing much more difficult. (Edit: @GWord just posted my exact thoughts!)

Has anyone anywhere tracked these changes? I know back in 2016 there used to be something for the S. The 2016 S had major changes.

 

[jebinc]

Season’s Greetings and Happy New Year, all!

 

[scottf200]

I got 30.4 a couple day ago and 30.6 today.

Everyone:

Aside: my installs

 

[BBTX]

Got 5.1 here (from .2, on HW3, so that matches what others are reporting).

Back to FSDb things though: I had noted 11.4.9's increased confidence earlier, but yesterday it misplaced its confidence twice!

At two distinct intersections now I've had it take a left aggressively without as much creep/look behavior as before, and the result was it missed a car coming from the right and tried to pull out confidently right in front of the other car going less than half the other car's speed. Obviously I disengaged before we actually came into their path in both cases, but I'm sure they were at least covering their brakes and annoyed.

Keep an eye out for 11.4.9 overconfidence!

 

[kabin]

So the question is - does Tesla have a robust automated test suit .. they need it to avoid problems like these.

They almost certainly do, but they also have an approximately 1 metric buttload of different vehicle configurations in the wild, resultant of their policy to integrate design changes at any time. I would hypothesize that this results in a more challenging QA environment than is typical (unless you are an Android developer )

Yep, maybe both test and configuration management. Live by the shortcut, die by the shortcut. Unless it was Elon approved, I'd be surprised if a couple of folks in the loop didn't lose their jobs.

 

[Ramphex]

Can anyone report any new useful features in the super awesome holiday update?

 

[kabin]

Got 5.1 here (from .2, on HW3, so that matches what others are reporting).

Back to FSDb things though: I had noted 11.4.9's increased confidence earlier, but yesterday it misplaced its confidence twice!

At two distinct intersections now I've had it take a left aggressively without as much creep/look behavior as before, and the result was it missed a car coming from the right and tried to pull out confidently right in front of the other car going less than half the other car's speed. Obviously I disengaged before we actually came into their path in both cases, but I'm sure they were at least covering their brakes and annoyed.

Keep an eye out for 11.4.9 overconfidence!

Yes! In this case assertiveness isn't because 11.4.9 knows what it's doing. They opted for less latency in lieu of safety.

 

[BBTX]

Oh and another interesting 11.4.9 behavioral change from yesterday: I was traveling down a road in a 50mph section at 55 (normal traffic speed, with +10% in my settings matching). There was a posted speed limit change down to 30mph. Most of the traffic ahead of me pretty much ignored the change, and FSDb matched them. The screen showed the limit change to 30, but had my set speed still at 55 but in blue, and popped a message up that said something to the effect of it maintaining speed with the traffic or conditions or something.

I've seen it go under-limit based on the road characteristics (narrow, driveways), but I've never before seen it choose to ignore a speed limit sign just because everyone else is doing it.

 

[GWord]

Oh and another interesting 11.4.9 behavioral change from yesterday: I was traveling down a road in a 50mph section at 55 (normal traffic speed, with +10% in my settings matching). There was a posted speed limit change down to 30mph. Most of the traffic ahead of me pretty much ignored the change, and FSDb matched them. The screen showed the limit change to 30, but had my set speed still at 55 but in blue, and popped a message up that said something to the effect of it maintaining speed with the traffic or conditions or something.

I've seen it go under-limit based on the road characteristics (narrow, driveways), but I've never before seen it choose to ignore a speed limit sign just because everyone else is doing it.

I think this was introduced in the early 11.4 branches actually. My car has been doing this for a while now. I have found it to be an elegant solution to driving with socially acceptable behavior yet still trying to comply with the NHTSA recall notice about speed limit change responsiveness.