FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Mardak]

Another new caption in 11.4? It seems like Tesla is eventually going to tell you everything the car is thinking

I believe I've seen the "Changing lanes away from merge" message (old oval message style in the bottom left) in 10.x potentially back to 10.5:

• Enabled behavior to lane change away from merges when safe to do so.
• Improved merge object detection recall by using multi-modal object prediction at intersections.
• Improved control for merges by increasing smoothness of arrival time constraints and considering possible merging objects beyond visibility.

But in general, it is good to have more insights into the thinking as it can potentially reveal how FSD Beta treats similar situations differently. For example, the "Choosing right fork to stay on route" does seem to be new in 11.4 but doesn't seem to show up for getting into temporary turn lanes that "fork" from the main straight lane, so does that mean it's behaving differently?

Similarly, 11.4 has new behavior to slow down in certain situations "Limiting max speed for low-speed road" (where I suppose the driver can manually adjust the max speed up if necessary):

• Added new Vision Speed network to infer the typical driving speed on a given road. This is used to limit the maximum allowed speed in environments such as parking lots and residential roads.

In this case, it dropped the max speed from 25mph (seems like +25% relative offset) to 24mph although it already knew the speed limit (seemingly from map data) was 20mph. Maybe this functionality was added to help address bad map data as well as infrequent/occluded speed limit signs and 11.4's continuing failure to detect some speed limit signs:

 

[Scott Chapman]

Another new caption in 11.4? It seems like Tesla is eventually going to tell you everything the car is thinking.

Totally agree. I think a lot of my disengagements are often me being startled by what it is doing when it is different than I might have done.
Plan disclosure is going to be important for folks to know when intervention is actually necessary versus introducing more risk

 

[Genie]

Totally agree. I think a lot of my disengagements are often me being startled by what it is doing when it is different than I might have done.
Plan disclosure is going to be important for folks to know when intervention is actually necessary versus introducing more risk

I know that none of my interventions have introduced more risks. I have intervened for comfort but it’s usually for safety.
Yesterday a mail truck was delivering on a 50mph county road, stopped at a mailbox with its hazards on and I gave it as long as I could to do the right thing before I took over, it never slowed at all.
Then minutes later slowed 10mph with no cars around, maybe sun angle.
That’s 2 types of interventions I find I have to do pretty often.

 

[Pdstaib9]

Does anybody know if the FSD beta enrollment has reopened? I wonder if I subscribe how soon I will get 2023.15. Currently on 2023.12.1.

 

[powertoold]

These are the kinds of challenges that Tesla faces when merging highway / city / parking lots into one stack. 3.6 probably thought it was in a parking lot when it wasn't. We saw this more often in the early 11.3 builds:

https://twitter.com/x/status/1652465627485929473

 

[willow_hiller]

Sounds like Elon is already testing 11.4.1. Maybe we'll get a full release by 11.4.2:

https://twitter.com/x/status/1652717957355642882

 

[powertoold]

Sounds like Elon is already testing 11.4.1. Maybe we'll get a full release by 11.4.2:

https://twitter.com/x/status/1652717957355642882

I'm loving the 11.4 videos. This year might finally be the year we get to 2x safety fsd with no interventions in most of the USA by land area.

 

[archae86]

Out of the blue my Y on FSDb 11.3.6 offered me autopark this morning. I had just pulled to a stop meaning to back into a space, when I noticed invitation on the screen and accepted it. It did a smooth and competent job.

 

[gsmith123]

Sounds like Elon is already testing 11.4.1. Maybe we'll get a full release by 11.4.2

Hopefully! Feels like this has been a relatively slow rollout from first employee sightings of 11.4.1 to first public tester releases, considering it's a much smaller release than switching to FSD code on highways.

 

[willow_hiller]

Hopefully! Feels like this has been a relatively slow rollout from first employee sightings of 11.4.1 to first public tester releases, considering it's a much smaller release than switching to FSD code on highways.

I don't think we've seen 11.4.1 on an employee vehicle, yet, have we? Last I saw it was 11.4, so this Tweet from Elon represents the first point release update from 11.4.

 

[gsmith123]

Oh, right, sorry I was thinking of the 11.4 Youtube videos.

 

[jebinc]

Didn’t someone say, on Friday night, that they had a good internal source that a “wide rollout of 11.4.x” was highly likely this weekend? I recall that one as I quickly applied the following reaction score to that post - “”

 

[JohnB007]

Is 2023.12.5 the next FSD? It differently 3 appeared in TeslaFi

 

[JohnB007]

Oh and another interesting fact. I've been voice reporting one spot where the car tries to completely stop in the middle of the road. This used to be temporary traffic light during construction about two years ago.

Could be a fluke, but finally, it ignored it and moved ahead! I didn't receive any new map, so Tesla must have a way to receive new mapping data without getting the full update.

 

[archae86]

Is 2023.12.5 the next FSD? It differently 3 appeared in TeslaFi

Seems unlikely. The two versions reported as previous installations by TeslaFi are 2023.12.1.1 and 2023.6.11, neither of which is listed as an FSDb version.

 

[JHCCAZ]

Is 2023.12.5 the next FSD? It differently 3 appeared in TeslaFi

I'm pretty sure that 2023.12.5 is simply an up-rev of the current wide software, not FSD beta. The latest known employee-testing only FSD beta is 11 4 on software version 2023.6.15. This is probably moving now lto 11.4.1 that Elon mentioned having. It is unlikely but possible that they have moved 11.4.1 onto software 2023.12.x.

Consistent with this, TeslaFi's listing of 2023.12.5 does not have the small tag text that we show a corresponding FSD beta version. (If you look for 2022.45.15 on TeslaFi, you'll see the tag for FSD beta 11.3.6.)

Also note, for FSD beta releases, the third number x it's almost always 15 or higher.

 

[aronth5]

Didn’t someone say, on Friday night, that they had a good internal source that a “wide rollout of 11.4.x” was highly likely this weekend? I recall that one as I quickly applied the following reaction score to that post - “”

That was a YouTube comment from TeslaBull FSDbeta who is posting the V11.4 videos. He stated being hopeful but never said highly likely.

 

[jebinc]

That was a YouTube comment from TeslaBull FSDbeta who is posting the V11.4 videos. He stated being hopeful but never said highly likely.

There’s a post in this thread or the master thread on Friday night that suggested this. I know as I was sure to “react” to it. Perhaps that OP was referring to the YT video you mention, idr. All I recall is how nonsensical the view was.

 

[Scott Chapman]

So... While we wait...

https://twitter.com/x/status/1652717957355642882

What is he saying that 11.4.1 is promising exactly?

 

[aronth5]

So while V11.3.6 continues with some nutty behaviors yesterday I drove 150 miles on the highway. Zero interventions but more importantly on my return drive (75 miles) it rained hard the entire time. To the point that when I had trouble seeing the highway lines myself I manually reduced the speed to 55mph. I could feel FSD was searching out the lines too as the car very slightly moved back and forth in the lane. But safe. Most everyone was staying at 60mph or below since visibility was pretty bad. But for me the key point is that FSD never disengaged. Before V11 FSD would have definitely disengaged so the drive was a major milestone. It was also the 4th time I've driven in moderate to heavy rain on V11.3.6 and FSD has not disengaged.

I had my doubts this would happen but it has so nice job Tesla development team.