FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[kabin]

Cars previous builds were v11.3.6
FSD Beta 11.4.4 (2023.26.10) Official Tesla Release Notes - Software Updates

11.4.4 misery loves company.

 

[Knightshade]

Given they don't happen very often, at least around here, they are probably low on the list. No right on red signs should be way higher on the list.

Given what most signs like that look like in HW3 cameras vs in HW4 ones I'm not sure I'd hold my breath for significant additional sign reading coming to HW3. I suppose right on red would be among the easier ones since at least in theory the car isn't moving when it reads it?

https://twitter.com/x/status/1694228602211188851

Seems to me that's a BAD sign for 11.4.7. If it was close to ready why would they roll 11.4.4 to a bunch of cars instead?

 

[johnny_cakes]

Interesting episode titled "The Self-Driving Experiment" from the Land of the Giants podcast: ‎Land of the Giants: The Self-Driving Experiment on Apple Podcasts

It highlights Tesla's FSD, what we were promised, and where things stand today. I'd say their experience testing FSD Beta it is similar to mine.

Summary of the episode if you don't want to spend the time listening to it is also available here.

 

[kabin]

Interesting episode titled "The Self-Driving Experiment" from the Land of the Giants podcast: ‎Land of the Giants: The Self-Driving Experiment on Apple Podcasts

It highlights Tesla's FSD, what we were promised, and where things stand today. I'd say their experience testing FSD Beta it is similar to mine.

Summary of the episode if you don't want to spend the time listening to it is also available here.

Tesla has so much riding on FSD's success. Gotta wonder if Elon transitioned from an honest 'cup half full' personality to a desperate opportunistic liar.

 

[uscbucsfan]

Given what most signs like that look like in HW3 cameras vs in HW4 ones I'm not sure I'd hold my breath for significant additional sign reading coming to HW3. I suppose right on red would be among the easier ones since at least in theory the car isn't moving when it reads it?





Seems to me that's a BAD sign for 11.4.7. If it was close to ready why would they roll 11.4.4 to a bunch of cars instead?

There's no evidence that FSD Beta can read any signs outside of speed and it's not likely reading it, just recognizing the pattern of the numbers trained in to their model.

The reason I say this is because almost every "No right of red" is different, so that would be extremely difficult to do. I think they were attempting to solve this with the car's reported map data and user interactions, but that's not a solution you could push out globally.

It's not impossible to program every single no right on red variant, including ones with specific times, but I would guess it's ridiculously low on the priority as it's not actual FSD and given that School Zones or specialty speed limits haven't been programmed in, nor bus stops, I think it's unlikely we see it for a while.

I agree about 11.4.7, too.

 

[johnny_cakes]

There's no evidence that FSD Beta can read any signs outside of speed and it's not likely reading it, just recognizing the pattern of the numbers trained in to their model.

I'm curious about its ability to recognize speed signs. In a recent road trip in BC, Canada, FSD Beta often didn't detect speed limit changes. This was especially true for temporary speed limit signs in construction zones (even though they used the exact same signs), but it seemed to also happen occasionally with permanent speed limit signs.

 

[JulienW]

There's no evidence that FSD Beta can read any signs outside of speed and it's not likely reading it, just recognizing the pattern of the numbers trained in to their model.....

....and Stop signs. Also I remember a video with a non standard speed limit sign of 12MPH and FSD Beta identified it as 15MPH. So saying it "reads" is anthropomorphism.

 

[uscbucsfan]

I'm curious about its ability to recognize speed signs. In a recent road trip in BC, Canada, FSD Beta often didn't detect speed limit changes. This was especially true for temporary speed limit signs in construction zones, but it seemed to also happen occasionally with permanent speed limit signs.

They have to be a certain size and color to work, but there's plenty of test proving it does.

My car has picked up a speed limit from the back of a semi that had a perfect sign and changed it's speed because of it. People have also fooled FSD with take on a Speed Limit sign.

 

[uscbucsfan]

....and Stop signs. Also I remember a video with a non standard speed limit sign of 12MPH and FSD Beta identified it as 15MPH. So saying it "reads" is anthropomorphism.

And it doesn't read stop signs. It recognizes a red octangular sign. We've seen videos of faded stop signs, grey ones, or bent ones not being recognized by FSD.

 

[JulienW]

Don't disagree except I think they should move straight to V12.

Elon started promoting V11 a full year before it was ready for release. So it is highly unlikely that V12 is even close to being ready.

 

[JulienW]

And it doesn't read stop signs. It recognizes a red octangular sign. We've seen videos of faded stop signs, grey ones, or bent ones not being recognized by FSD.

Didn't say it "reads" and the latter part of my post I even say it "identifies" and the saying it is reading is anthropomorphism of FSD Beta.

 

[uscbucsfan]

Read my post and the whole post. Didn't say it "reads" and the latter part of my post I even say it "identifies" and the saying it is reading is anthropomorphism FSD Beta.

Agreed, but it's important to distinguish, especially in the context of no right on red. There are hundreds of variations of that sign. It's a difficult one to include.

 

[JulienW]

Agreed, but it's important to distinguish, especially in the context of no right on red. There are hundreds of variations of that sign. It's a difficult one to include.

I suspect that using map data to identify "no turn on red" is going to be a far better option than trying to identify all the non standard and oddly placed signs.

 

[fasteddie7]

This also puts those of us with 11.4.6 in a weird place, base software is below, FSDb software is above. Wonder how that's gonna work.

 

[JulienW]

This also puts those of us with 11.4.6 in a weird place, base software is below, FSDb software is above. Wonder how that's gonna work.

Not as f'n weird as us that have 11.4.4/23.7.20, but you are close. This is a mess and makes me pissed off. Well at least not pissed on.

 

[uscbucsfan]

This also puts those of us with 11.4.6 in a weird place, base software is below, FSDb software is above. Wonder how that's gonna work.

Agreed.

I have a feeling this 11.4.4 is not for actual "testers" like the previous 11.3.6 was, but I hope I'm wrong.

 

[kabin]

Interesting episode titled "The Self-Driving Experiment" from the Land of the Giants podcast: ‎Land of the Giants: The Self-Driving Experiment on Apple Podcasts

It highlights Tesla's FSD, what we were promised, and where things stand today. I'd say their experience testing FSD Beta it is similar to mine.

Summary of the episode if you don't want to spend the time listening to it is also available here.

Yep. I listened to the first 10 mins or so and their FSD test drive was the same as mine. Lots of interventions and missing the mark of a reliable, relaxed safe drive.

 

[rlsd]

Expecting a software company to release new versions of software within couple weeks is not realistic.
We should not act like kids asking their mom to give uncooked food then complain the food is bad.
Serious software development needs time to test.

How long do I have to wait for the Maps a… - Apple Community

discussions.apple.com

 

[kabin]

Expecting a software company to release new versions of software within couple weeks is not realistic.
We should not act like kids asking their mom to give uncooked food then complain the food is bad.
Serious software development needs time to test.

How long do I have to wait for the Maps a… - Apple Community

discussions.apple.com

Teslafi's database would beg to differ. If it was serious software....

 

[uscbucsfan]

Expecting a software company to release new versions of software within couple weeks is not realistic.
We should not act like kids asking their mom to give uncooked food then complain the food is bad.
Serious software development needs time to test.

How long do I have to wait for the Maps a… - Apple Community

discussions.apple.com

To be fair, Elon originally said FSD Beta would get a new update every two weeks. That didn't last long, but the expectation is driven by Elon continuously saying a new update is coming, when it doesn't, coupled with bad experiences with the current build.