FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[aronth5]

WTFK

Having read many who say 11.4.4 is worse than 11.4.2 and many who say the opposite. As someone who was on 11.4.2 for a looooong time and just got 11.4.4 so far I find them anecdotally indistinguishable and an overall regression from 11.3.6. So what the hell is Tesla next move with this convoluted world of multi FSD Betas, software combinations and HW3/HW4?

Simple 11.4.5 which is now rolling out to employees. according to Teslascope. Probably supports L6

https://twitter.com/x/status/1677618232889593857

 

[Alset2]

Well, I finally had something happen that I have not seen that seems worth posting.
Pulled up to intersection with FSDb engaged. Been here dozens of times and this is new. I had to disengage because it was trying to go around the car in front of me.
Its kind of funny...except that its not
11.4.4

 

[LowlyOilBurner]

Simple 11.4.5 which is now rolling out to employees. according to Teslascope. Probably supports L6

https://twitter.com/x/status/1677618232889593857

At least: , and magical

 

[Supcom]

Well, I finally had something happen that I have not seen that seems worth posting.
Pulled up to intersection with FSDb engaged. Been here dozens of times and this is new. I had to disengage because it was trying to go around the car in front of me.
Its kind of funny...except that its not
11.4.4
View attachment 954312

FSDb used to do this quite often, especially at a long line of cars at a stop sign, but has improved greatly. BTW, you could have used the turn signal to prevent the lane change instead of disengaging.

 

[kabin]

Well, I finally had something happen that I have not seen that seems worth posting.
Pulled up to intersection with FSDb engaged. Been here dozens of times and this is new. I had to disengage because it was trying to go around the car in front of me.
Its kind of funny...except that its not
11.4.4
View attachment 954312

They must be using training data from emergency vehicles at intersections. They just need sirens and lights to complete the effect.

 

[kabin]

FSDb used to do this quite often, especially at a long line of cars at a stop sign, but has improved greatly. BTW, you could have used the turn signal to prevent the lane change instead of disengaging.

FSD is pretty stubborn and will likely try again - I disengage.

 

[JulienW]

Simple 11.4.5 which is now rolling out to employees. according to Teslascope. Probably supports L6

https://twitter.com/x/status/1677618232889593857

I say screw 23.7.x/11.4.5 since it will be relatively the same (I know some will swear it is much better or much worse). I'd be 100x more happy to just see 11.4.4 in 23.20.x or even 1000x more happy to see it in 23.26.x. Now that would be REAL objective improvements we could all agree on.

 

[fasteddie7]

I’ve got a new one. Two stop signs near each other, it now stops at the first and as it turns left into the other, it sees it as the same sign and blows right through it.

 

[Pdubs]

Well, I finally had something happen that I have not seen that seems worth posting.
Pulled up to intersection with FSDb engaged. Been here dozens of times and this is new. I had to disengage because it was trying to go around the car in front of me.
Its kind of funny...except that its not
11.4.4
View attachment 954312

Most of my disengagements on this update have been due to this. I say most because after this stupid bullsht, I don’t reengage. It is untenable.

 

[Pdubs]

FSDb used to do this quite often, especially at a long line of cars at a stop sign, but has improved greatly. BTW, you could have used the turn signal to prevent the lane change instead of disengaging.

Dude, when you’re stopped behind a line of cars at a light, (my hands are on the yoke btw), there is no signal indicator prior to the attempt by FSDj. It just yanks the yoke right and gives it juice.

So no. BTW, There is no “just use the turn signal to prevent…”. It just moves. Very confidently and you just react.

 

[Twiglett]

Well, I finally had something happen that I have not seen that seems worth posting.
Pulled up to intersection with FSDb engaged. Been here dozens of times and this is new. I had to disengage because it was trying to go around the car in front of me.
Its kind of funny...except that its not
11.4.4
View attachment 954312

yep - 11.4.2 did that a few times and 11.4.4 does it more so.
11.4.x releases obviously have added extra "lane discipline, we don't need no stinking lane discipline"

 

[PACEMD]

My wife’s 3 does not have FSD and she got this version last night

That's dyslexic bigamy, isn't it?

 

[sleepydoc]

I’ve got a new one. Two stop signs near each other, it now stops at the first and as it turns left into the other, it sees it as the same sign and blows right through it. View attachment 954340

I was having issues with some temporary stop signs they put up at a few intersections on my way to work. First the car wouldn't stop at all. Not it stops but waits until the very last second to slam on the brakes. The interesting part is how it renders the stop signs. There are 2 signs, both placed right at the corners but FSD will display 3 or 4 at varying locations and sometimes turn the signs 90º. It's hard to tell what is strictly rendering vs what FSD is actually using but if it can't accurately tell where the signs are it's no wonder it has issues with other things.

 

[Alset2]

FSDb used to do this quite often, especially at a long line of cars at a stop sign, but has improved greatly. BTW, you could have used the turn signal to prevent the lane change instead of disengaging.

Thanks everyone for the feedback, first time I have ever seen it

My experience is what others stated, I tried canceling 3 or 4 times with turn signal, it just kept Turing signal right back on and inching forward with the wheel cranked ATW to the left...it was not waiting around for any cars at a light!

 

[Therealm80]

I still love the car even though every now and then it tries to kill me.

 

[swedge]

It also likely allows for Shadow Mode data collection in preparation for global FSD Beta deployment.

Ah! Good point.

(This pertains to why 11.3.6 is doing a huge rollout out in the production branch, 2023.20.7. Previously, FSD beta only went out to subscriber and purchasers who asked and agreed to also be "testers".)

 

[aronth5]

Helpful thread even though Telescope says they are just speculating on some of their answers.

https://twitter.com/x/status/1677764913778003968

 

[Tronguy]

Helpful thread even though Telescope says they are just speculating on some of their answers.

https://twitter.com/x/status/1677764913778003968

And one can't read it unless one has an account. Effectively paywalled.

 

[jebinc]

Dude, when you’re stopped behind a line of cars at a light, (my hands are on the yoke btw), there is no signal indicator prior to the attempt by FSDj. It just yanks the yoke right and gives it juice.

So no. BTW, There is no “just use the turn signal to prevent…”. It just moves. Very confidently and you just react.

FSDj, just being FSDj…

 

[kabin]

I’ve got a new one. Two stop signs near each other, it now stops at the first and as it turns left into the other, it sees it as the same sign and blows right through it. View attachment 954340

Yep. An FSD youtuber uploaded similar results.

It's as if the planner rules out any possible chance of a secondary stop sign. Maybe a hard code bug fix or another one of those short term, one-off fixes the team adds that eventually comes back to haunt. This is NHTSA recall stuff that still fails.