FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[rlsd]

Who gets to decide how much 11.4.x training compute gets allocated away from v12?

https://twitter.com/x/status/1686513363495346178

There's probably business goals such as getting some 11.4.x stable enough to deploy to new HW4 vehicles, but that would come at the cost of delaying v12 alpha testing then early access then eventual wide deployment to reach other goals. Seems likely there'll be regressions in early alpha versions that will take some time to work through either by restoring some control code or collecting enough training data to generally solve regressions.

"There's probably business goals such as getting some 11.4.x stable enough to deploy to new HW4 vehicles, but that would come at the cost of delaying v12 alpha testing then early access then eventual wide deployment to reach other goals."

V12 needs V11 for training. 11.4.x needs to be fixed so that more users can use FSD to feed data back for training. V12 needs data for all difficult situations that V11 collects in the next several months. Besides, Tesla needs to fix critical bugs so that 11.4.x can be used for new Tesla owners with HW4. This will be used to push year end sales.

 

[Jeff N]

From the "other companies can screw this up, too" department:

I had a Cruise taxi almost hit me last week in downtown San Francisco. It was shortly before 10 pm. Intersection of Mason and O'Farrel. I was most of the way across Mason with an active walk signal when the Cruise came eastbound on O'Farrel and made a right turn across the crosswalk without slowing down. I basically jumped to the curb.

I'm not sure if it would have actually hit me, but if it missed it would have still come way closer than any car should come to a pedestrian without substantially slowing. Inches, not feet. It either completely failed to see me or it has a very warped idea of personal space for a pedestrian.

Likely the latter. In some circumstances, as a Cruise backseat test rider, I’ve seen it drive too close (in my opinion) to pedestrians in crosswalks even in unobstructed daylight. They also drive a little too close behind vehicles ahead of them (tailgating). In general, I’d like to see them give a little more space between the Cruise car and adjacent objects in typical driving not because they are actually going to hit anything but because it sometimes makes me (and other drivers or pedestrians) uncomfortable. I’ve given them feedback on that.

 

[Rocinante777]

Likely the latter. In some circumstances, as a Cruise backseat test rider, I’ve seen it drive too close (in my opinion) to pedestrians in crosswalks even in unobstructed daylight. They also drive a little too close behind vehicles ahead of them (tailgating). In general, I’d like to see them give a little more space between the Cruise car and adjacent objects in typical driving not because they are actually going to hit anything but because it sometimes makes me (and other drivers or pedestrians) uncomfortable. I’ve given them feedback on that.

I can sort of see the tailgating decision, since I assume it doesn't need to account for human response time in its following distance. OTOH, I agree they need to factor the comfort of other drivers in that equation. But I think passing that close to a pedestrian in a crosswalk (close enough to make the pedestrian try to dodge the car) would probably get a human driver ticketed if a police officer was present. (Maybe not in SF, I guess.)

I am now considerably less surprised by reports of local residents complaining about and even sabotaging them.

Update: I just looked up the Texas rules on a pedestrian in a crosswalk. They are pretty explicit to say that a vehicle may not cross the crosswalk if a pedestrian is in the half of the road through which the vehicle is traveling or about to enter that half of the road. So this would have been a clear violation. I could not find such a requirement for CA, other than a due care requirement and maybe a requirement to slow down. I wonder if Cruise taxis behave differently in Austin than in San Francisco.

 

[swedge]

I see that TeslaFi is now indicating that 2023.26.x contains 11.3.6. Phew! I was worried that they had pulled FSD from the production releases.

But the rollouts of newest version on both main and beta branches appear to be on hold still.

 

[Larry in TN]

I just added FSD and have v11.1 (2023.20.9).

I would like to have Autopilot > Offset set to FIXED but the choice is greyed out and set to PERCENTAGE.

The Set Speed setting, directly above, is also greyed out on SPEED LIMIT. Not sure if that is related.

Am I missing some setting or is this not changeable on my build?

Thanks.

 

[MP3Mike]

I would like to have Autopilot > Offset set to FIXED but the choice is greyed out and set to PERCENTAGE.

The Set Speed setting, directly above, is also greyed out on SPEED LIMIT. Not sure if that is related.

Am I missing some setting or is this not changeable on my build?

Due to complaints from NHTSA the fixed option was removed for FSD, the only option for the offset is percentage based.

 

[Mardak]

V10 lasted for 1½ years and Elon started V11 talk about work on it a year before its release (look at the date of the start of this thread). V11 is still less than 5 months old.

Yeah, we could be on 11.x for a long time similar to 10.x going up to 10.69. But to be clear, "11" was specially reserved for "single stack" highway driving, so major architectural changes like the addition of Occupancy Network were "just" a 10.x minor version. Maybe 12's Control Network will similarly require other architectural changes incrementally tested as part of 11.x?

 

[Mardak]

Long term it seems more efficient to drop 11.x.x entirely and focus fully on v12

I wonder if 11.4.x hasn't replaced 11.3.6 on the main vehicle software branch because engineers have been busy with v12? Elon Musk did say they're not engineer constrained, so maybe some can be spared to integrate 11.4.4? People have reported regressions, so maybe Tesla's data also shows 11.4.x so far isn't safe enough compared to 11.3.6, but I guess not unsafe enough to push us back?

What are those engineers doing when waiting for training to complete?

 

[kpanda17]

I’m 11.3.6, what do I do to get to 11.4.6/7?

 

[LowlyOilBurner]

I’m 11.3.6, what do I do to get to 11.4.6/7?

Wait.

 

[Twiglett]

I wonder if 11.4.x hasn't replaced 11.3.6 on the main vehicle software branch because engineers have been busy with v12? Elon Musk did say they're not engineer constrained, so maybe some can be spared to integrate 11.4.4? People have reported regressions, so maybe Tesla's data also shows 11.4.x so far isn't safe enough compared to 11.3.6, but I guess not unsafe enough to push us back?

What are those engineers doing when waiting for training to complete?

I think that its almost certain that 11.4.4 hasn't replaced 11.3.6 because they introduced so many regressions that after multiple attempts(11.4.3/4/5/6) they still can't make it goo enough to release. Each step after 11.4.2 seemed to get worse and worse, so much so that they can't even complete the release to the rest of the 11.4.x users yet.

 

[kpanda17]

I think that its almost certain that 11.4.4 hasn't replaced 11.3.6 because they introduced so many regressions that after multiple attempts(11.4.3/4/5/6) they still can't make it goo enough to release. Each step after 11.4.2 seemed to get worse and worse, so much so that they can't even complete the release to the rest of the 11.4.x users yet.

Crazy, in software development it’s called version control
You don’t move forward to the next release until priority bugs are fixed

 

[JulienW]

New V11 poll of this week. Will we see evidence (not release) of 11.4.7 before next week.

I going out on a limb and say 50-50 chance.

 

[kpanda17]

Waiting for anything v11.4 to hit the vehicle
Weird being stuck on 11.3.6

 

[rlsd]

We scare Tesla. They won't announce any new version anymore. We will only know the new version when it's released to us.

 

[JulienW]

Waiting for anything v11.4 to hit the vehicle
Weird being stuck on 11.3.6

That is because you are not an "early access" Beta tester and are running the last stable branch released (boy do I miss 11.3.6 ) . You will have to wait until there is a stable branch of 11.4.x before it is released on the "regular" software updates.

 

[Pdubs]

Waiting for anything v11.4 to hit the vehicle
Weird being stuck on 11.3.6

Be careful what you wish for

 

[kabin]

New V11 poll of this week. Will we see evidence (not release) of 11.4.7 before next week.

I going out on a limb and say 50-50 chance.

I gotta wonder if HW3 even has capacity to retain more training data without cognitive regression emesis. At some point they could train with a fifth gen Dojo until the cows come home and it won't make a difference. Cost over function is alive and well at Tesla.

 

[AlanSubie4Life]

TG AlanSubie4Life is back!!

Welcome back @AlanSubie4Life!

I never left.

There was just nothing left to say and nothing new to add.

Same stuff, different day.

This is the Acceptance stage, and it is good to see people finally coming to terms.

And (shocked, not shocked) it of course has no impact on the ability of Tesla to be successful as a business. It just doesn’t matter. It would only matter if someone else had general autonomy, and of course they do not, nor is there evidence they will soon. Someday maybe!

Looking forward to incremental improvements to continue to improve the L2 performance for the next couple years, then I think HW3 will be kind of wrapped up with a bow. I think that is the general conventional wisdom at this point.

I thought it would take at least 5-10 years, four years ago (March 3rd, 2019, see post), and starting to think that might be fantastically optimistic! It seems like the advances in AI technology have only pushed out the time when this will actually work (current AI state of the art, while disruptive in many domains, seems wholly incompatible with performing the DDT!).

 

[Mardak]

Tesla needs to fix critical bugs so that 11.4.x can be used for new Tesla owners with HW4

Yeah, and it's not just FSD Beta for new HW4 vehicles. The switch away from ultrasonic sensors only recently got Park Assist presumably based on 11.x Occupancy Network predictions, but there's other features like Summon that are still missing and could happen before v12. I don't know the take rate of Enhanced Autopilot these days, but if FSD Capability is still around 20%, there's probably a lot of grumpy customers that paid for features that still don't work on their new cars.