FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[bayareaever]

Most of the important changes slated for V11 have already been made. The biggest change in V11 is single stack - which shouldn't affect city driving at all. So, I'd keep my expectations low - yes, I'm betting on "like just another point release, which have had notable but only incremental improvements along with some regressions".

Maybe that was the original plan but I personally don’t think enabling current fsdb at highway speeds with multiple lanes and traffic would be very safe, mainly because of hard braking but for other reasons too. High profile, high speed car accidents are definitely the last thing they need. NoA has hard braking too, but not to the same degree as fsdb. They might have realized that a few months ago and have had to keep refining before release which would jive with Elon’s recent comments.

 

[JulienW]

Most of the important changes slated for V11 have already been made. The biggest change in V11 is single stack - which shouldn't affect city driving at all.....

Actually we should stop saying Single Stack since it's not a SINGLE Stack. Maybe should be called Combined or Dual Stack *now. Single Stack would include all ("complex") parking lots, "Actually" Smart Summon and maybe even basic parking too.

*and by NOW I don't really mean NOW but when V11 is actually released.

 

[tmoz]

What is the status of Smart Summon? I don't see many videos of it passing or failing. Amazing and useful or not ready for prime time?

 

[VanFriscia]

What is the status of Smart Summon? I don't see many videos of it passing or failing. Amazing and useful or not ready for prime time?

It's glacially slow and cumbersome, and practically speaking not very useful at all. You have to be within a pretty close distance to your vehicle to use it. I HAVE used it a couple times to avoid getting wet in parking lots where nobody was around and that was kinda cool, but it's basically just a neat trick to show your friends. Maybe v11 will update this functionality too? FSDb has shown that the system is capable of doing quite a bit more, but without a human behind the wheel to stop the random stupidity...I don't know.

 

[JulienW]

What is the status of Smart Summon? I don't see many videos of it passing or failing. Amazing and useful or not ready for prime time?

Should be renamed "Embarrass yourself with Not a Smart Summon".

 

[Dewg]

What is the status of Smart Summon? I don't see many videos of it passing or failing. Amazing and useful or not ready for prime time?

The software code for Smart Summon was scrapped and is being re-written by Tesla. The previous code, which hasn't been updated in over a year, relied on ultrasonics and vision to handle the parking functions and summon functions, and still works on some vehicles with limited success. With the removal of ultrasonics from recent vehicles, the code has to be re-written to be pure vision. My guess is that we'll see basic parking functions (autopark) based on cameras only released first. Then we'll see Smart Summon once the validation of vision-autopark is finished. No timetable on the software release.

 

[JulienW]

....My guess is that we'll see basic parking functions (autopark) based on cameras only released first. Then we'll see Smart Summon once the validation of vision-autopark is finished. No timetable on the software release.

We probably won't get anything until "Single Stack" integrated (Actually) Smart Summon is released. Musk said "almost done" in Oct of last year so who knows. Hell at the rate V11 is (not) coming (Actually) Smart Summon may be ready for Stack integration by the time V11 is released.

The lack of USS Parking sensors (or Vision replacement) was an obtuse move by Tesla. The USS should have NEVER been removed UNTIL the Vision software was 100% ready, FULL STOP. Makes no sense selling newer cars month after month that is missing basic functionality that the same older model has.

 

[JulienW]

Not a big deal and not too applicable at this time (but what is ) but we now know V11 will be on a 23 branch since 23.2.0.5 has been released. So goodbye to 22.x.x.x when V11 is released. Although we are likely to get a 69 update or 2 or....on a 23 branch before V11 is released.

 

[Mardak]

Elon Musk on the 2022Q4 earnings call said he was with the Autopilot/AI team "late last night" (Tuesday) and asked who was close to a general solution for self-driving, but presumably they reviewed FSD Beta 11 / single stack for wider release. But there wasn't very much said about FSD Beta single stack handling highways throughout the call, so maybe it's still not ready for release this week?

There was a very brief mention of highways during the opening remarks:

With respect to Autopilot, as of now we deployed Full Self-Driving Beta for city streets to roughly 400,000 customers in North America. This is a huge milestone for autonomy as FSD Beta is the only way any consumer can actually test the latest AI powered autonomy. And we're currently at about 100 million miles of FSD outside of highways. Our published data shows that improvement in safety statistics is very clear. We would not have released FSD Beta if these safety statistics were not excellent.​

​

I suppose potentially because they have not released FSD Beta for highways, their safety statistics were not as excellent.

 

[swedge]

I suppose potentially because they have not released FSD Beta for highways, their safety statistics were not as excellent.

How could Telsa know the safety statistics for FSD Beta for highways (I'll call it FSDB4H) without having a large number of miles driven with that feature engaged?

Per Tesla's Vehicle Safety Report for 2022 Q3, 400,000 cars with beta have accumulated around 100 million miles. On average that is only 250 miles per car.

Per Tesla's Vehicle Safety Report for 2022 Q3, AutoPilot crashed once in 6.26 million miles, or 16 times in 100 million. I am no statistics wiz, but it seems like it'll take several hundred million miles of experience to measure any change in the safety.

I suspect they are fine tuning the way it drives so it does not "regress" too much, while also testing (in simulation) against all sorts of danger cases. I'm guessing they'll roll it out to Beta folks first to start accumulating miles and feedback. It could be a as a dual stack release with a setting to revert it back to standard AutoPilot on highways so they can see if drivers really like FDSB4H well enough to use it rather than the legacy version.

With 400,000 FSD Beta users, it won't take too long to accumulate enough miles to prove the safety. With employees only, it would take way too long...

 

[Mardak]

It could be a as a dual stack release with a setting to revert it back to standard AutoPilot on highways so they can see if drivers really like FDSB4H well enough to use it rather than the legacy version

Perhaps all these 10.69.x releases have been "dual stack" releases without the option to use the incoming single stack on highways. (It was noted that while city streets using FSD Beta needs to be able to fall back to standard Autopilot at any moment, highway usage only needs to rely on standard Autopilot and has spare compute cycles and time to load in FSD Beta neural networks or shadow mode networks.)

If the latest FSD Beta 11 neural networks are part of 10.69.25.2 running in shadow mode when you're on a highway and with the wide release to 400k vehicles, even a quarter of the population (like TeslaFi's FSD Beta population upgrading to this latest public version by Monday earlier this week) could mean 100k vehicles driving say an average of 30 miles per day with half on highways means 1.5M miles accumulated daily. More data would be better, but Tesla can probably get some early signal of whether it's even close or not.

Basically, the internal deployment of FSD Beta 11 to NDA employees can give some qualitative feedback while shadow mode can provide better quantitative data albeit with some limitations of not actually controlling the vehicle.

 

[Supcom]

Per Tesla's Vehicle Safety Report for 2022 Q3, 400,000 cars with beta have accumulated around 100 million miles. On average that is only 250 miles per car.

That is very misleading. Probably 300,000 of those cars have had beta for about 2 months, or less. The bulk of those 100 million miles should be on about 60,000 cars.

 

[Mardak]

The bulk of those 100 million miles should be on about 60,000 cars

The original estimate was indeed misleading, but the overall intent was probably that there's actually really low usage of FSD Beta.

I've estimated monthly miles from the cumulative FSD Beta miles Tesla has shared. There was about 60k vehicles with FSD Beta until about May 2022 for a cumulative miles of around 30M accumulating roughly 4M miles/month since January 2022 which comes out to ~70 miles per month on FSD Beta or just barely over 2 miles/day/vehicle. After expanding to 100k, the next few months accumulated about 7M miles/month which again comes out to ~70 miles/month/vehicle. And again to 160k with about 11M miles/month yet again averaging close to ~70 miles/month/vehicle.

If FSD Beta fleet of 60k kept accumulating the same miles for the 7 months after expansion to 100k and 160k, we would have reached a cumulative miles of over 50M indeed the majority of the miles so far.

Potentially that average miles per vehicle will be even lower with wide release as 100k+ vehicles were recently added including those who didn't even want to participate in limited early access FSD Beta. Then again, with FSD Beta 11, highway driving accumulates a lot of miles quickly.

 

[EVNow]

If FSD Beta fleet of 60k kept accumulating the same miles for the 7 months after expansion to 100k and 160k, we would have reached a cumulative miles of over 50M indeed the majority of the miles so far.

Potentially that average miles per vehicle will be even lower with wide release as 100k+ vehicles were recently added including those who didn't even want to participate in limited early access FSD Beta. Then again, with FSD Beta 11, highway driving accumulates a lot of miles quickly.

AFAIK there have been no reported accidents with FSD, right ? Is there anything in the reports Tesla has to send to NHTSA ... ?

 

[Mardak]

Is there anything in the reports Tesla has to send to NHTSA ... ?

Potentially yes, but we can't see them. Each accident reports "ADAS/ADS Version" as well as "Within ODD?" and "Narrative" but all of Tesla's entries are "[REDACTED, MAY CONTAIN CONFIDENTIAL BUSINESS INFORMATION]." For example, there's a likely ADAS entry for the San Francisco Bay Bridge crash with the driver saying he had FSD Beta, but there doesn't seem to be any noticeable difference in the NHTSA public data for that vs regular Autopilot.

Maybe with wide release and potentially increased usage on highways with FSD Beta 11 there should be an increase of accidents running on FSD Beta software although not necessarily active, and that could result in some detectable pattern in the reports.

 

[jabloomf1230]

How could Telsa know the safety statistics for FSD Beta for highways (I'll call it FSDB4H) without having a large number of miles driven with that feature engaged?

Per Tesla's Vehicle Safety Report for 2022 Q3, 400,000 cars with beta have accumulated around 100 million miles. On average that is only 250 miles per car.

Per Tesla's Vehicle Safety Report for 2022 Q3, AutoPilot crashed once in 6.26 million miles, or 16 times in 100 million. I am no statistics wiz, but it seems like it'll take several hundred million miles of experience to measure any change in the safety.

I suspect they are fine tuning the way it drives so it does not "regress" too much, while also testing (in simulation) against all sorts of danger cases. I'm guessing they'll roll it out to Beta folks first to start accumulating miles and feedback. It could be a as a dual stack release with a setting to revert it back to standard AutoPilot on highways so they can see if drivers really like FDSB4H well enough to use it rather than the legacy version.

With 400,000 FSD Beta users, it won't take too long to accumulate enough miles to prove the safety. With employees only, it would take way too long...

Are you sure that 400,000 NA cars have FSD beta installed? Tesla only claims they they have "made FSDb available" to 400,000 cars.

 

[jeremymc7]

Per Musk on Dec

“Enabled anyone in North America who has purchased FSD to request FSD Beta, reaching 285k cars total.”

I read this as there are 285k cars eligible to install beta (back in Dec) but I could also see how this oils be taken as 285k cars running beta. This would prob be around 400k by now.

 

[BritneySpears]

Does anyone actually use FSD beta after they've been given it and tested it out a few times? I stopped using it. In the maybe 10 instances I've used it it failed/freaked-out trying to do a U-turn and generally just moves too slow or brakes akwardly causing me anxiety and clearly confusing drivers behind me.

 

[powertoold]

Does anyone actually use FSD beta after they've been given it and tested it out a few times? I stopped using it. In the maybe 10 instances I've used it it failed/freaked-out trying to do a U-turn and generally just moves too slow or brakes akwardly causing me anxiety and clearly confusing drivers behind me.

I use 25.2 all the time, mostly in situations where I know it'll work decently. I actually use it when I want to relax. It's starting to become like how I use NoA / AP in general. When I first started using AP, it was some anxiety, but nowadays, I know when it's useful vs not as useful.

 

[Porkalletta]

I use 25.2 all the time, mostly in situations where I know it'll work decently. I actually use it when I want to relax. It's starting to become like how I use NoA / AP in general. When I first started using AP, it was some anxiety, but nowadays, I know when it's useful vs not as useful.

Same here. After you gain the confidence in its abilities you learn to relax but stay observant and ready to intervene. Just like airline pilots use autopilot. Not at the beginning or end of flights. For now anyway.