FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Wi11iam]

I’m curious what this means for cars without FSD? Will the basic autopilot gains the new stack as well or it will stuck with the old implementation?

 

[Nikon201068]

Latest from Elon

 

[tmoz]

The specificity of V11.3 gives me a tiny kernal of hope that this might actually be around the corner, but I'm keeping my expectations low:

https://twitter.com/x/status/1611787267340926976

"Many major improvements". Has he become more grounded in his assessments? Not "mind blowing" anymore, just "major improvements?" Maybe Elon is fatiguing from the numerous updates?

 

[Supcom]

"Many major improvements". Has he become more grounded in his assessments? Not "mind blowing" anymore, just "major improvements?" Maybe Elon is fatiguing from the numerous updates?

I don't doubt that there are many major changes in V11, hopefully they will (mostly) be improvements. If there weren't major changes, V11 would have come out a year ago.

I'm most interested in how Tesla will stage the rollout. If the intent is to roll this out to everyone in the next FSDb release, which appears to be the plan, then it will be interesting to see how many waves and versions of 11.3.x go out before it goes to everyone.

 

[swedge]

I don't doubt that there are many major changes in V11, hopefully they will (mostly) be improvements. If there weren't major changes, V11 would have come out a year ago.

I'm most interested in how Tesla will stage the rollout. If the intent is to roll this out to everyone in the next FSDb release, which appears to be the plan, then it will be interesting to see how many waves and versions of 11.3.x go out before it goes to everyone.

Hmmm.... Using the FSD code for highways would be a change to the standard Auto Pilot and Enhanced Autopilot functioning. As such, it should eventually, be included in the standard version.

But before changing what everyone has gotten used to, it would make sense to roll it out in a series of incremental steps to make sure it does not degrade the performance or confuse the drivers. It will need to behave the same, but better. i.e. "fix it but don't change it!"

What we've been testing and calling "FSD beta" is actually called "Autosteer on city streets" in the sales screens. FSD has long included stop signs and lights on city streets. So, it seems to me that once FSD beta was released to anyone who had bought it, the "beta" testing program essentially over. Hence the lack of the report button. But the high safety score population would seem a good group for early roll out of the merged FSD/EAP code. It would be nice if Tesla explains the process a bit. Or maybe, they can just run simulations to compare the merged code against the old code to insure it acts the same, only better? We'll see...

SW

 

[N00bMaster69]

Ok- I'm just unclear what you meant by "There isn't even any video, so it's not there at all." because as I pointed out, there are no non-employees with the software who could post such video.

Thus your claim the lack of the video means the features don't exist seems somewhat a non sequitur?

You realize no one except employees have v11 right?

 

[Knightshade]

You realize no one except employees have v11 right?

You realize that is literally what I said in the post you quote, right?

Relevant bit pulled out below:

there are no non-employees with the software

 

[JulienW]

You realize that is literally what I said in the post you quote, right?

Relevant bit pulled out below:

Yea I read that this morning and was confused. Knew from your activity that you understood that BUT you were actually saying it right there in the quote. I keep reading to see if it was something I didn't understand but all I could see was.

 

[Mardak]

I’m curious what this means for cars without FSD?

In the near term, FSD Beta will probably only be active when its Autopilot setting is toggled and that FSD Beta software has only gone to vehicles with FSD Capability purchased/subscribed. Eventually it seems natural for plain Autopilot lane keeping to switch over to FSD Beta's single stack, and this could also start with those with FSD Capability then go to vehicles without that software option if Tesla is keeping early deployment testing to the FSD Beta fleet.

Some have suggested that Tesla will make the switch over at the same time of FSD Beta 11 / single stack being available, but that seems to have a bit of technical risk of changing too many things at once.

 

[Mardak]

when/if he does it's always "2+" weeks out… There isn't even any video, so it's not there at all

The lack of video evidence doesn't mean something doesn't exist just like how we didn't even see a 11.1. It seems like he gives "2 weeks" predictions as it probably matches up with their development cadence, so if FSD Beta 11.3 is actively being worked on (vs planning on needing a 11.3 because 11.2 is known to be insufficient), the plan is to have it released by the end of the cycle if no critical issues are found.

 

[Mardak]

This is probably the first time Elon has referred to V11 as a major "improvement" rather than just enabling highway or architectural change.

Hmm… Indeed in the past he downplayed the importance of FSD Beta 11:

https://twitter.com/x/status/1547384267952586752

But given his recent comments in the Twitter spaces, I believe he's still referring to the architectural changes. "The thing about the V11 is there are a bunch of neural nets that are architecturally much better than 10.69." The practical user noticeable improvements will still probably be relatively minor when comparing to existing Navigate on Autopilot, but from a technical aspect there can be major improvements especially towards generalized real-world AI such as having neural networks work well more independent of the current velocity.

For example, traveling on highways can have high longitudinal speeds while making a U-turn can have high lateral speeds, so in some sense, maybe the existing 10.69 neural networks get "dizzy" / overloaded when too much changes either longitudinally or laterally.

Clearly, he thought single stack was "just" turning it on for highways, but what they found with the internal 11.0/11.1 versions was it wasn't so simple and required more fundamental changes than quick fixes adjusting behaviors for highways.

 

[JulienW]

Hmm… Indeed in the past he downplayed the importance of FSD Beta 11:....

He has pumped V11 before as bringing improvements. Here is over a year ago.

 

[willow_hiller]

I was thinking about the delay in V11...

Given that it only exists on employee vehicles at the moment, is it possible that the holdup is due to additional hardware being needed, rather than additional software work?

We know from Tesla's FCC filing that they're planning on marketing a product with high-definition radar in mid-January. That would line up with Elon's most recent "two weeks."

 

[JulienW]

I was thinking about the delay in V11...

Given that it only exists on employee vehicles at the moment, is it possible that the holdup is due to additional hardware being needed, rather than additional software work?

We know from Tesla's FCC filing that they're planning on marketing a product with high-definition radar in mid-January. That would line up with Elon's most recent "two weeks."

But that would NOT line up with "....wide release..."

 

[willow_hiller]

But that would NOT line up with "....wide release..."

If it comes with a hardware retrofit, I don't see why not.

They could be sending retrofit parts out to service centers as we speak.

At first I thought the mid-January FCC confidentiality extension was going to be for the $25k car, but Tesla's last press release said they're going to be discussing that on March 1. So the HD radar is for something else.

 

[JulienW]

If it comes with a hardware retrofit, I don't see why not....

Many reasons. I believe the current 3/Y don't have a wiring harness. So that is a no go. Also likely the current wiring harness won't work since the new radar is a multichannel digital unit. Which also means it would probably not work with HW3 without at least supporting chips. Plus the mounting points are likely to be different also.

 

[jebinc]

Many reasons. I believe the current 3/Y don't have a wiring harness. So that is a no go. Also likely the current wiring harness won't work since the new radar is a multichannel digital unit. Which also means it would probably not work with HW3 without at least supporting chips. Plus the mounting points are likely to be different also.

But Elon (aka, “the Charlatan”) said…

 

[JulienW]

But Elon (aka, “the Charlatan”) said…

Tesla never said cars would be continuously hardware upgraded. In fact we have reached a point were it is logistically and financially near impossible. Besides my previous post here is more impediments and they are numbers.

Tesla could easily have >300k cars in the US alone to upgrade (and more added every day).

If the upgrade cost Tesla $1K per car that is $300,000,000 right off the bottom line.

Tesla has about <300 Service Centers in the US. That means each SC would have to upgrade about 1,000 cars and add in 3 or 4 man hours and you have MANY months before it could be completed. Plus regular service would be negatively effected.

There is NO WAY we are getting any mass substantial hardware upgrade since it just can't be supported. Tesla has grown too far and that boat has sailed.

 

[jebinc]

Tesla never said cars would be continuously hardware upgraded. In fact we have reached a point were it is logistically and financially near impossible. Besides my previous post here is more impediments and they are numbers.

Tesla could easily have >300k cars in the US alone to upgrade (and more added every day).

If the upgrade cost Tesla $1K per car that is $300,000,000 right off the bottom line.

Tesla has about <300 Service Centers in the US. That means each SC would have to upgrade about 1,000 cars and add in 3 or 4 man hours and you have MANY months before it could be completed. Plus regular service would be negatively effected.

There is NO WAY we are getting any mass substantial hardware upgrade since it just can't be supported. Tesla has grown too far and that boat has sailed.

No, the Charlatan only said in 201x that all cars built at that time were fully capable for L5 FSD!

Of course, the Charlatan did what a Charlatan would do… slowly redfine FSD over time…. A “history revisionist,” so to say….
@WilliamG
@diplomat33

 

[Knightshade]

Tesla never said cars would be continuously hardware upgraded.

Except, of course, they did say they'd provide needed upgrades- and said so repeatedly, through several revisions of hardware now- as long as they were neesary to delivery the features promised.

Starting way back when HW2.5 became known.

Tesla has been working on a backup plan in case its self-driving promises fail

Meet ‘HW 2.5’

www.theverge.com

"we still expect to achieve full self-driving capability with safety more than twice as good as the average human driver without making any hardware changes to HW 2.0,” the spokesperson said. “If this does not turn out to be the case, which we think is highly unlikely, we will upgrade customers to the 2.5 computer at no cost.”

Then, when both of those turned out to be incapable of FSD, HW3 came out. And again Tesla made it crystal clear HW upgrades were free since they were needed to deliver what you paid for:

Anyone who purchased full self-driving will get FSD computer upgrade for free.

So we've been given no reason to doubt they'd continue to keep that back-to-HW2.5 days promise to upgrade at no cost anyone for whom the HW changes are needed to deliver what they were promised.

In fact we have reached a point were it is logistically and financially near impossible.

Upgrading the computer, including labor, would be less than $1000 cost to Tesla. For customers they'd otherwise need to refund significantly more than $1000 if they can't actually deliver FSD.

Even if they had to upgrade the cameras and radar as well it'd be less than even the 3k the older buyers paid.

FAILING to do so (and trying to avoid a full, with interest, refund) would destroy any future sales of FSD too since nobody would trust the company to deliver going forward.

Besides my previous post here is more impediments and they are numbers.

Tesla could easily have >300k cars in the US alone to upgrade (and more added every day).

If the upgrade cost Tesla $1K per car that is $300,000,000 right off the bottom line.

And if they had to refund anywhere from $3000 or more to each of them-- that's an even larger #.

They'd have to do one of those two.

Upgrades are significantly cheaper.

Tesla has about <300 Service Centers in the US. That means each SC would have to upgrade about 1,000 cars and add in 3 or 4 man hours and you have MANY months before it could be completed. Plus regular service would be negatively effected.

If it's just the computer it's about half the time you estimate. If it's all the cameras and radar sure could be 3-4. And sure it might indeed take months. HW3 upgrades certainly did and it was fine. This would be too.[/QUOTE]