FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Twiglett]

It's been pretty quiet on the '11.4.7 going wide in a week release.' (moderator edit) The way 11.4.x releases have been going maybe it's another dud not worth seeing the light of day.

its interesting that it hasn't even made it out to the YouTubers yet, so I'm guessing its another redo of 11.4.5

 

[rlsd]

its interesting that it hasn't even made it out to the YouTubers yet, so I'm guessing its another redo of 11.4.5

YouTubers will no longer have higher priority than TMC members. If Tesla engineers look at TMC members' input here then they will fix the bugs faster.

 

[Twiglett]

YouTubers will no longer have higher priority than TMC members. If Tesla engineers look at TMC members' input here then they will fix the bugs faster.

Haven't seen evidence of that

 

[jebinc]

Was told to choose between his duties as an air traffic controller and a Tesla blogger?

Seriously, what happened to him?

 

[kabin]

Seriously, what happened to him?

Ramphex did a cross country FSD test to his house?

 

[kabin]

its interesting that it hasn't even made it out to the YouTubers yet, so I'm guessing its another redo of 11.4.5

It's funny how there's never a peep about those missed wide releases.

 

[jebinc]

It's funny how there's never a peep about those missed wide releases.

Or the “mind-blowing” ones, either!

 

[JulienW]

V12 question: What did everyone think about the X live stream drive that Elon made on v12 to Zuck's house for a fight (nasty fight but let's leave that out)? From what I saw v12 is really going to be "mind blowing" and looks to be robotaxi ready right now. Seem to remember that during the v12 live stream drive that Elon said that v12 would be in "wide release in two weeks"? So I think it is all be certain from what we saw that we will have roboxixes before "the end of the year".

Now after watching the impressive v12 drive live screen video I guess we need a v12 thread since we know Elon is spot on with release timing.

 

[LowlyOilBurner]

V12 question: What did everyone think about the X live stream drive that Elon made on v12 to Zuck's house for a fight (nasty fight but let's leave that out)? From what I saw v12 is really going to be "mind blowing" and looks to be robotaxi ready right now. Seem to remember that during the v12 live stream drive that Elon said that v12 would be in "wide release in two weeks"? So I think it is all be certain from what we saw that we will have roboxixes before "the end of the year".

Now after watching the impressive v12 drive live screen video I guess we need a v12 thread since we know Elon is spot on with release timing.
View attachment 965760

Is there a link for this said v12 drive?

 

[JulienW]

Is there a link for this said v12 drive?

Yes, but you need a Neuralink™ connection to Elon's brain.

 

[LowlyOilBurner]

Yes, but you need a Neuralink™ connection to Elon's brain.

So, it’s all BS then?

 

[JulienW]

So, it’s all BS then?

No, here I am watching it.

 

[fasteddie7]

It's been pretty quiet on the '11.4.7 going wide in a week release.' If SOP the shills should already be driving it. The way 11.4.x releases have been going maybe it's another dud not worth seeing the light of day.

With all the effort being focused on the v12 complete rewrite, wonder if 11.4.x has any engineering time being dedicated to it other than critical bugfixes, if there’s even that going on, and most of the teams have been pulled to work on the rewrite instead. If they are going to toss almost all of the 11 code out, makes sense for development to slow to a crawl.

 

[kabin]

With all the effort being focused on the v12 complete rewrite, wonder if 11.4.x has any engineering time being dedicated to it other than critical bugfixes, if there’s even that going on, and most of the teams have been pulled to work on the rewrite instead. If they are going to toss almost all of the 11 code out, makes sense for development to slow to a crawl.

We've been told Tesla has enough engineers. On that topic Rob Mauer says Tesla sent a number of engineers to China to grease the skids for possible FSD rollout. This could get interesting. If recent history is any indicator, Chinese customers and residents may not be as willing to blow off issues as we are here in NA. EVs sales in China are becming more competitive and Tesla might need another revenue source like FSD.

 

[Madmolecule]

no, there was no livestream the doxing of Zucks home address with live FSD fail drive along. Elon is not fighting anyone, other the wrestling Lex in jeans. I am waiting for V14 with 1/2 stack and city streets and other really really cool stuff, V3 hardware is another story. V14 should have the new catquest 2001 FSD Odyssey, and fart mode premium connectivity. no V13, that would be bad luck

 

[lascavarian]

With all the effort being focused on the v12 complete rewrite, wonder if 11.4.x has any engineering time being dedicated to it other than critical bugfixes, if there’s even that going on, and most of the teams have been pulled to work on the rewrite instead. If they are going to toss almost all of the 11 code out, makes sense for development to slow to a crawl.

You may be on to something.

I will just say that for FSD to be a readily licensed product/service, it will need to be easily applied to a variety of vehicle hardware. Similarly Tesla vehicles will need to support both new 48v and legacy 12v systems. Motors will be evolving too.

I suspect v12 is as much about having a “product” that maps easily to most/any EV hardware as it is about any leap in ability.

I could easily see v12 being an observable step back in abilities while suddenly being much more adaptable to almost any hardware (includes Optimus).

Just like rockets need to reusable so FSD needs to be usable on any hardware from the metal up. YMMV

 

[willow_hiller]

I suspect v12 is as much about having a “product” that maps easily to most/any EV hardware as it is about any leap in ability.

I could easily see v12 being an observable step back in abilities while suddenly being much more adaptable to almost any hardware (includes Optimus).

Good points. I always assumed that the vision models in FSD Beta were relatively camera-placement agnostic, but the fact that Tesla cannot just downscale HW4 cameras and run FSD Beta means the system as it stands isn't that flexible.

 

[MP3Mike]

but the fact that Tesla cannot just downscale HW4 cameras and run FSD Beta means the system as it stands isn't that flexible.

Is it the different cameras that are a problem, or is it a different AI accelerator that is the problem?

 

[willow_hiller]

Is it the different cameras that are a problem, or is it a different AI accelerator that is the problem?

Do you mean the Neural Processing Units on the HW4 FSD chip? It's my understanding that the HW4 NPU has one additional core, but otherwise they're similar architectures.

HW3 NPU description from wikichip:

"Each cycle, 256 bytes of activation data and an additional 128 bytes of weight data is read from the SRAM into the MACs array where they are combined. Each NPU has a 96x96 multiply-accumulate array for a total of 9,216 MACs and 18,432 operations. For the FSD chip, Tesla uses an 8-bit by 8-bit integer multiply and a 32-bit integer addition."

HW4 NPU description from semianalysis:

"Each cycle, 256 bytes of activation data and 128 bytes of weight data are read from the SRAM to the Multiply Accumulate units (MACs). The MACs design is a grid, with each NPU core having a 96x96 grid for a total of 9,216 MACs and 18,432 operations per clock cycle."

 

[MP3Mike]

Do you mean the Neural Processing Units on the HW4 FSD chip? It's my understanding that the HW4 NPU has one additional core, but otherwise they're similar architectures.

Yes. Doesn't it seem weird that AP/NoA run just fine on HW4, with both the new cameras and HW4 FSD chip, but FSD beta doesn't? So obviously they can make them work, but it seems like they have chosen not to at this time. (Which isn't totally true, as there are some HW4 vehicles with FSDb.)