FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[jebinc]

Sure - but not as simple as you thought it was. You need that metadata first maintained for all the highways in US - which may not even be available with any 3rd party.

BTW, are you guys doing any testing this weekend?

I would be good to see if there is a difference between the X/S implementation, vs 3/Y.

 

[jebinc]

Do we have any videos of 11.4.1 on HW4?

Or running on a refreshed S or X?

 

[Knightshade]

I don't know about HW4, but I do know back when HW3 came out, for quite a long while it was just running the HW2.x code in emulation mode.... and there was a MUCH larger difference in general architecture between 2.x and 3 than there is 3 and 4, so it would be pretty trivial to get HW4 running the same HW3 code initially.

 

[kabin]

Um. I've been having fun reading the thread. Some of the YouTubers with 11.4.1 have HW3; some of them have HW4. So, that part of it appears settled.

What was the debate?

 

[Knightshade]

I think Tesla no longer needs to give priority to early testers like Chuck Cook, Whole Mars Catalog,... These people were helpful in the past. But now we are at a different milestone. Their repeated tests don't cover daily use. I think their tests are performed mostly in low traffic zones and light traffic. FSD should be tested in areas having high traffic and having speed limit at least 40 mph.

Maybe Tesla should fix those low-traffic low speed things it still can't reliably do before moving on to more advanced tasks?

Tesla should hire more employees to test FSD. The employees should take time to perform the tests. After testing by employees, FSD should be given to all FSD users.

They literally did this for Chucks left turn thing...sent them to his exact test location. For weeks. Months ago.

It still can't do it reliably today in 11.4.1

 

[kabin]

I don't know about HW4, but I do know back when HW3 came out, for quite a long while it was just running the HW2.x code in emulation mode.... and there was a MUCH larger difference in general architecture between 2.x and 3 than there is 3 and 4, so it would be pretty trivial to get HW4 running the same HW3 code initially.

For optimal performance HW4 might require HW4 specific training data especially with HW4 cameras being higher resolution, possibly different placements, etc.

 

[powertoold]

I do know back when HW3 came out, for quite a long while it was just running the HW2.x code in emulation mode....

Yes but if HW4 has indeed been released, Elon says it's not retrofitable and the resolution is much higher than HW3. The situation is different vs hw2 to HW3, where the difference was only increased CPU ability (Nvidia to fsd computer).

But you may be right, perhaps the higher res HW4 cameras are being downsized to ~1080p...

 

[archae86]

From TesliFi, I live in North Carolina, have a Model S Plaid with the 510xxx Vin. If I am reading the table below correctly, it appears that I am the only HW4 vehicle on this version of FSD?

Joe

That needs to add the caveat "among Teslafi Subscribers". Rather fewer than 10% of FSDb cars are TeslaFi subscribers, and it is not a random sample. I'd guess there most likely are a few more HW4 vehicles that got this version, but just are not TeslaFi subscribers yet.

 

[JHCCAZ]

My dog sits in the front seat - there’s no car seat setting for the front seat. (At least there wasn’t, I haven’t checked in a while)

Ah, Dog is your co-pilot. ;-)

My daughter's happy doggy in her M3, heading for the dog park

 

[SSDTester#1]

Has anyone been keeping track of how many versions and subversions Elon called ? I might switch from @jebinc numbering scheme to my own.

 

[fasteddie7]

Has anyone been keeping track of how many versions and subversions Elon called ? I might switch from @jebinc numbering scheme to my own.

Wonder if the software engineers know the exact routes Elon will use to test and give them a little extra love for job security.

 

[D Good]

From TesliFi, I live in North Carolina, have a Model S Plaid with the 510xxx Vin. If I am reading the table below correctly, it appears that I am the only HW4 vehicle on this version of FSD?

Joe

View attachment 937205

You are probably the first in a new group of the “HW4 Chuck Cooks” of the world. Essentially, they seem to be starting a new group that goes along with the OG YouTubers, only this time they need people with HW4 to test it on early.

Isn’t it also very strange that Tesla themselves have never once mentioned the fact that HW4 even exists? Strange indeed. Are we sure ourselves that what we are seeing now is HW4?

 

[LowlyOilBurner]

Has anyone been keeping track of how many versions and subversions Elon called ? I might switch from @jebinc numbering scheme to my own.

4.20, or .69, not sure.

 

[SSDTester#1]

4.20, or .69, not sure.

There’s definitely been at least 5

 

[kabin]

Maybe Tesla should fix those low-traffic low speed things it still can't reliably do before moving on to more advanced tasks?

They literally did this for Chucks left turn thing...sent them to his exact test location. For weeks. Months ago.

It still can't do it reliably today in 11.4.1

And the downside is seemingly never getting rid of Chuck's UPL overtraining - stop well short of stop signs, creep delay, turn right to turn left, herky-jerky accel/decel before entering intersection. I hope they have other tricks in the bag as these overtraining science projects are a double edge sword.

 

[Supcom]

Teslafi reporting 2023.7.5 added 23 more cars.

 

[JHCCAZ]

There’s definitely been at least 5

v12 will be Yuge

 

[archae86]

Teslafi reporting 2023.7.5 added 23 more cars.

Now it is up to 71 pending. Seeing the posts here, I found my model 3 had just finished downloading, so I started the install.

 

[clydeiii]

How so? I mean, the compute power on paper of HW4 isn’t really that much greater than HW3, so I get that. The cameras are definitely better than HW3, that’s a no-brainer.

The issue isn’t the resolution of the cameras. Coding self-driving AI is just super hard. No magic bullet with additional hardware…

 

[powertoold]

11.4.1 cuts it really close wrt pedestrians...

And also here where the pedestrian yells at 11.4.1: