FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[AlanSubie4Life]

TGC strikes again

I tire of this pejorative sobriquet for the greatest salesman of all time.

 

[Dennisis]

69.25 is scared to death of parked cars and moves over way to much in its lane freaking out adjacent cars. I am hoping this is fixed in V11, which I hope I get within the next year

Not certain I recall correctly but check out the Dirty Tesla video on post #3781 - did well with parked cars, which mirrors my experience. If not that one there are others linked on this thread.

 

[Dennisis]

I tire of this pejorative sobriquet for the greatest salesman of all time.

Wow had to google that one!

 

[jebinc]

I tire of this pejorative sobriquet for the greatest salesman of all time.

I just tire…

 

[AndrewZ]

I like how this thread has devolved (evolved?) into an existential debate over whether or not traffic laws are even necessary.

If 11.4 isn’t released soon, we’ll be debating the need for roads. After all, FSD would be much more efficient if it could drive directly to your destination and not have to follow all of those silly arbitrary paths to get there.

In all seriousness, it just took me 2.5 hours to get from Manhattan to JFK (a 10 mile trip!!) in an Uber - due to a combination of 50% psychotic and 50% idiotic drivers. The sooner we get these humans off the road the better.

The road system and traffic conventions are really a truly remarkable thing when you think about them and what they enable, Elon has on occasion comment how amazing it is that more accidents don’t happen.

But the rules create structure and predictability that make it all possible, and the rules are probably a big part of what make autonomous vehicles even theoretically possible: green light go, red light stop, turner yield to cars travelling straight, lane lines separating flow, etc etc. Without the structure, probably none of this would be possible and we see autonomous systems struggle more in gray areas and stuff like unmarked roads where structure is lacking.

 

[Ramphex]

I just tire…

I had to get a new tire… look at this crap

 

[swedge]

I was referring to automated enforcement that is external to the vehicle.

Understood.

I was joking about the alternative, automated cars enforcing the law.

But what if Elon's Optimi (what is the proper plural of Optimus?) get hired (purchased) to serve as RoboCops to arrest Tesla speeders? Kidding again. Sort of.

SW

 

[Mardak]

Anyone have a guess as to when 11.3.4 will be released?

Tesla could be working on 11.4. There's been multiple 10.x and 10.69.x releases that only went to 20%-30% of the FSD Beta fleet before moving on to the next "minor" version that has dedicated release notes as opposed to a "dot" release without explicit changes. For example, replacing more legacy neural networks with surround video ones is probably worthy of a 11.4 while adjusting some thresholds to try to avoid some potential safety issues could be 11.3.4.

Maybe the Autopilot team is trying to get back to their previous 2-week public release cycle? 11.3.1 was spotted on the 8th then 11.3.2 on 19th and 11.3.3 on 25th. It's somewhat curious that they felt 11.3.3 was urgent enough to push out quickly to those already on 11.3.2 but not pull the old update from those who had it ready to install. Maybe this indicates there's some regression with single stack that was made slightly less bad with a quick fix but still bad enough not to push to the whole fleet? So an actual regression fix probably requires additional training of neural networks as part of 11.4?

 

[azred]

Now that this ancient V11.X thread has reached 194 pages, is it too early to start the V12.X thread in eager anticipation of its arrival in 2025?

 

[JulienW]

Now that this ancient V11.X thread has reached 194 pages, is it too early to start the V12.X thread in eager anticipation of its arrival in 2025?

Just checked my App and.......well V11.x hasn't arrived.

 

[Mardak]

Anybody have ideas of what legacy neural networks would still need to be replaced? Presumably the main ones used for primary driving functionality have been already covered by 11.3's single stack, so perhaps secondary Autopilot functionality such as Autopark / Summon / Smart Summon related that would only be conditionally activated when using those features?

Tesla is still needing to deliver on these features for new vehicles without Ultrasonic Sensors. It's probably not a coincidence that FSD Beta 11.3.2 started deployment on 19th then 2023.6.9 on 23rd restored Park Assist where the former probably included Occupancy Network improvements with Shadow Mode to test out the latter. So 11.4 might expand single stack to parking lot functionality?

 

[JulienW]

Anybody have ideas of what legacy neural networks would still need to be replaced?....

I vote to replace all the wacky conspiracy loving and Tweeter needing neural networks in Elon's brain. This would be huge benefit for Tesla.

 

[willow_hiller]

Anybody have ideas of what legacy neural networks would still need to be replaced? Presumably the main ones used for primary driving functionality have been already covered by 11.3's single stack, so perhaps secondary Autopilot functionality such as Autopark / Summon / Smart Summon related that would only be conditionally activated when using those features?

Whatever network is responsible for placing orange cones and trash cans in the scene is definitely not running on the FSDb stack. Usually FSDb places a grey blob in the correct location for a cone, but then the cone models are placed according to the old AP stack (and are typically in the wrong spots).

 

[tivoboy]

I had to get a new tire… look at this crap

View attachment 922960

Front or rear, which side? I’ve had a P zero peel off the sidewall like that WAY prematurely, but not the SS.

How often do you check your tires, and I don’t just mean PSI

 

[JohnB007]

I had to get a new tire… look at this crap

View attachment 922960

How much mileage did you end up getting? (Sorry, sidetrack)

 

[rlsd]

Impressive drive by DirtyTesla. I wonder if the car did so well because of the overcast. Do shadows (especially long ones near sunrise and sunset) erode FSD's confidence?

It seems it's easier for FSD to navigate in Michigan: the roads are straight and flat, the intersection turn angle is 90 degrees, the car is not required to get into a short segment at the corner (sometimes shared with bike lane) for right turn or left turn. In my area the roads are up and down, have lots of curves, some intersections have 70 degrees for left or right turn and traffic is heavy, and speed limit is between 40 and 55 on non highway streets.

 

[nvx1977]

Understood.

I was joking about the alternative, automated cars enforcing the law.

But what if Elon's Optimi (what is the proper plural of Optimus?) get hired (purchased) to serve as RoboCops to arrest Tesla speeders? Kidding again. Sort of.

SW

Optimus is actually a latin adjective, so the ending will follow what word its describing (bot). Unfortunately, "bot" or "robot" were not concepts that existed during the time Latin was still a living language. However the Greeks at the time wrote about "self-acting, autonomous, mechanical beings" and the word they chose for that was automatos (singular) and automata (plural). That word eventually made it into the Latin vocabulary as "automatum" which by its ending implies a neuter gender (appropriate!).

So with that, the best way to translate "multiple optimus bots" into plural latin form would be "automata optima," which translates to "the best bots." Google Translate however thinks "the best puppets" is the best translation.

The word "robot" was invented in a Czech novel, and when translated to English, the word robota came about, defined as "forced labor." If you assume robota is a feminine Latin noun, then using the optimus adjective with it in plural would be "robotae optimae." But since Musk is already using the masculine form of optimus, we can assume he considers "bot" or "robot" masculine. After all, Optimus Prime was clearly a masculine Autobot. Then going back to the derived greek term and forcing a male gender, the proper plural use would be "automati optimi."

Or you can just say "Optimus bots" to not be weird.

I also nominate myself for most off-topic post in the thread.

 

[Dennisis]

It seems it's easier for FSD to navigate in Michigan: the roads are straight and flat, the intersection turn angle is 90 degrees, the car is not required to get into a short segment at the corner (sometimes shared with bike lane) for right turn or left turn. In my area the roads are up and down, have lots of curves, some intersections have 70 degrees for left or right turn and traffic is heavy.

So you’ve tried 11.3.3 in your neighborhood then?

 

[rlsd]

So you’ve tried 11.3.3 in your neighborhood then?

I don't have 11.3.3 yet. I hope to have 11.3.4 soon. I usually avoid unprotected left turns even when I drove ICE cars. I'd rather make u-turns or go around the blocks.

 

[jebinc]

I had to get a new tire… look at this crap

View attachment 922960

I’d rather not, as it’s too tiring.