FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Twiglett]

I take this as a hint from Whole Mars Catalog that we won't see a 10.9.x release which isn't a surprise. He is also being careful to say "might be" which may be to protect his source. His track record is very good

View attachment 762732

https://twitter.com/search?q=next big push might be&src=typeahead_click

Feels more like being first is more important than being accurate for FMC, as the next day he posts the 10.10 release notes

 

[planetary]

Feels more like being first is more important than being accurate for FMC, as the next day he posts the 10.10 release notes

Yeah. "His track record is very good" -- Ha.

 

[lakefiller]

RCS is not universal or standardized and if you’re worried about security (moderator edit). Secure messaging applications like Telegram have been available for years.

(moderator comment: evidence that it is becoming a universal standard here. A counter argument on RCS here.

This is false. Google Messages is now shipping with almost all android phones in the United States and other countries around the world and it has cross carriers RCS messaging out of the box. It follows the universal profile standard with the exception of adding signals E2E protocol. Even Samsung is ending Samsung messages in the US for Googles RCS implementation. Also, telegram does not encrypt messages by default like Google Messages and doesn't come with the phone as the default messaging app. You have to do.secret chats on telegram which loses some core features.

 

[Mardak]

Some of the cost functions can be explicit cost functions like collisions, comfort, traversal time…​

Looks like potentially another FSD Beta 11 change is being tested on 10.x with the first item of FSD Beta 10.10 notes: "Smoother fork maneuvers and turn-lane selection using high fidelity trajectory primitives." From using FSD Beta, it seems like the predicted path is closer to a neural network output sometimes very unstable vs what was shown at AI Day for path optimization, and I've definitely noticed it unnecessarily swerving multiple lanes to get to the inner lane for double/triple left turns.

Hopefully this means FSD Beta 11 will have the optimized trajectories for more than "just" forks and getting into turn lanes.

 

[JulienW]

Sounds like Tesla has been discussing the rolling stop thing in aggressive with NHTSA since about Jan 20, so one theory is they stopped pushing new FSD versions around then because they knew they'd need to push one that removed that.

Yea, looks like now we know and also explains whyTesla/Musk have been silent on why 10.9 was halted. Hind sight is 20/20.

 

[JulienW]

Just to get a little back on topic. Now we know that we will get 10.10 and 11.0 will not happen until at least mid or late Feb. The question is now will we get 11.11 and 11.0 (single stack) pushed back until March or Spring?

Elon the clock is ticking and fall will be here before we know it. So Level 3 (12.0) in early summer and Level 4 (14.0) in December?

 

[dramsey]

Yea, looks like now we know and also explains whyTesla/Musk have been silent on why 10.9 was halted. Hind sight is 20/20.

Well, it explains why FSD rollouts stopped. It doesn’t explain Tesla’s/Musk’s silence.

But now that we know 10.10 is in the field, I’m sure I’ll be enrolled in the program any day now.

 

[impastu]

Well, it explains why FSD rollouts stopped. It doesn’t explain Tesla’s/Musk’s silence.

But now that we know 10.10 is in the field, I’m sure I’ll be enrolled in the program any day now.

in about "two weeks"

 

[sleepydoc]

From using FSD Beta, it seems like the predicted path is closer to a neural network output sometimes very unstable

That may also explain the behavior I’ve experienced where FSD will veer into some turn lanes when it’s supposed to go straight but then handle the same turn lane without issue the next time I drive that route.

 

[dramsey]

A dissenting opinion on whether rolling stops should have been removed:

Feds Make Tesla Remove ‘Rolling Stops.’ It’s A Terrible Decision

NHTSA is forcing Tesla to do a "recall" to turn off the ability to do rolling stops in their FSD prototype. Is this obvious safety or a big overreach? Probably more of the latter.

www.forbes.com

 

[EVNow]

Looks like potentially another FSD Beta 11 change is being tested on 10.x with the first item of FSD Beta 10.10 notes: "Smoother fork maneuvers and turn-lane selection using high fidelity trajectory primitives." From using FSD Beta, it seems like the predicted path is closer to a neural network output sometimes very unstable vs what was shown at AI Day for path optimization, and I've definitely noticed it unnecessarily swerving multiple lanes to get to the inner lane for double/triple left turns.

View attachment 763793

Hopefully this means FSD Beta 11 will have the optimized trajectories for more than "just" forks and getting into turn lanes.

I hope 11 will be better than "primitive"

Not sure what they mean by "high fidelity trajectory primitives". A web search doesn't bring up anything useful ...

 

[momo3605]

I hope 11 will be better than "primitive"

Not sure what they mean by "high fidelity trajectory primitives". A web search doesn't bring up anything useful ...

It's not that definition of primitive. You can think of it more as a pattern.

 

[Dan D.]

I hope 11 will be better than "primitive"

Not sure what they mean by "high fidelity trajectory primitives". A web search doesn't bring up anything useful ...

Here are two concepts that you try linking together to make sense of that phrase. I'm just guessing here.

Googled "high fidelity" WRT engineering

Space mapping - Wikipedia

en.m.wikipedia.org

Space mapping methodology for modeling and design optimization of engineering systems... uses relevant existing knowledge to speed up model generation and design optimization of a system.

The space mapping methodology employs a "quasi-global" formulation that intelligently links companion "coarse" (ideal or low-fidelity) and "fine" (practical or high-fidelity) models of different complexities


Googled "trajectory primatives" and picked one:
Kinematic Constraints and Motion Primitives

Kinematic Constraints and Motion Primitives | Search-Based Planning Lab

sbpl.net

Motion primitives allow us to encode the kinematic constraints of the robot into any environment that the robot is planning in. They also allow for different costs associated with each motion - for example, if we want to avoid reversing the car, we might assign a high cost to that particular motion primitive.

 

[EVNow]

Here are two concepts that you try linking together to make sense of that phrase. I'm just guessing here.

Each of those words / phrases makes sense - but not sure what it all means in the context of smoother fork-behavior and turn lane selection.

Is this a new methodology they are applying for these cases (not used elsewhere) or what they are using elsewhere ("cost optimization") is being applied now to these two cases which they didn't apply earlier ?

Basically we know have two methods to figure out the trajectory.
- A cost optimization hybrid method they use now
- NN based Montecarlo simulations they are working on

Is "high fidelity trajectory primitives" the name they have given to one of the above or is it entirely different ?

Also, is this for the trajectory of the car or for other vehicles on the road ?

 

[lakefiller]

Just to get a little back on topic. Now we know that we will get 10.10 and 11.0 will not happen until at least mid or late Feb. The question is now will we get 11.11 and 11.0 (single stack) pushed back until March or Spring?

Elon the clock is ticking and fall will be here before we know it. So Level 3 (12.0) in early summer and Level 4 (14.0) in December?

Musk said it will be better than human drivers this year so he will probably change the narrative and say human drivers are worse than level 3. Lol

 

[EVNow]

Musk said it will be better than human drivers this year so he will probably change the narrative and say human drivers are worse than level 3. Lol

Level 3 and Human driving are entirely different things.

Levels are stupid SAE constructs. Elon doesn't talk about those. We shouldn't too.

I think if the disengagement rate is better than 1 in 10k miles, FSD Beta would be better than humans in city driving.

Considering the disengagement rate is probably < 10 miles now, the chances of that getting 1000x better this year are close to zero.

 

[Daniel in SD]

I think is the disengagement rate is better than 1 in 10k miles, FSD Beta would be better than humans in city driving.

What if you have to disengage every 11k miles to avoid hitting a pedestrian?

 

[EVNow]

What if you have to disengage every 11k miles to avoid hitting a pedestrian?

Then, it would be worse. But the chances of that actually happening are low i.e. FSD able to avoid even minor problems for 10k miles, but hits a pedestrian every 11k miles.

But the good thing is, when the disengagement rates for Tesla fall that much, they will be able to analyze all the disengagements to figure out crash rates for all levels - L1 through L4. They will know what the likely serious accident rates would be as well. Welcome to stats with large fleets

 

[Daniel in SD]

Then, it would be worse. But the chances of that actually happening are low i.e. FSD able to avoid even minor problems for 10k miles, but hits a pedestrian every 11k miles.

But the good thing is, when the disengagement rates for Tesla fall that much, they will be able to analyze all the disengagements to figure out crash rates for all levels - L1 through L4. They will know what the likely serious accident rates would be as well. Welcome to stats with large fleets


View attachment 764017

Exactly, you have to analyze the disengagements.

The chances have to be very low to exceed human performance.
2 billion miles per year travelled in SF
30 pedestrian deaths = 1 per 67 million miles = 1 out every 6,700 disengagements if your disengagement rate is 1 per 10k miles!
600 pedestrian severe injuries = 1 per 3.3 million miles

 

[EVNow]

Exactly, you have to analyze the disengagements.

The chances have to be very low to exceed human performance.
2 billion miles per year travelled in SF
30 pedestrian deaths = 1 per 67 million miles = 1 out every 6,700 disengagements if your disengagement rate is 1 per 10k miles!
600 pedestrian severe injuries = 1 per 3.3 million miles

Why are you only taking SF anyway - Tesla is not geohacked. We should use the above numbers.

Rule of thumb is 10x the rate i.e. for serious crashes better than human rate is less than 10 per 3 million miles.

60k testers probably clock 3 million miles every 10 days.