FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[MarcG]

When it finally achieves Smart ASS status, it will be time for the Tesla Shop to begin offering early-adopter ASS hats.

I'll wait for the SASS hats myself, more my jam.

 

[kabin]

The latest V11.3 video promotes the benefit of shifting within the lane when passing large vehicles. I could understand if the large vehicle wasn't centered in its lane but otherwise lane shifting causes more issues than it solves. I've noticed this on city streets and always prefer to disengage. I imagine motorcyclists in lane splitting/sharing states will have issues with this feature.

https://twitter.com/x/status/1628459969292156929

 

[LowlyOilBurner]

“Legacy” NoA moves over for trucks already on highways and has for the last year? This isn’t new.

 

[scottf200]

“Legacy” NoA moves over for trucks already on highways and has for the last year? This isn’t new.

We don't know all the subtle differences. ie. didn't this stay over when it saw to trucks in a row? vs ping pong in "legacy"
We don't know what happens when there are vehicles in the lane to our left either.

 

[EVNow]

Stage name WMC. I feel bad saying it as it probably gives a bad name to real drama queens.

If you think he is the "drama queen" you obviously don't watch too many youtubers

 

[thewishmaster]

“Legacy” NoA moves over for trucks already on highways and has for the last year? This isn’t new.

I feel like they either broke or removed this feature at some point because it stopped moving over a long time ago for me. Only FSDb. Though right now it's overly sensitive to parked cars and tries to change lanes away from them (claiming "obstruction") for absolutely no reason

 

[Mardak]

Seems like Tesla/Autopilot team has been busy trying to catch software up with ultrasonic sensor removal and now introduction of HW4 computer/cameras/sensors on new vehicles pending delivery while also getting FSD Beta 11.x single stack ready for wide release and the "usual" progress towards global deployment and production vehicle software updates. I suppose a management style of setting "arbitrary" deadlines like end-of-year holiday update and now risking revenue loss with unsellable inventory can be strong incentives for some engineers.

https://twitter.com/x/status/1628603576850460673

 

[EVNow]

Seems like Tesla/Autopilot team has been busy trying to catch software up with ultrasonic sensor removal and now introduction of HW4 computer/cameras/sensors on new vehicles pending delivery while also getting FSD Beta 11.x single stack ready for wide release and the "usual" progress towards global deployment and production vehicle software updates. I suppose a management style of setting "arbitrary" deadlines like end-of-year holiday update and now risking revenue loss with unsellable inventory can be strong incentives for some engineers.

https://twitter.com/x/status/1628603576850460673

Nope.

https://twitter.com/x/status/1628613857567711238

 

[jeewee3000]

Nope.

https://twitter.com/x/status/1628613857567711238

Indeed.

That Teslascope update is basically listing all the hopes and dreams of the community and portraying them as "coming soon"/"they're working hard on it" with some vague (most likely BS) explanation of why so and so has not happened by now.

Trying to save face after their HW4 fake leak, in a nutshell.

Not one specific prediction, so it's not even possible to properly fact check them in a month or two.

 

[emmz0r]

Indeed.

That Teslascope update is basically listing all the hopes and dreams of the community and portraying them as "coming soon"/"they're working hard on it" with some vague (most likely BS) explanation of why so and so has not happened by now.

Trying to save face after their HW4 fake leak, in a nutshell.

Not one specific prediction, so it's not even possible to properly fact check them in a month or two.

Wait was the HW4 fake?

 

[JulienW]

.....For anyone who thinks that it's not driving the way they want it, having a simpler "assist" would be nice for the parts that they do like - just as the current highway stack allows with lane changes......

Then you will then need to switch On and use the legacy AP Stack. It will likely still be available for a while. We will have to wait and see how you select Legacy AP for highways or FSD Beta for all roads but likely it is in the Menu since it doesn't seem to show a toggle switch while driving.

If you select FSD Beta for all roads then there is no distinction between highways or any other roads. All driving will be the same FSD Beta we have now.

 

[Knightshade]

Wait was the HW4 fake?

Greens computer and other info was accurate.

Teslascope specifically, saying a HW4 Model X had been delivered to a customer, was fake (or, not true anyway)

 

[jeewee3000]

Wait was the HW4 fake?

We still don't know for sure if HW4 is spotted in the wild.

But I was referring to their (definitely) fake HW4 leak on the white Model X:

https://twitter.com/x/status/1627587255085985794

 

[willow_hiller]

But I was referring to their (definitely) fake HW4 leak

I don't think "fake" is the right term. They confirmed that a Tesla employee told the vehicle's owner it was HW4. But, as often is the case, the Tesla employees know as little as we do.

 

[jeewee3000]

I don't think "fake" is the right term. They confirmed that a Tesla employee told the vehicle's owner it was HW4. But, as often is the case, the Tesla employees know as little as we do.

That's what Teslascope is saying, but we don't know if that's true.
This is what Teslascope is saying (not for you willow_hiller, you read this, but for others reading this post):

As for our now-confirmed erroneous post regarding an HW4 Model X Plaid delivery, we now know why this confusion occurred. We asked the individual told by their delivery advisor that their vehicle had HW4 to reach out and ask for clarification. Via a phone call, the delivery advisor apologized to the customer and explained that they misunderstood information presented within the company that "HW4 now in production" meant "my customer's vehicle has this" when inquired upon delivery day. This was ultimately not a significant deciding factor, so the vehicle owner is still pleased with their beautiful new vehicle. However, this is ultimately what led to the misinformation presented in our tweet, and we will work much harder to validate future information shared by vehicle owners and, when possible, to wait to confirm it with our employee sources, even if that means waiting a few days in some instances. We will be limiting replies/interactions to that tweet after midnight.

All I read is "we made assumptions without fact-checking" = fake news. Even if it's true what they are saying now, the way they presented the HW4 Model X leak ("first confirmed glimpse") says enough about their trustworthiness.

 

[kabin]

“Legacy” NoA moves over for trucks already on highways and has for the last year? This isn’t new.

With limited videos it's hard to say but V11's lane shift looks pretty extreme. At the least, a feature as silly as this should have a customer enable/disable switch.

 

[kabin]

Nope.

https://twitter.com/x/status/1628613857567711238

That link is dead for me.

 

[kabin]

I feel like they either broke or removed this feature at some point because it stopped moving over a long time ago for me. Only FSDb. Though right now it's overly sensitive to parked cars and tries to change lanes away from them (claiming "obstruction") for absolutely no reason

For me it's most noticeable and aggravating when driving through intersections with a truck in the left turn lane. The speed differential with needless swerving is an awful combination.

 

[JB47394]

With limited videos it's hard to say but V11's lane shift looks pretty extreme. At the least, a feature as silly as this should have a customer enable/disable switch.

I agree. This is the point I was trying to make about "assists". If Tesla wants to provide an assist for "shifting within a lane away from large vehicles", that's fine. But don't assume that I want that assist. This particular assist may be grounded in some kind of technical restriction where the cameras do a better job of understanding their environment if they're at a greater distance from a large object like a semi. I'm sure we've all noticed how semis in the current FSD tend to jump around in the visualization when close by.

I think the shift shown in the video is less safe than staying in the center of the lane. Those who think that would be safer could turn on the assist. In the end, we may be subject to whatever driving statistics are collected, and only find out by experimentation whether we correctly understand the statistics and then use them appropriately.

If anyone doesn't like the term "assist", then replace it with "feature" or "behavior" or whatever else seems appropriate.

 

[kabin]

I agree. This is the point I was trying to make about "assists". If Tesla wants to provide an assist for "shifting within a lane away from large vehicles", that's fine. But don't assume that I want that assist. This particular assist may be grounded in some kind of technical restriction where the cameras do a better job of understanding their environment if they're at a greater distance from a large object like a semi. I'm sure we've all noticed how semis in the current FSD tend to jump around in the visualization when close by.

I think the shift shown in the video is less safe than staying in the center of the lane. Those who think that would be safer could turn on the assist. In the end, we may be subject to whatever driving statistics are collected, and only find out by experimentation whether we correctly understand the statistics and then use them appropriately.

If anyone doesn't like the term "assist", then replace it with "feature" or "behavior" or whatever else seems appropriate.

Yep, and any bike rider knows debris collects near the lane lines and oil/debris collects in the lane center so not good places to frequently drive and especially so when FSD doesn't perform well with debris detection and avoidance.