FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[uscbucsfan]

The re-engage bug is fixed in 11.4.1. Had some really good drives eith it today. Was much more assertive committing to turns.

 

[Bladerskb]

There are though.

Hell you can find more than 5 different answers to what an HD map "really" is right here in the FSD threads just in the last 6 months-- this gets debated often.

A quick search in that period found mm-accurate, cm-accurate, accurate lane-level, lidar-mapped, larger than 10kb/km, and any map with road views rather than just the roads themselves given as >5 different definitions of what an HD map "really" is.

Generally the definition seems to just be used to try and make a point for whichever side of the debate you're on... if you want whatever you're discussing to be an HD map, you find some aspect of it you think can go into a nebulous "HD maps" term and call it an HD map... if you want whatever you're discussing to NOT be an HD map you just no-true-scotsman whatever the specific thing is as not making it an HD map.

I'm not looking at what some random person is saying. I'm looking at what the HD map industry is saying. Its not up for debates. The only people that debate it are people who don't want to accept the facts. So they come up with their own ideas.

 

[Knightshade]

I'm not looking at what some random person is saying.

So you were fundamentally wrong when you insisted my writing that if you ask 5 random people you get 5 different answers wasn't true.

Thanks for clarifying!

 

[Bladerskb]

So you were fundamentally wrong when you insisted my writing that if you ask 5 random people you get 5 different answers wasn't true.

Thanks for clarifying!

Look. All I care about is that TomTom is doing with Tesla what they were doing with Hella Aglaia back in 2019.

Tesla uses these HD map in conjunction with live perception output from NN to fed into BEV / Lane Network.
You can't just feed random metadata, low-quality, in-accurate map information if you are looking for multiple 9s accuracy.
Because if your map is off-centered, you would compromise your live vision NN output when you try to merge the information to feed it into the BEV/Lane network.

 

[Rikster]

It seems like 11.4.1 is being deployed at a slow but steady rate, and definitely beyond just employees and You-Tubers. I see from TeslaFi that there have been a couple hundred cars receiving it so far (or queued to receive it). Hope that means it will go wide soon! I know we would normally expect to see an 11.4.2 version soon that would be for a wider audience, but 11.4.1 seems to be going out fairly widely already?

 

[uscbucsfan]

It seems like 11.4.1 is being deployed at a slow but steady rate, and definitely beyond just employees and You-Tubers. I see from TeslaFi that there have been a couple hundred cars receiving it so far (or queued to receive it). Hope that means it will go wide soon! I know we would normally expect to see an 11.4.2 version soon that would be for a wider audience, but 11.4.1 seems to be going out fairly widely already?

I got it yesterday on one car. I'm not in the early groups.

 

[Radio Kid]

Wow. A lot of posting here since I last posted a few weeks ago. Here's an update. I'm now on FSD Beta v11.3.6 (2022.45.15). The "left turn only lane fetish" has disappeared on the few places that I've been able to re-test so far. That's good, but it took them over a year of reports to get around to it.

(Left turn only lane fetish -> the nav system shows the correct directions to go straight through an upcoming intersection. The car is in the correct lane to go straight. At the intersection, the car shifts out of the correct lane and into the left turn only lane. The car then proceeds to go straight through the intersection, illegally, from this left turn only lane.)

 

[Knightshade]

Look. All I care about is that TomTom is doing with Tesla what they were doing with Hella Aglaia back in 2019

Then why did you reply to a comment that wasn't about that at all, and wrongly claim it was not accurate?

 

[EVNow]

@verygreen could you provide outputs of the latest hd maps kind of how you did with the initial AP maps?

Doesn't matter what TomTom calls it. I've seen $10 speakers advertised as "Hi-Fi".

If you don't know the difference between TomTom's maps and the HD maps Waymo uses - readup.

 

[sleepydoc]

There does not appear to be a standard definition of the term ‘HD maps’ beyond ‘a highly accurate map that provides more details than a traditional map.’ That definition is incredibly vague and leaves a lot open to interpretation. If one considers a ’traditional’ map to be the old paper maps then every digital map currently in use could be considered HD. Since there’s no official definition and Tesla says they partner with TomTom to build ’HD Maps’ one can say that Tesla is using HD maps. Whether that matches your definition or not is another question.

On TomTom’s website they have a HD Map product sheet that lists:

• lane level geometry
• lane level speed limits
• lane markings
• traffic lights
• lane borders and guardrails
• lane connectivity
• Complete on/off ramp coverage

 

[Bladerskb]

Then why did you reply to a comment that wasn't about that at all, and wrongly claim it was not accurate?

It's cool bro. When I said, "There are not 5 different answers". I wasn't saying that random forum posters wouldn't give differing answers. I was referring to the AV and HD Map industry at large. Basically I was saying there are not 5 different answers even though 5 people might give you 5 different answers.

 

[Bladerskb]

Doesn't matter what TomTom calls it. I've seen $10 speakers advertised as "Hi-Fi".

If you don't know the difference between TomTom's maps and the HD maps Waymo uses - readup.

Oh the typical, "If its not what Waymo is doing its not HD map" logic that Tesla fans use to absolve themselves from admitting that Tesla is using HD maps.

 

[Mardak]

It seems like 11.4.1 is being deployed at a slow but steady rate, and definitely beyond just employees and You-Tubers

There's 5 vehicles from the pre-Safety-Score group (+1 HW4) on TeslaFi that got the initial push of 11.4.1 on Thursday 11th then it expanded to a 1% rollout on Friday. There was another push to a total of ~5% rollout last night, so with over 400k vehicles on FSD Beta, that's still 20k vehicles -- nearly double the original 100-Safety-Score group from October 2021, so it is indeed going wider. Although the audience is probably much more diverse than the original group of super eager testers, and hopefully the rest of us will get 11.4.x soon -- including MCU1 which I believe only got 10.69.3.3 and generally still waiting for any 11.x.

 

[kabin]

This looked like a close call. The stop short probably qualifies as a stop. FSD's uncertainty is evident from the speed increasing and then slowing and then increasing while the semi accelerates to the intersection and ego's path.

https://twitter.com/x/status/1658134418911895553

 

[JHCCAZ]

There does not appear to be a standard definition of the term ‘HD maps’ beyond ‘a highly accurate map that provides more details than a traditional map.’ That definition is incredibly vague and leaves a lot open to interpretation. If one considers a ’traditional’ map to be the old paper maps then every digital map currently in use could be considered HD. Since there’s no official definition and Tesla says they partner with TomTom to build ’HD Maps’ one can say that Tesla is using HD maps. Whether that matches your definition or not is another question.

On TomTom’s website they have a HD Map product sheet that lists:

• lane level geometry
• lane level speed limits
• lane markings
• traffic lights
• lane borders and guardrails
• lane connectivity
• Complete on/off ramp coverage

I can't comment too confidently here because I think there are others who follow this much more closely. I do think the list you just gave from TomTom represents the kinds of things that Tesla is now aiming to download from its servers every time a route is calculated, and maybe further updateable along the way. And my impression is that these things are not at all exclusive to TomTom. By the way, the list doesn't mention evolving traffic conditions in real time.

Tesla could contract with Tom Tom or other providers for the data, but I would guess it's going to move in the direction of Tesla crowdsourcing its own database from its growing fleet. If Tesla doesn't want to write and manage the mapping code infrastructure themselves, I guess they might make a deal to use someone else's framework in return for contributing their near real time fleet data. I'm only speculating how they might approach this.

But I don't think this is the kind of centimeter level HD map that is used for the "virtual rails" robotaxi approach. And that's not any attempt to defend Tesla from accusations of getting ideas from Waymo et al, it's just an attempt at clarification regarding the terminology and the intent.

I'm going to ping @diplomat33 because he's reported about this on and off, including some posts a year or two back about Medium Density (or Medium Definition) maps.

 

[EVNow]

Oh the typical, "If its not what Waymo is doing its not HD map" logic that Tesla fans use to absolve themselves from admitting that Tesla is using HD maps.

Yes - $10 Altec-lansing "hi-fi" speaker is not the same as $100k Wilson speakers.

And stop trash talking about "Tesla fans" .... (uninvited) guests should not insult hosts.

 

[gtg465x]

Crud. I haven't been following too closely and didn't realize Tesla still has the request beta button disabled. I've been staying on 2023.6.11 for a while now, denying updates to 2023.12.5, and signed up for a month of FSD when I saw a bunch of people start getting 2023.7.5 the other day, but now I'm thinking I done goofed and wasted $99. I really want Tesla to make it easier to subscribe and get on Beta without having to jump through hoops, ignore updates every time you get in your car, and having no easy way of knowing whether your subscription money is going to be wasted.

 

[powertoold]

My 2 cents is that Tesla isn't using HD maps or feeding any cm-level road geometry maps into FSDb's visualizations at all.

I was being kinda facetious when I posed the question about whether Tesla is using HD maps.

I think the reason Tesla's road geometry predictions are "superhuman" is because they're trained on 100k's or 1m's of intersection videos. FSDb's NN has essentially seen all manner of road geometries, and it's able to make predictions based on the most minute details (like a street name sign from afar means there's a crossroad there).

 

[MP3Mike]

Since there’s no official definition and Tesla says they partner with TomTom to build ’HD Maps’ one can say that Tesla is using HD maps.

Where/when has Tesla said that they are partnering with TomTom to build HD maps? Was that way back when AP started and Tesla went down the road of HD maps being a big driver for FSD, for which they failed, and abandoned that solution? (Bladerskb's quote didn't appear to be from Tesla, and had no link to verify it is even real. I did some quick Internet searches and couldn't find that quote anywhere.)

 

[PR0FESSOR]

@all,

My MSP is HW4 and 11.4.1. I just came back from a 97 minute drive (round trip) with 18 total stops and turns. On 16/18 stops, regenerative braking was used exclusively while on two stops, regenerative braking was used at first, and then close to the stop, friction brakes were used. The trip did not require any interventions, however, speed limits are an issue. In my rural area, it seems that the roads default to 25 mph, no matter what the actual speed is. For example, there are no speed limit signs near the house on the road I live on, but the speed limit is 55 mph. When I exit my driveway, FSD thinks the speed limit is 25 mph.

Although I did not intervene on this trip, I had one freeway lane change from the low speed lane to the off-ramp lane that was too abrupt for my liking. The freeway is 70 mph, and when FSD transitioned to the cloverleaf exit ramp, the transition was very abrupt. This is roughly the third time since I have had 11.4.1 that I experienced a very abrupt transition. All in all, I am pleased with the performance so far, however, there is no way that I would trust FSD enough to not pay close attention to what it is doing.

Joe