FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[AlanSubie4Life]

I just received notification that 2023.44.30.7 is available for my car. Consolidation continues...

Will it diverge when FSDb v12 is made available?

I assume that 2023.44.30.5.1 had deleterious effects on your insurance driving score (sorry if I missed a scoring post from you). I got back to my car yesterday and with a couple drives today (first since a brief 10 minutes before Christmas), that version was awful. It's got the horrible oscillating slowdowns again (excessive slowing using friction brakes, followed by reduced slowing, followed by friction brake and regen), and it just hammered the brakes from 10mph or so when a truck in front turned right (maybe not signaling - don't remember - but of course it was obvious they were turning) into a parking lot which I was ALSO turning into (Google Maps) . I wish I had got video of that one. The friction brake display went about 3/4 of the way to the left and on the wet roads the tires slipped slightly. Never any danger of course, not even close to hitting the guy. But just an awful display, for that reason - there was no reason to do anything other than regen smoothly and make the right turn into the parking lot.

Also got the bizarre behavior at stop signs where it uses the brakes from 6mph to 2mph then doesn't use them after that. So weird. Why on earth are they using brakes above 2mph??? I can understand using them to bring the vehicle to a halt promptly, but it's not what they're doing.

It also seems to stop really far back from lead cars sometimes. It's weird. Way further back than the typical "see the bottom of the tires of the vehicle in front" rule of thumb.

Just a very big regression in terms of smoothness. No idea why they're continuing to iterate on this stuff. Maybe it's to keep the interns busy?

I sure hope v12 leads to an actual increase in smoothness, rather than the normal pronouncements from the people with social media presence that that's what it has, followed by no improvement at all, actually. My hopes are not high, though. Best to keep very low expectations.

Hopefully these iterations are easier in v12 - just throw it all into a black box and not have to tweak any physics equations and redo all the tweaking and tuning, I guess. I could see that being an advantage if they have an actual way to measure what is "good" and just train for that. But I'm not too confident in that either since Tesla demonstrably does not seem to care about regressions. It's not even clear they use a simulator to measure performance in the simplest possible non-real-world scenarios after settling on a given release.

We're hurtling towards v12 and it seems likely to me it will be an incredibly disappointing but entirely predictable substantial incremental improvement (I hope!). Just not going to get to where it needs to be, probably. Total speculation of course based on zero facts at all. It's just a feeling.

 

[Scott Chapman]

I had noticed that since I updated to .5.1 that my car was not updating the date of the last check; been showing Dec 24th since that update in spite of reboots and other tricks which usually works.
woke up this morning and my car told me an update was ready.
I think it's been a while since checking for an update has resulted in anything material happening.

 

[TK211X]

2023.44.30.8

 

[kabin]

2023.44.30.8

Not much more before this holiday release turns into a travesty of a mockery of a sham.

 

[Mardak]

Just a very big regression in terms of smoothness. No idea why they're continuing to iterate on this stuff.

Presumably the changes in 11.4.8 for the new efficient video module were part of the 12.x neural network architecture that they wanted earlier evaluation of their training pipeline and real world performance with minimal changes to 11.x control. Given your experiences of control not getting smoother, these cases probably weren't an issue for perception, e.g., stopping for traffic control or another vehicle hasn't been a problem related to perceiving those objects but how FSD Beta was programmed to control the vehicle.

With control completely changing with 12.x, it does make sense that Tesla would not bother iterating on code that will not be reused, but unfortunately for us, that's what we have until the next major release.

 

[jebinc]

Not much more before this holiday release turns into a travesty of a mockery of a sham.

 

[JulienW]

Not much more before this holiday release turns into a travesty of a mockery of a sham.

So we should have the final Holiday update by Sant Patrick's day. Guess it is a holiday too.

 

[jebinc]

So we should have the final Holiday update by Sant Patrick's day. Guess it is a holiday too.

Nah, we are talking about "Uncle Elon," here.... it needs to be on April 1st!

 

[JulienW]

Nah, we are talking about Elon here.... it needs to be on April 1st!

Well then that would have to be April 20th (420) then.

 

[jebinc]

Well then that would have to be April 20th (420) then.

 

[gtgarner]

I know this might have been covered in this thread, however I can't find it.

Question: What happens after 5 forced disengagements?


I have 2 forced violations and not thinking that I will get any more, however - are forced disengagements ever zeroed out?

Will I have my forced disengagements forever?

 

[AlanSubie4Life]

Will I have my forced disengagements forever?

Yes if you don’t get any more. Unless they decide to reset them.

Question: What happens after 5 forced disengagements?

You lose use for a week. Count is reset to zero.

At least that is what I have heard. I have never had one in nearly 2.5 years.

 

[gtgarner]

Yes if you don’t get any more. Unless they decide to reset them.


You lose use for a week.

At least that is what I have heard. I have never had one in nearly 2.5 years.

Thanks,

Once I lose use for a week, does my counter start back at zero?

 

[AlanSubie4Life]

Thanks,

Once I lose use for a week, does my counter start back at zero?

Sorry, yeah, I clarified that above. (Added it just after your reply!)

Really gone out on a limb here. I have no idea how it works.

 

[gtgarner]

Sorry, yeah, I clarified that above.

Really gone out on a limb here. I have no idea how it works.

Ahh...sorry, I didn't read it closely enough. Thanks

 

[aronth5]

This is the first time I've seen 11.4.9 do this, so it may be a fluke. I do notice that 11.4.9 tends to weave within lanes more than prior versions, unsure if this is to avoid adjacent cars or possible obstructions or potholes or what

I'd just like FSD to slow down for speed bumps. I have several huge speed bumps which FSD treats as launching pads. I disengage because of their size. They are well marked too.

 

[gtgarner]

I'd just like FSD to slow down for speed bumps. I have several huge speed bumps which FSD treats as launching pads. I disengage because of their size. They are well marked too.

I agree 100%.

Speed bumps will have to be solved before robo taxi can even be a possibility...IMHO.

I wonder if there can be a vision only based speedbump avoidance system.

Humans have a very nice speedbump vision-based avoidance system.

 

[Knightshade]

I've had FSD slow down for speed bumps quite often... occasionally there's an update that seems to kill the feature but probably at least 3/4 of the time it's there... it only slows down to like 17ish, but it for sure slows down

 

[willow_hiller]

I've had FSD slow down for speed bumps quite often... occasionally there's an update that seems to kill the feature but probably at least 3/4 of the time it's there... it only slows down to like 17ish, but it for sure slows down

In my experience, it slows 100% of the time for speed bumps with large chevrons painted on them. If the bump is unpainted, it's a toss up.

Also, unless they're able to update the map metadata very quickly, it must be at least partially vision-based. My neighborhood had new speed bumps paved a couple of weeks ago, and FSD Beta started slowing for them as soon as they were installed.

 

[MP3Mike]

Also got the bizarre behavior at stop signs where it uses the brakes from 6mph to 2mph then doesn't use them after that. So weird. Why on earth are they using brakes above 2mph??? I can understand using them to bring the vehicle to a halt promptly, but it's not what they're doing.

I don't see behavior like that at all. It almost exclusively uses regen, unless the battery is cold in which case it usually sort of rides just over the regen limit.

Stops are generally smooth, and very acceptable, unless it gets "spooked" by something like a pedestrian or crossing vehicle.

My standard 13 mile trip today was completed with no disengagements or interventions. It made one unnecessary lane change.

The acceleration "event" was triggered when someone turned off into a restaurant in front of me and it was wide open for about half a mile in front of me, it really kicked it to get up to speed. The two places it identified braking events were unremarkable. I didn't think it was excessive at all, there were a couple other spots that I would have expected it to get dinged at but it didn't.

Config:

• Model Y LR w/HW3 and USS
• FSDb 11.4.9
• blend friction in if regen is limited enabled
• MLC disabled
• driving in the "Average" setting
• set to 23% over the speed limit