FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[amadeusneo]

There's no evidence that FSD Beta can read any signs outside of speed and it's not likely reading it, just recognizing the pattern of the numbers trained in to their model.

The reason I say this is because almost every "No right of red" is different, so that would be extremely difficult to do. I think they were attempting to solve this with the car's reported map data and user interactions, but that's not a solution you could push out globally.

It's not impossible to program every single no right on red variant, including ones with specific times, but I would guess it's ridiculously low on the priority as it's not actual FSD and given that School Zones or specialty speed limits haven't been programmed in, nor bus stops, I think it's unlikely we see it for a while.

I agree about 11.4.7, too.

For what it's worth, the car used to not respect "no right on red" but has lately for me for a while, I don't remember which beta version added it. Now, this also could be a function of them streaming that information to the car instead of the car "seeing" the sign. Who knows. We don't get detailed change notes on stuff like this. Just AI technospeak.

 

[JulienW]

For what it's worth, the car used to not respect "no right on red" but has lately for me for a while, I don't remember which beta version added it. Now, this also could be a function of them streaming that information to the car instead of the car "seeing" the sign. Who knows. We don't get detailed change notes on stuff like this. Just AI technospeak.

Tesla is using map data. I have it also not turning at several rights that aunt allowed. There is even one that used to be no right on red but now has the sign removed and it still won't turn on red. Which it used to try when the sign was up.

Remember we get "invisible" Navigation data file Delta updates all the time.

 

[uscbucsfan]

Tesla is using map data. I have it also not turning at several rights that aunt allowed. There is even one that used to be no right on red but now has the sign removed and it still won't turn on red. Which it used to try when the sign was up.

Remember we get "invisible" Navigation data file Delta updates all the time.

We get map updates with every single drive...but as others have pointed out, not all may be accurate.

 

[JulienW]

I actually posted this dumb a$$ rant in another thread by accident so I will plagiarize my foolish babbling here.

Tesla needs to update all of us on 23.7.x/11.4.4 and 11.4.6 (it is only a bug update and nothing major from 11.4.4) to 23.26.10 and when 11.5.x is released give us a choice (maybe a switch) of moving to 11.5.x and keeping them on about the same software. Or at worse keep 11.5.x on 23.26.x while 11.4.4 moves to 23.30.x.

Now that would sure make me a happy camper (at least for a day or so ).

 

[AlanSubie4Life]

Even my insurance rated good:

11.4.4 has been notably better than prior versions in terms of jerk when stopping, so hopefully your 11.4.6 is even better, and future versions better still.

I still intervene a lot to avoid sharp slowdowns, but the ones where I permit the car to handle it itself are definitely improved from prior versions. Finally; this has only become evident over the last couple versions.

 

[Twiglett]

they're seriously going to rollout 11.4.4 to everyone with 11.3.6 - they are going to be sooo pleased
Maybe the upside is that 11.4.69 will be on a modern branch.

 

[MP3Mike]

So did FSDb get a root canal? I'll be curious if this helps it's attitude...........

Some counseling might also help.

Nope, it wasn't the one getting dental work done... (I wasn't either, I was just the safety driver for this trip.)

The trip back was ~8 miles and had one disengagement because it wasn't merging when a lane was blocked for construction as early as I wanted. (There was one other "intervention" where I forced a lane change to the left so that it would miss the right turn navigation wanted to take that I didn't want to take. I could have avoided this if I bothered to pick the correct alternate route when starting the trip.) There were a number of "extra" uses of the turn signal, but it didn't actually make any unnecessary lane changes. (Well, in one case it made a lane change to follow the route, but to behind someone going under the speed limit so it shortly made a lane change back to the "faster" lane. So, like where I think they added logic to no longer switch to faster lanes when getting close to a turn, they need to add some more logic to not change into a slower moving lane unless it is close to the next turn.)

And the insurance rating for this drive:

I don't recall a hard braking event, but it is shown at about the place I took over for the construction merge.

 

[JulienW]

they're seriously going to rollout 11.4.4 to everyone with 11.3.6 - they are going to be sooo pleased
Maybe the upside is that 11.4.69 will be on a modern branch.

All the new HW4 owners are pissed now waiting for 11.4.4.

 

[AlanSubie4Life]

I don't recall a hard braking event, but it is shown at about the place I took over for the construction merge.

Have you determined how many g it takes to get a single demerit on the braking/acceleration metrics? I would think it wouldn't care about anything below 0.2g but maybe it is more sensitive than that. 0.2g is pretty noticeable - though of course the perception greatly depends on the associated jerk.

 

[Twiglett]

All the new HW4 owners are pissed now waiting for 11.4.4.

What I don't understand is that they take 11.4.4 - which has bugs so serious that they can't seem to release its replacement, even after three attempts - but use that release to update all the 11.3.6 folks. Boggles the mind.

 

[MP3Mike]

Have you determined how many g it takes to get a single demerit on the braking/acceleration metrics? I would think it wouldn't care about anything below 0.2g but maybe it is more sensitive than that. 0.2g is pretty noticeable - though of course the perception greatly depends on the associated jerk.

No, I can't find anywhere that they publish the details behind the program. I'm sure there is probably some regulatory filing, but I haven't taken the time to look for it.

 

[aronth5]

All the new HW4 owners are pissed now waiting for 11.4.4.

FSD transfer to new Model Y. Had FSD 11.4.4. Picked up the car on 8/15 and tonight is the first opportunity to connect to WiFi. Interesting that when I select the software button it says my car is up to date as of 8/15. On my prior 2 Teslas it would always check for the software version when I selected Software and then typically would state the software was up to date as of "todays date".
Same version when I picked up the car. 2023.20.100.1. Anxious to see if anything happens over the next couple of days.

The new Model Y is significantly better than my last Model Y which was VIN 15xxx.

 

[amadeusneo]

FSD transfer to new Model Y. Had FSD 11.4.4. Picked up the car on 8/15 and tonight is the first opportunity to connect to WiFi. Interesting that when I select the software button it says my car is up to date as of 8/15. On my prior 2 Teslas it would always check for the software version when I selected Software and then typically would state the software was up to date as of "todays date".
Same version when I picked up the car. 2023.20.100.1. Anxious to see if anything happens over the next couple of days.

The new Model Y is significantly better than my last Model Y which was VIN 15xxx.

Software update is only allowed to check once every 24 hours now. Has been for a while. I have no idea why they added this restriction.

 

[uscbucsfan]

Software update is only allowed to check once every 24 hours now. Has been for a while. I have no idea why they added this restriction.

Because most/if not all updates are pushed, not pulled. The check for update queries if a push has been made, but mostly the car will get it when an update comes.

Having millions of cars query the server many times per day is likely an issue, especially when bug pushes go out.

 

[AlanSubie4Life]

I still intervene a lot to avoid sharp slowdowns

And today I did allow this coming into Sunset Ridge from Spring Canyon eastbound and sure enough it slams on the brakes. Totally unnecessary and normally I would intervene.

So still a long way to go with longitudinal control. One of the basics. If they nail that they may have a good driver assist. The first!

It’s all about the fundamentals, after all.

 

[aronth5]

Software update is only allowed to check once every 24 hours now. Has been for a while. I have no idea why they added this restriction.

Because most/if not all updates are pushed, not pulled. The check for update queries if a push has been made, but mostly the car will get it when an update comes.

Having millions of cars query the server many times per day is likely an issue, especially when bug pushes go out.

You both missed my point. The date displayed is 8 days old which is different behavior than my last 2 cars. I would have expected once I clicked on the software button to display today's date and confirm the software is up to date. I suspect the behavior is related to the FSD transfer.

 

[Supcom]

Because most/if not all updates are pushed, not pulled. The check for update queries if a push has been made, but mostly the car will get it when an update comes.

Having millions of cars query the server many times per day is likely an issue, especially when bug pushes go out.

Somehow, I find it hard to believe that simple update availability checks do not present much difficulty for a server farm that is accustomed to ingesting location updates every few seconds while those same millions of cars are in gear, push out gigabytes of data for every update and pull gigabytes of data after every FSDb drive.

 

[uscbucsfan]

Somehow, I find it hard to believe that simple update availability checks do not present much difficulty for a server farm that is accustomed to ingesting location updates every few seconds while those same millions of cars are in gear, push out gigabytes of data for every update and pull gigabytes of data after every FSDb drive.

I believe it was green who postulates it. It was changed during a big push, but there's likely a valid reason.

 

[PACEMD]

FSD transfer to new Model Y. Had FSD 11.4.4. Picked up the car on 8/15 and tonight is the first opportunity to connect to WiFi. Interesting that when I select the software button it says my car is up to date as of 8/15. On my prior 2 Teslas it would always check for the software version when I selected Software and then typically would state the software was up to date as of "todays date".
Same version when I picked up the car. 2023.20.100.1. Anxious to see if anything happens over the next couple of days.

So the new Model Y, with the FSD transfer from the old Model Y, had FSD with FSDb software installed when you picked it up? I'm picking up tomorrow.

 

[archae86]

You both missed my point. The date displayed is 8 days old which is different behavior than my last 2 cars. I would have expected once I clicked on the software button to display today's date and confirm the software is up to date. I suspect the behavior is related to the FSD transfer.

I've occasionally had one of our cars slip into that behavior, when the other did not, and there were no wide reports. Following a suggestion posted here, I tried a two-finger reset, and got out of that mode.