FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[andrewFW]

Probably not the case for anyone that posts in the FSD beta thread on the tesla motors club dot com forums.

 

[sleepydoc]

If we're honest, we've all experienced FSD(wtf) at least once

oh, yes. I'm sure even Omar has (not that he would admit it!) The difference is how some people consistently fixate and perseverate on the flaws and do nothing else.

 

[jebinc]

If you do the new acronym will be fsdBJ.

That function, if implemented correctly, may be worth $15,000 - especially if, “on demand”!

 

[kabin]

oh, yes. I'm sure even Omar has (not that he would admit it!) The difference is how some people consistently fixate and perseverate on the flaws and do nothing else.

We must all be assimilated. Resistance is futile. Then we can all be in harmony complaining about the few who complain.

 

[Buckminster]

Apologies for the Ryder Cup ad - I will try to raise the par in future

 

[LowlyOilBurner]

120 downloads on TeslaFi so far today for 11.4.7.2, basically double since that of yesterday, still a pretty slow roll out.

 

[PACEMD]

But that is the problem since they wipe more the dryer it is. So moving the Death Vally would just lead to more dry wiping with more sand on the windshield. I just want the dry wiping to stop already and then I can move on to complaining about something else.

But still hurry up roll out 27.6 since I also want me some new features. Should come with a free set of wipers too.

Oh, yeah, you're right. I don't know, maybe adopt one of those homeless guys with the squeegees and the dirty buckets?

 

[swedge]

120 downloads on TeslaFi so far today for 11.4.7.2, basically double since that of yesterday, still a pretty slow roll out.

It looks pretty similar to the 1.4.7 rollout back in August. It started with roughly 100 on 8/24, 200 2 days later, 500 2 days after that, and then over 1,000. Downloads peaked at 1,600 per day a week after it began.

I am looking forward to Charge on Sunshine, a solar plus PowerWall plus car charging thing. In June I got our first NEM2 true-up. Unlike NEM1, charging the car during solar production instead of at night, reduces the "Non-Byapssable Charges" which were what I had to pay at the annual true-up. I've been monitoring the PW and adjusting the charge rate as solar production and house consumption fluctuate. The alternative which I'd prefer not need to do is to write a script to do it for me, but hopefully the new CoS feature will spare me that effort.

 

[swedge]

11.4.7 - Wipers under FSD. They really need to push out a fix, just let us have the option to turn off the wipers during FSD. They can take it away when FSD is Level 4. This is horrific with the wiping. Everyday you turn on FSD and the wipers come on for no reason. Even under manual driving they come on, an while in Park too, but at least you can turn them off. How was this release ever approved with this glaring malfunction. It seems to be getting worse by the day. Who is responsible for this, somebody needs to be held accountable.

On this wiper thing, some of us are not having the problem. No rain here to see if they work at all, but no dry wipes on 11.4.7. Others have reported the same.

This would seem to raise a question about this bug: What is the difference between cars with the problem and those without.

For the record, mine is a 2021 Model Y LR, 38,000 miles, 2023.7.30 / 11.4.7. Wipers on Auto, dry weather. The car lives in the garage.

I did get a low washer fluid notice the other day. I'll top it up and see if that matters.

 

[gforce2002]

It would be interesting to see the distribution of wiper problems based on latitude. This time of year the further north you are the Sun is dropping quite rapidly each day closer to the horizon and I know that there are a number of angles to the Sun that seems to trigger my wipers.

 

[ikester]

Wipers… I’m betting it has to do with unintended side effect of the cleanup of the video feeds. Wipers mistaking more video resolution or definition …. for raindrops. I bet it magically goes away with retraining.

 

[JulienW]

Wipers… I’m betting it has to do with unintended side effect of the cleanup of the video feeds. Wipers mistaking more video resolution or definition …. for raindrops. I bet it magically goes away with retraining.

That is post processing and for human consumption feature that is not related to the driving computer.

 

[Yippydog]

11.4.7 - Wipers under FSD. They really need to push out a fix, just let us have the option to turn off the wipers during FSD. They can take it away when FSD is Level 4. This is horrific with the wiping. Everyday you turn on FSD and the wipers come on for no reason. Even under manual driving they come on, an while in Park too, but at least you can turn them off. How was this release ever approved with this glaring malfunction. It seems to be getting worse by the day. Who is responsible for this, somebody needs to be held accountable.

I have had this(dry wipe) happen a few times. Then I notice that my dry windshield had water spots. So I just hit the windshield washer a few times. After the wipers cleared the residual fluid off, the issue stopped. This is not a theory or even a hypothesis. Just one observation I had which made me think, dirty or streaky windshield looks like rain or occlusion to video processing? Also noticed for me that the dry wiper phenomenon only has happened about 5-10% of the time for me. Also seemed to exacerbate in high sunlight/glare situations.

 

[Ramphex]

oh, yes. I'm sure even Omar has (not that he would admit it!) The difference is how some people consistently fixate and perseverate on the flaws and do nothing else.

Because talking about what’s right doesn’t really matter, what’s right is supposed to be right, it’s not a surprise.

You got a “doc” in your name, do you go to your patients and give them a list of everything that’s right with them? Likely not, because it also doesn’t matter lol.

 

[JHCCAZ]

I have an interesting data point to report, and some thoughts about what it means:

My own car is still on 2023.7.30 as of today, but my son bought a new Model Y about 3 weeks ago. Surprisingly, that car just received 2023.27.6, replacing the factory firmware 2023.26.100.

To me this was unexpected, because his new car does not have paid FSD, and he had not yet even requested the beta for his 3 month free-trial FSD (because said request button was greyed out).

It had been my impression that 2023.27.6 was only going to those of us who were previously stuck on 2023 7.30 = FSDb 11.4.7 - and that the newer "wide release" customers we're all on a different track of more normal releases, presently running FSDb 11.4.4.l as the latest "stable" FSDb.

Again, he does not have paid FSD nor FSD subscription on that car, only the free-trial FSD.

To me this indicates one of the following possibilities:

1. Tesla is starting to merge everyone into the same FSD beta track, and the distinction between the prior tester group and the more recent wide-release group will disappear.
2. Maybe they aren't going for a general merge yet, but they want to get more data from the newest HW4 vehicles for 11.4.7, and/or in anticipation of an important upcoming build (v12?), so his car got pushed into the more experimental track. Arguing against this point, they don't know whether his car will even be on FSDb after the trial period.
3. A variation of the #2 theory is that the new 2023.27.6 build is specifically equipped to run v12 in Shadow mode for comparison and video data mining, and they want to test out such configurations on some new HW4 cars before deploying it more widely. In this scenario, they can get useful comparison data whether he's running FSDb or just driving manually.
4. Or possibly it's just a mistake that his car received the tester-group upgrade instead of graduating to the even newer (2023.32.x) wide-release build.

i think the most interesting explanation (but maybe not the most likely one) is #3 above. They've already mentioned running v12 in Shadow mode, and it's entirely possible that this requires a code base that hasn't yet been deployed into the normal firmware track. If so, it means that we could all be participating as we get this new version - but very few of us, as prior testers, are running with the newest hardware.

If indeed the next phase of development involves big Shadow mode collection of fleet data, it begs the question of which comparison is most useful to Tesla:

v12 prototype against v11 FSD driving, or​

v12 prototype against manual human driving?​

 

[WilliamG]

I have an interesting data point to report, and some thoughts about what it means:

My own car is still on 2023.7.30 as of today, but my son bought a new Model Y about 3 weeks ago. Surprisingly, that car just received 2023.27.6, replacing the factory firmware 2023.26.100.

To me this was unexpected, because his new car does not have paid FSD, and he had not yet even requested the beta for his 3 month free-trial FSD (because said request button was greyed out).

It had been my impression that 2023.27.6 was only going to those of us who were previously stuck on 2023 7.30 = FSDb 11.4.7 - and that the newer "wide release" customers we're all on a different track of more normal releases, presently running FSDb 11.4.4.l as the latest "stable" FSDb.

Again, he does not have paid FSD nor FSD subscription on that car, only the free-trial FSD.

To me this indicates one of the following possibilities:

1. Tesla is starting to merge everyone into the same FSD beta track, and the distinction between the prior tester group and the more recent wide-release group will disappear.
2. Maybe they aren't going for a general merge yet, but they want to get more data from the newest HW4 vehicles for 11.4.7, and/or in anticipation of an important upcoming build (v12?), so his car got pushed into the more experimental track. Arguing against this point, they don't know whether his car will even be on FSDb after the trial period.
3. A variation of the #2 theory is that the new 2023.27.6 build is specifically equipped to run v12 in Shadow mode for comparison and video data mining, and they want to test out such configurations on some new HW4 cars before deploying it more widely. In this scenario, they can get useful comparison data whether he's running FSDb or just driving manually.
4. Or possibly it's just a mistake that his car received the tester-group upgrade instead of graduating to the even newer (2023.32.x) wide-release build.

i think the most interesting explanation (but maybe not the most likely one) is #3 above. They've already mentioned running v12 in Shadow mode, and it's entirely possible that this requires a code base that hasn't yet been deployed into the normal firmware track. If so, it means that we could all be participating as we get this new version - but very few of us, as prior testers, are running with the newest hardware.

If indeed the next phase of development involves big Shadow mode collection of fleet data, it begs the question of which comparison is most useful to Tesla:

v12 prototype against v11 FSD driving, or​

v12 prototype against manual human driving?​

You sure you don’t have the trial? What options are available in the Autopilot section?

 

[FSDtester#1]

Why are we not seeing a bunch here on the latest version?
We got some OG's on this thread. Going on 2 years for a bunch of us....

 

[JHCCAZ]

You sure you don’t have the trial? What options are available in the Autopilot section?

As I said, he has a 3 month trial of FSD on the new car. So it's not surprising he can access FSD, including beta, now that he's off of factory firmware. He can request FSD options because of the trial, and that's not the issue.

The unexpected point is that his car downloaded a version previously targeted to those in the prior "Request Access" FSD beta group. Until now, everyone else who has purchase a new car, or recently purchased or subscribed to FSD, have been on the 2023 "Wide Release" software builds. In earlier months, those builds contained the prior FSDb 10.69 x which notably wasn't compatible with some of newer HW4 cars. Then more recently, the wide release version was bumped up to 11 4.4, roughly around the time the longstanding tester group moved to 11.4.7. As a well-known side effect and source of complaints, the long-standing group was way behind the non-FSD feature improvements until the present "catch up" rollout of 2023.27.6. Until now, I had never heard of any 2023 cars being added into the prior testing group.

Barring a simple (unlikely) mistake, it means that something is changing regarding the prior segregation of these FSD beta user groups. I'm speculating as to whether it's a general merge, or a specific intentional exception to get certain needed data results from the newest cars.

 

[swedge]

I did get a low washer fluid notice the other day. I'll top it up and see if that matters.

I added wiper fluid and got a bit of light rain here in Oakland, CA. Wipers worked perfectly as far as I could tell.

This was only a short test, but consistent with all my 11.4.7 drives, no weird wiping.

So my question stands: what distinguishes the cars with wiper chaos from those without that particular chaos?

Re: latitude, I am at 37.7 N.

Maybe I should have added some blinker fluid too. ;-)

 

[archae86]

...

that car just received 2023.27.6, replacing the factory firmware 2023.26.100.

To me this was unexpected, because his new car does not have paid FSD, and he had not yet even requested the beta for his 3 month free-trial FSD (because said request button was greyed out).

It had been my impression that 2023.27.6 was only going to those of us who were previously stuck on 2023 7.30 = FSDb 11.4.7 - and that the newer "wide release" customers we're all on a different track of more normal releases, presently running FSDb 11.4.4.l as the latest "stable" FSDb.
...
but very few of us, as prior testers, are running with the newest hardware.

Relying on Teslafi reporting at the time I type, of 349 subscriber installations of 2023.27.6,
27 came from releases that look likely to be "factory builds"
10 from 2023.26.100
17 form 2023.26.100.1

This is way out of line with the historic pattern, which might suggest that some of these got elevated selection preference for development-related reasons.

Furthermore, out of 349 subscriber installations 31 are listed as AP4. By contrast, TeslaFi report that only 2 remaining 2023.7.30 installations are AP4, you of a total of 4,276.

So strong selection bias over random in favor of AP4 cars, and at the very least a removal of the historic barrier against direct leap from factory build to this version.

Personally, I more strongly suspect need for testing than opportunity for shadow development, but that is just an interpretation.