FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[legendsk]

I think we all agree - the ‘auto’ wiper function is FUBARed In The current release(s).

But remember, the volume control got fixed to vertical. Got to get the important stuff first.

 

[jebinc]

The release notes were changed to reflect the new version numbers.

And the amount of junk in the bag seems to have increased, exponentially. See “Black Tesla” YT vid posted earlier.

 

[fasteddie7]

I’m also not convinced that the wipers not wiping in the rain is not a bug but a sneaky way of retrieving tons of footage to train dojo on rain. Why hire people when customers pay you to provide footage.

 

[WilliamG]

Black Tesla,” - One of the most objective FSDj testers (re re re refreshed Model S, not 3/Y). He just released his 11.4.7.3 review. MORE and increasingly more JUNK!!!

View attachment 984030

Good gracious that’s awful. I just got this update this afternoon. Maybe it fixed the wipers. One can hope…

 

[D Good]

Could it be that the cameras are too good? I grabbed several dashcam clips while the wipers weren’t going and I could kind of see while driving, and barley see anything while in a parking lot, and to my surprise, the view of the dashcam did not represent what I could see through the windshield, you could barely tell it was raining. View attachment 984031View attachment 984032View attachment 984033

Yes. This is the problem. Ever watched a televised sports game being played in the rain. You literally cannot see the rain. It has to do with camera focus etc. I’ve known that it would be impossible to detect rain with cameras since day one because of this.

 

[jebinc]

Good gracious that’s awful. I just got this update this afternoon. Maybe it fixed the wipers. One can hope…

The new FSDj chant… “One step forward, and three steps back…”. Someone “gimme a beat”!

 

[WilliamG]

Yes. This is the problem. Ever watched a televised sports game being played in the rain. You literally cannot see the rain. It has to do with camera focus etc. I’ve known that it would be impossible to detect rain with cameras since day one because of this.

While this is true about sports, the wipers have kinda sorta been mostly okay up until the recent updates. Something has significantly changed.

 

[jebinc]

While this is true about sports, the wipers have kinda sorta been mostly okay up until the recent updates. Something has significantly changed.

The ingredients changed. There’s more “junk” now in the mix

 

[JulienW]

As far as being a "major game changer" 11.4.7.3 it is one of the tiniest updates at under 300MB that I remember. So it can't be much more that a couple of lipstick on bugs. With "all hands on V12" there is probably a single intern working on V11 in their spare time. I would say what a waste of bandwidth but it was more of a wast of just Install Time.

 

[old pilot]

Got 23.27.7 FSD 11.4.7.3 last night and installed. I would say it is a step backward, but a step backward would be and improvement. The earlier versions were much better. Driving on a clear night no rain just dark. Still has a fetish for the dedicated right turn lanes, more phantom slowing or braking. Something new, passing through an intersection at speed on a yellow light beyond the stop line the car decides not to go through the yellow light (going through was the correct move) and stops in the middle of the intersection, abruptly. Left unprotected turns a lot of indecision with the steering wheel moving rapidly left and right.

 

[Scott Chapman]

Since I got 11.4.7.2 when I check the car manually for updates, it says up to date as of <date current version was installed> instead of <current time/date>.

Is this how it works now?

 

[aronth5]

Since I got 11.4.7.2 when I check the car manually for updates, it says up to date as of <date current version was installed> instead of <current time/date>.

Is this how it works now?

This how my new Model Y has worked since I upgraded to HW4 in mid August. Couple times a week I refresh the screen and then the current date/time displays. Several times the screen refresh has also triggered an update including 1 map update. I am on 2023.32.9, FSD 11.4.4

 

[Scott Chapman]

This how my new Model Y has worked since I upgraded to HW4 in mid August. Couple times a week I refresh the screen and then the current date/time displays. Several times the screen refresh has also triggered an update including 1 map update. I am on 2023.32.9, FSD 11.4.4

BY refresh the screen you mean you display a different tab, and then go back to the software tab? I don't see any evidence that the content is "refreshed" per se

 

[aronth5]

BY refresh the screen you mean you display a different tab, and then go back to the software tab? I don't see any evidence that the content is "refreshed" per se

No, holding down both scroll buttons for a few seconds. The display goes dark and "reboots". Then I go to the software tab.

(edit- Just checked and 2023.32.9 does appear to be working correctly. However I didn't get 2023.32.9 until I did the screen refresh a couple of days ago)

 

[Wnuk]

Looks like they pulled the latest update

 

[zoomer0056]

View attachment 984092
Looks like they pulled the latest update

What was the "latest" version?

 

[jebinc]

Got 23.27.7 FSD 11.4.7.3 last night and installed. I would say it is a step backward, but a step backward would be and improvement. The earlier versions were much better. Driving on a clear night no rain just dark. Still has a fetish for the dedicated right turn lanes, more phantom slowing or braking. Something new, passing through an intersection at speed on a yellow light beyond the stop line the car decides not to go through the yellow light (going through was the correct move) and stops in the middle of the intersection, abruptly. Left unprotected turns a lot of indecision with the steering wheel moving rapidly left and right.

See post 12,686.

 

[Twiglett]

View attachment 984092
Looks like they pulled the latest update

that may just be your car, there are a whole bunch more showing waiting than there were yesterday

 

[Twiglett]

One thing I did get for the first time with 11.4.7.2
While on the highway with FSD enabled, it was doing pretty darned well but I wanted to be in the right lane because on this section its always faster there.
I signaled as usual but the car just kept driving along in its current lane with the indicator flashing, completely ignoring the request to move over.
It happened again a few miles further down the highway, no traffic to speak of in the right lane (they're all in the left lane for some reason).
Both times I had to take over to move the car over, but still left a message for the team to ignore.

 

[jabloomf1230]

Yes. This is the problem. Ever watched a televised sports game being played in the rain. You literally cannot see the rain. It has to do with camera focus etc. I’ve known that it would be impossible to detect rain with cameras since day one because of this.

I feel uncomfortable defending the Tesla rain sensing approach because the IR source-sensor pair works so well for sensing rain in other brands of cars.

However, the Tesla approach probably could be made to work better. As the windshield becomes covered with droplets of rain, the front camera's vision becomes blurred, unlike the quoted sports analogy above. The Tesla approach attempts to adjust wiper speed based by comparing changes between successive front camera video frames. That all sounds good on paper, but in practice it has worked poorly.