FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[powertoold]

11.4.2 is the beginning of the end for Waymo and Cruise

 

[JHCCAZ]

11.4.2 is the beginning of the end for Waymo and Cruise

Simple, to the point, and completely non-controversial.
I mean, no one here can argue with that.
I suppose...

 

[aronth5]

So far HW4 is the same. It stops where it should for a stop sign, but most of the time that’s not where a human driver would stop based on obstructions, so what ends up happening is a jittery creep forward and an angry driver behind me.

The difference is without a camera further forward FSD cannot make the same good decision a human can. We simply see better by leaning forward. I'm still trying to figure out how Tesla is going to implement robotaxi when the vision on obstructed view intersections is clearly compromised. Nothing new just hoping H4 would address this but so far that doesn't appear to be the case but we'll have to wait and see. If you look at the picture I posted previously the B-pillar view is actually worse than the what the photo showed from the drivers seat. (improved angle)

 

[JHCCAZ]

The difference is without a camera further forward FSD cannot make the same good decision a human can. We simply see better by leaning forward. I'm still trying to figure out how Tesla is going to implement robotaxi when the vision on obstructed view intersections is clearly compromised. Nothing new just hoping H4 would address this but so far that doesn't appear to be the case but we'll have to wait and see. If you look at the picture I posted previously the B-pillar view is actually worse than the what the photo showed from the drivers seat. (improved angle)

Yes, the case for better camera placement has always been based on rather simple geometry POV considerations. I've tried to allow the possibility that I'm missing something, but I simply cannot find (what I consider) a credible argument against a more corner weighted placement. It would be a significant advantage in many common situations.

The most encouraging thing I heard recently, but which hasn't appeared yet, is the addition of a second forward-facing camera in each side repeater, overlapping and complementing the rear-coverage camera. Though not optimal, it's placed more forward than the driver's head. And for left turns, it affords the possibility of seeing better past the view blockage of cars in the facing left-turn lane or bay.

As far as I can tell, the latest reports of "HW4" repeater cameras, now appearing on model Y, do not have the two-camera design. It looks like we'll have to continue to wait.

 

[heapmaster]

Yes, the case for better camera placement has always been based on rather simple geometry POV considerations. I've tried to allow the possibility that I'm missing something, but I simply cannot find (what I consider) a credible argument against a more corner weighted placement. It would be a significant advantage in many common situations.

The most encouraging thing I heard recently, but which hasn't appeared yet, is the addition of a second forward-facing camera in each side repeater, overlapping and complementing the rear-coverage camera. Though not optimal, it's placed more forward than the driver's head. And for left turns, it affords the possibility of seeing better past the view blockage of cars in the facing left-turn lane or bay.

As far as I can tell, the latest reports of "HW4" repeater cameras, now appearing on model Y, do not have the two-camera design. It looks like we'll have to continue to wait.

Or make right only turns and gun it... lol

 

[kabin]

11.4.2 is the beginning of the end for Waymo and Cruise

Yes and positively on (PoT).

 

[n.one.one]

News...or...not news???
1st is just an image, 2nd is a link to the tweet.

https://twitter.com/x/status/1662202904151838720

 

[kabin]

News...or...not news???

View attachment 941431

https://twitter.com/x/status/1662202904151838720

This is The Way.

 

[JulienW]

So last weekend is already here this weekend (maybe)? I wasn't expecting last weekend to at least next weekend or so. Is Elon getting better or just lucky on this one?

 

[n.one.one]

Now this IS news!
From TeslaFi.

 

[MarcG]

View attachment 941431

The question is, how wide - and to whom

 

[JHCCAZ]

Or make right only turns and gun it... lol

Maybe a workable theory!
So how about, they give me free Acceleration Boost and I'll stop posting about mistakes in the camera placement...

One of my firmest moral principles is that I can be bought off.

 

[aronth5]

Or make right only turns and gun it... lol

Funny that you mention that. 1 block away that is exactly what FSD does since with foliage back FSD can only see about 95 feet with cars typically going 30-35 mph. I won't do a left hand turn there due to risk.

 

[Mardak]

The question is, how wide - and to whom

You can view the versions that are getting the update even before installation completes on 2023.7.10 Official Tesla Software Update | Teslascope

It's going basically only to those already on 11.4.1 and not even that whole population, so for most of us still on 11.3.6 / 2022.45.15, we'll probably need to wait a bit longer. Roughly 3% of the FSD Beta fleet, so assuming around 400k vehicles, that's probably around 12k getting 2023.7.10.

 

[cowleggies]

You can view the versions that are getting the update even before installation completes on 2023.7.10 Official Tesla Software Update | Teslascope

It's going basically only to those already on 11.4.1 and not even that whole population, so for most of us still on 11.3.6 / 2022.45.15, we'll probably need to wait a bit longer. Roughly 3% of the FSD Beta fleet, so assuming around 400k vehicles, that's probably around 12k getting 2023.7.10.

While I agree most of us on 2022.45.15 are going to have to wait a bit longer, there is at least one car on 2022.45.15 queued up for 2023.7.10 so... there's hope?

 

[boonedocks]

Our ‘17 X just finished installing it

 

[MarcG]

Our ‘17 X just finished installing it View attachment 941444

What was your previous version?

 

[Mark II]

In release notes for 11.4.2
- Reduced highway false slowdowns related to underestimated velocities for faraway objects by adding 68K videos to the training set with improved auto-labeled ground truth.

Does this hopefully mean less phantom braking? Can't wait!

 

[powertoold]

In release notes for 11.4.2
- Reduced highway false slowdowns related to underestimated velocities for faraway objects by adding 68K videos to the training set with improved auto-labeled ground truth.

Does this hopefully mean less phantom braking? Can't wait!

Every single release tries to reduce unnecessary braking lol

 

[uscbucsfan]

I have 11.4.2!