FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Natal]

I got a push

View attachment 941510

I’m surprised I received the update to 2023.7.10, since I haven’t been receiving any early updates in quite a while. Unfortunately, my 2020 MY is at home and I’m on a trip; will have to wait until next Saturday to try it out.

 

[bd7349]

Anyone know if 11.4.2 resets strikes? I just got my fifth one tonight while looking straight ahead and now have lost beta.

Feels really bad considering these strikes have been adding up since August 2022 and each one was when I was looking directly straight ahead/squinting or wearing glasses. They really need to fix these false positives and reset strike numbers every 2 weeks instead of only resetting it once you’ve hit 5/5 strikes after 2 weeks.

 

[n.one.one]

Anyone know if 11.4.2 resets strikes? I just got my fifth one tonight while looking straight ahead and now have lost beta.

Feels really bad considering these strikes have been adding up since August 2022 and each one was when I was looking directly straight ahead/squinting or wearing glasses. They really need to fix these false positives and reset strike numbers every 2 weeks instead of only resetting it once you’ve hit 5/5 strikes after 2 weeks.

An important change in 11.4.2 (2023.7.10) is that the suspension is now reduced to one week. It will be interesting to see what yours is. Good luck, and please let us know.

 

[JulienW]

We can't lean farther forward than the windshield, where the camera is.

The wide angle front camera doesn't help much in seeing down the road. But if the B-Pillar is blocked you can lean forward and look back. Here the B-Pillar is blocked so behind the red line is mostly "blind" but you can lean forward and look back and around. And this doesn't even include all the other advantages we have over fixed position cameras like tilting, ducking, shifting angle, peaking under, around or even through occlusions.

EDIT: If you want an example of how limiting a fixed camera is get in a car with a rear view mirrors (especially side view) that are cameras and back up. You will be lowering and tilting you head to see more in the mirrors but it doesn't work because it is fixed. We just do this naturally.

 

[Scott Chapman]

The wide angle front camera doesn't help much in seeing down the road. But if the B-Pillar is blocked you can lean forward and look back. Here the B-Pillar is blocked so behind the red line is mostly "blind" but you can lean forward and look back and around. And this doesn't even include all the other advantages we have over fixed position cameras like tilting, ducking, shifting angle, peaking under, around or even through occlusions.

EDIT: If you want an example of how limiting a fixed camera is get in a car with a rear view mirrors (especially side view) that are cameras and back up. You will be lowering and tilting you head to see more in the mirrors but it doesn't work because it is fixed. We just do this naturally.

I really don't think the problem is that there are blind spots where the cameras can't see something. You have to remember that they are monitoring events preceding that image; specifically, if there IS something in that blind spot it got there by passing through space where the sensors DID see it. In other words, the car does know there is something there even if at that specific time it is in the blind spot.
The car does know where the blind spots are, and also knows when things enter/exit them.

 

[JohnB007]

May be they are holding off updating people on 2022.45.15 to 2023.12.10 because v11.4.* is in a good shape?

Yes I can hope...

Wow I was right!

 

[LowlyOilBurner]

An important change in 11.4.2 (2023.7.10) is that the suspension is now reduced to one week. It will be interesting to see what yours is. Good luck, and please let us know.

View attachment 941524

What was it before? I got two strikes a while ago (months, it seems) for changing the playlist on Spotify, the strikes never went away, even with software updates (11.3.2 to 11.3.6, ect).

 

[aronth5]

We can't lean farther forward than the windshield, where the camera is. The front-back position isn't the problem. If anything, it is sometimes a disadvantage, because our eyes' vision doesn't get completely obscured by a single drop of rain, thanks to the drop of rain blocking 0.2 degrees of arc relative to our eyes instead of 30 degrees of arc.

The main difference is that because we are far from the window, we can get a useful approximation of an image at a steep angle through the glass, looking to both sides, whereas with a single front camera, or even with a normal and zoomed-in front camera, you can't do that.

What they need to do is replace the cluster of three cameras with three cameras that are zoomed in somewhat more than they currently are, and with the outer two cameras angled outwards at about 45 degrees. That would greatly improve their ability to pull out of parking places, make unprotected turns, etc. This should be an easy change, even for existing cars.

The other thing they need is two corner cameras at the rear, for parking purposes. By the time the side repeater cameras or B-pillar cameras have a view of anything, you're already out in the lane, and it is too late for them to matter. However, you can work around that by always backing into every non-slanted parking place.

I didn't say the front facing camera was the problem since it cannot see 90 degrees to either side which humans can. The problem is relying on the B-pillar camera too much. When I view FSD beta testers they never show these obstructed view intersections. For most I don't believe they have this situation.

 

[aronth5]

Why is there a continuing undercurrent of folks who claim that their two eyes can see better
than eight cameras? This reminds me of "audiophiles" who claim they can hear things better
than 16-bit audio, while never doing better in A-B comparison tests. If humans need to lean forward
to see better at blind intersections, then Tesla cars can creep forward to do the same thing.

You're missing one important fact. If you creep far enough into the intersection for the B-pillar to see you're affecting traffic since your too far into the intersection. It's easy to tell when the crossing cars have to go across the center line so they don't hit you. I've had traffic stop in both directions because of this. It has nothing to do with 2 human eyes vs 8 cameras and is simply due to trees and bushes too close to the road edge.

 

[n.one.one]

What was it before? I got two strikes a while ago (months, it seems) for changing the playlist on Spotify, the strikes never went away, even with software updates (11.3.2 to 11.3.6, ect).

The suspension has been 2 weeks for a long time. You are locked out of beta after 5 strikes if you have an internal camera and 3 strikes if you don't have an internal camera. Unfortunately, except for a reset to zero strikes that Tesla granted to everyone a few years ago the strikes don't go away until you get kicked out for the suspension period.

 

[aerodyne]

So, it appears from reading recent posts that 11.4.2 is either better or worse than 11.3.6.

Should I wait until 11.6.9?

 

[FSDtester#1]

You're missing one important fact. If you creep far enough into the intersection for the B-pillar to see you're affecting traffic since your too far into the intersection. It's easy to tell when the crossing cars have to go across the center line so they don't hit you. I've had traffic stop in both directions because of this. It has nothing to do with 2 human eyes vs 8 cameras and is simply due to trees and bushes too close to the road edge.

Agreed, and the only workaround seems for the car to angle itself more to get a better view from the B pillar while creeping

 

[Rikster]

Looks like a huge push of 11.4.2 is happening this morning! I just got a notification that it's downloading. Looking forward to giving it a try. I expect some minor improvements in smoothness compared to my current 11.3.6 version.

 

[jebinc]

Here's one with refresh S:

My MS Plaid is now downloading it….

 

[jebinc]

I hear ya. For the sake of progress the expectation bar might need to be lowered for now. It's almost like FSD ran out of fingers to plug the leaks in the dike. Fill a new leak only to have an old leak return.

Will be retesting the dike shortly....

 

[aerodyne]

Will be retesting the dike shortly....

The biggest leak is the removal of Radar.

 

[FloridaJohn]

I really don't think the problem is that there are blind spots where the cameras can't see something. You have to remember that they are monitoring events preceding that image; specifically, if there IS something in that blind spot it got there by passing through space where the sensors DID see it. In other words, the car does know there is something there even if at that specific time it is in the blind spot.
The car does know where the blind spots are, and also knows when things enter/exit them.

That, of course, is the goal, but it appears that currently the car’s “memory” is less than a second. You can see this when you are sitting at an intersection behind another car. Watch the cross traffic disappear as the car loses sight of the cross traffic.

The predictive capabilities are currently very limited, which is why it frequently makes poor decisions.

 

[rlsd]

I really don't think the problem is that there are blind spots where the cameras can't see something. You have to remember that they are monitoring events preceding that image; specifically, if there IS something in that blind spot it got there by passing through space where the sensors DID see it. In other words, the car does know there is something there even if at that specific time it is in the blind spot.
The car does know where the blind spots are, and also knows when things enter/exit them.

I think that way too. Even when the camera is placed near the headlight, blocking still exists. Here is my daily situation: my car exits from the freeway to a 6 lane 50 mph street (3 lanes on each side) and there are 3 exit lanes (1 lane for left turn and 2 lanes for right turn). When I creep up the car to make right turn from the rightmost exit lane, another driver who also wants to make right turn on the left blocks my view for the traffic coming from the left. This really frustrates me.

 

[Mardak]

Looks like a huge push of 11.4.2 is happening this morning!

Yup approximately a doubling of the rollout. Pretty quick expansion from initial 3% to 10% now 20% within 24 hours. All coming from previous FSD Beta 11.x although so far no confirmed installations of 11.4.2 on HW4?

 

[old pilot]

Got 11.4.2 this morning. For the past month, with 11.3.6, when I activate the auto pilot at the end of my driveway, and a route programed that requires a left turn, the car has consistently gone right. I have obediently pushed the right prayer button and spoken to the neural network god telling him the car has turned the wrong way again. Today my prayers were answered.