FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[dgatwood]

The difference is without a camera further forward FSD cannot make the same good decision a human can. We simply see better by leaning forward.

We can't lean farther forward than the windshield, where the camera is. The front-back position isn't the problem. If anything, it is sometimes a disadvantage, because our eyes' vision doesn't get completely obscured by a single drop of rain, thanks to the drop of rain blocking 0.2 degrees of arc relative to our eyes instead of 30 degrees of arc.

The main difference is that because we are far from the window, we can get a useful approximation of an image at a steep angle through the glass, looking to both sides, whereas with a single front camera, or even with a normal and zoomed-in front camera, you can't do that.

What they need to do is replace the cluster of three cameras with three cameras that are zoomed in somewhat more than they currently are, and with the outer two cameras angled outwards at about 45 degrees. That would greatly improve their ability to pull out of parking places, make unprotected turns, etc. This should be an easy change, even for existing cars.

The other thing they need is two corner cameras at the rear, for parking purposes. By the time the side repeater cameras or B-pillar cameras have a view of anything, you're already out in the lane, and it is too late for them to matter. However, you can work around that by always backing into every non-slanted parking place.

 

[Mark II]

Every single release tries to reduce unnecessary braking lol

Most the other stats they give a % improvement, except this one. So it could be as little as 0.42% improvement, or as much as 6.9%!

 

[uscbucsfan]

I have 11.4.2!

2023.7.10

 

[JohnB007]

2023.7.10

Any obvious differences?

 

[Twiglett]

In release notes for 11.4.2
- Reduced highway false slowdowns related to underestimated velocities for faraway objects by adding 68K videos to the training set with improved auto-labeled ground truth.

Does this hopefully mean less phantom braking? Can't wait!

Sadly there doesn't seem to be any difference in the notes from 11.4.1->11.4.2

 

[aerodyne]

As Elon Mush said "Major update coming in a few months"

Better than "Soon"

 

[Ramphex]

Perhaps TMC should develop its own AutoPilot to steer it's readers away from potheads. Uh pot holes. Maybe an AutoIgnore feature?

Seriously, though, with only 368 pages of posts (so far) since the beginning of 2022, that is less than one page per day. What else are we supposed to do while we wait for the next version?

What might be nice would be wikipedia style documents about the FSD beta process and each version. Ditto on all sorts of other model specific and energy product issues and questions. A me-too click on each reported bug. Maybe a collective, dynamic FAQ site? TMC's network news style of does not seem to support easily summarizing and updating what we have learned collectively.

And, no, I am not volunteering to create such.

The problem is, that’s nearly impossible and borderline useless. There legit no actual knowledge of anything that’s going on at Tesla in between these updates other than some mumbo jumbo changelog that 99% of the people don’t understand how it applies in real life situations.

So between that and the fact that every year, model, submodel, hardware, seems to have a different experience between different drivers in different cities and states, such thing would be moot.

Unless you need a FAQ to tell you to “clean your cameras”, “calibrate your cameras”, “do a two scroll wheel reset” then what else could be useful for anyone in there?

Not trying to be offensive, just realistic.

 

[uscbucsfan]

Any obvious differences?

I'm a little intoxicated. I'll tell you tomorrow.

 

[aerodyne]

The problem is, that’s nearly impossible and borderline useless. There legit no actual knowledge of anything that’s going on at Tesla in between these updates other than some mumbo jumbo changelog that 99% of the people don’t understand how it applies in real life situations.

So between that and the fact that every year, model, submodel, hardware, seems to have a different experience between different drivers in different cities and states, such thing would be moot.

Unless you need a FAQ to tell you to “clean your cameras”, “calibrate your cameras”, “do a two scroll wheel reset” then what else could be useful for anyone in there?

Not trying to be offensive, just realistic.

I always wanted to be a 1 percenter

 

[willow_hiller]

Quick turnaround video by Chuck:

 

[Mengy]

Quick turnaround video by Chuck:

That was a really good drive Chuck had there. FSD is starting to look really confident and capable now in ways it hasn't before.

 

[boonedocks]

What was your previous version?

2023.7.5

 

[jebinc]

Quick turnaround video by Chuck:

Not impressive…

 

[kabin]

Quick turnaround video by Chuck:

We are so overdue for anything post 11.3.6 garbage that I don't want to crap on this but that unprotected left turn was scary at best. By the mercy of gawd it didn't run into the karmann ghia but not unusual for FSD UPLs. Sometimes FSD doesn't see and/or respond in time. One phantom brake in the first 5 1/2 mins. Gotta hope PBs aren't back with a vengeance.

 

[jebinc]

We are so overdue for anything post 11.3.6 garbage that I don't want to crap on this but yikes! The unprotected left turn was scary at best. By the mercy of gawd it didn't run into the karmann ghia but not unusual for FSD UPLs. Sometimes FSD doesn't see and/or respond in time. One phantom brake in the first 5 1/2 mins.

I didn’t think I would be saying, “Junk” so early in this release cycle….

 

[D Good]

We are so overdue for anything post 11.3.6 garbage that I don't want to crap on this but that unprotected left turn was scary at best. By the mercy of gawd it didn't run into the karmann ghia but not unusual for FSD UPLs. Sometimes FSD doesn't see and/or respond in time. One phantom brake in the first 5 1/2 mins. Gotta hope PBs aren't back with a vengeance.

Loved to see the ghia! I have a 74 convertible myself. Great car.

 

[Twiglett]

Quick turnaround video by Chuck:

that was OK, not great.
Chuck didn't notice that the car started signalling to move right just before his last left turn at the end of the trip.
It signalled right for a few blinks then stopped. Other road users in that area must think that Tesla drivers are always lost.
Also noticed that it was happy to just sit in the left lane for miles for no reason.
Not pleased to see those random slowdowns either. That felt like more than I'm seeing on 11.3.6

 

[sleepydoc]

I'm a little intoxicated. I'll tell you tomorrow.

celebrating the new version?

 

[uscbucsfan]

celebrating the new version?

Celebrating Friday

 

[JHCCAZ]

Celebrating Friday

TGIFSD