FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Ramphex]

I 100% think my car would jam on the brakes like that in that scenario. It would be extremely bad. Your video is exactly what I would expect.

I 100% believe you have horrible jerky braking in other scenarios too. And that it is very common and not unusual.

That is what I keep saying, after all.

That behavior in traffic makes me feel like I'm going to get rear ended during the slam portion. I don't understand where things go wrong. It starts to slow down gradually as it should, then slams the brakes, then goes back to gradual slowing.

 

[FSDtester#1]

It'll be junk until the braking is fixed. Phantom braking and the spill your coffee braking. I have been complaining about both for many months now, used to always hit the camera button for this slam-the-brakes nonsense and it's yet to be resolved.

Once again, granted i have only 30 miles on it, braking is 100% better for me.
I hated not using regen... now it does for me all the way to full stop and the hold.

 

[Ramphex]

Once again, granted i have only 30 miles on it, braking is 100% better for me.
I hated not using regen... now it does for me all the way to full stop and the hold.

Interestingly enough, if there's no cars at the red light, it comes to a complete stop smoothly using regen.

 

[FSDtester#1]

Individual tolerance level for its behavior; good, bad and ugly. Perspective. Differing expectations, etc.

You forgot how much junk in the frunk...

 

[FSDtester#1]

To rid yourself of anger when this happens, just think of and shout out “bitch-FSDb” every time it happens. Guaranteed to feel better!

Yes, this, but do it into voice notes!

 

[FSDtester#1]

There are definitely changes beyond highways. For me it’s been a bit of a 2 steps forward, 1 step back, kind of release but I have found 2-3 situations where lane positioning or lane detection issues have been resolved

That was routine when they originally were updating every 2 to 3 weeks

Edit: the 2 steps forward, 1 step back

 

[AlanSubie4Life]

That behavior in traffic makes me feel like I'm going to get rear ended during the slam portion. I don't understand where things go wrong. It starts to slow down gradually as it should, then slams the brakes, then goes back to gradual slowing.

Exactly.

Interestingly enough, if there's no cars at the red light, it comes to a complete stop smoothly using regen.

Keep driving! You’ll find this is not always the case.

 

[FSDtester#1]

Interestingly enough, if there's no cars at the red light, it comes to a complete stop smoothly using regen.

So adding cars or traffic to the mix is what screws it up?

 

[EVNow]

Videos:
1) Dash & road view of unnecessary excessive braking. Used the community gate as it's a static object, and easier to recreate the issue multiple times to record different angles than playing with the camera in traffic. It starts slowing down as it normally should, then goes full retard.

iCloud Photos - Apple iCloud

View, organize, and share photos and videos with iCloud Photos on the web. Changes will sync across your devices with iCloud.

share.icloud.com

2) Side view of the above video, what it feels like driving on beta for many releases now, mostly the reason I barely use it anymore.

iCloud Photos - Apple iCloud

View, organize, and share photos and videos with iCloud Photos on the web. Changes will sync across your devices with iCloud.

share.icloud.com

For the record… I can’t see the videos. I get browser not supported … on iPad (using TMC app).

 

[FSDtester#1]

For the record… I can’t see the videos. I get browser not supported … on iPad (using TMC app).

They are worth the watch, try a computer, if people have those anymore.

 

[EVNow]

They are worth the watch, try a computer, if people have those anymore.

Yes, when I get to that later (I still mostly use my “desktop” that sits under the table).

 

[FSDtester#1]

Yes, when I get to that later (I still mostly use my “desktop” that sits under the table).

Desktop? Is that considered a computer? I thought those were extinct, and recently laptops were put on the endangered species list

 

[mtbwalt]

Beware of left turns into oncoming traffic lanes. Had two today in different locations using 11.3.4. One was technically a shared turning lane but the vehicle basically ignored the turn markings on the ground. The other was into the left turn lane for oncoming traffic

 

[Ramphex]

So adding cars or traffic to the mix is what screws it up?

Or a gate... apparently

For the record… I can’t see the videos. I get browser not supported … on iPad (using TMC app).

It's the TMC app, copy pasta links into Safari and they'll work.

 

[am_dmd]

Did you watch my videos from my trip getting the hemorrhoid cream from the CVS earlier? How do you guys not get annoyed by that braking behavior?

Do you have “apply brake when regen is limited”?
I don’t have 11 but hard braking is so annoying on 69. I don’t why beta can’t figure out to use regen for smooth stopping. It sees the red light at good enough distance and sees the traffic but yet drives at full speed until it’s within pre-programmed car length and then brakes like oh s**t

 

[Ramphex]

Do you have “apply brake when regen is limited”?
I don’t have 11 but hard braking is so annoying on 69. I don’t why beta can’t figure out to use regen for smooth stopping. It sees the red light at good enough distance and sees the traffic but yet drives at full speed until it’s within pre-programmed car length and then brakes like oh s**t

Yea it’s been doing that in every revision. Aggressive breaking too, that’s why I compare it to spilling coffee or deep throating straws from drinks

 

[FSDtester#1]

Yea it’s been doing that in every revision. Aggressive breaking too, that’s why I compare it to spilling coffee or deep throating straws from drinks

Turn it off, then test. I never have used it.

 

[AlanSubie4Life]

Turn it off, then test. I never have used it.

It doesn’t make any difference in my experience. When it is off, it just makes the regen/brake bar (in the models that have that) show no friction braking contribution. The brake is still used of course (you can feel it if you have your foot over it). It just does not display it.

The use of brakes should be identical in both cases when under computer control. It has a target so it is what it is. Physics dictates the rest.

 

[FSDtester#1]

It doesn’t make any difference in my experience. When it is off, it just makes the regen/brake bar (in the models that have that) show no friction braking contribution. The brake is still used of course (you can feel it if you have your foot over it). It just does not display it.

The use of brakes should be identical in both cases when under computer control. It has a target so it is what it is. Physics dictates the rest.

I agree.. I just want @Ramphex to set all his settings like mine and see if there are any changes. For example, I switched to assertive about a month ago from chill.
That made beta better for me. Also, Maybe try the new setting for limited lane changes at start of every drive. I haven't tried that yet, but that may help people with poor lane selections or frequent bouncing between lanes.
I am not a coder, but wonder if 1 setting can have an effect on other things that are unrelated.

 

[am_dmd]

It doesn’t make any difference in my experience. When it is off, it just makes the regen/brake bar (in the models that have that) show no friction braking contribution. The brake is still used of course (you can feel it if you have your foot over it). It just does not display it.

The use of brakes should be identical in both cases when under computer control. It has a target so it is what it is. Physics dictates the rest.

You are correct no difference if it is on or off. My reasoning for turning it off is to avoid giving a rain check to tesla to use friction brakes whenever they want (was hoping 11 changed it but looks like not ).