FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[enemji]

And this is why we keep both hands on the wheel even before anticipating circumstances.

Path planner flipped out (car seemed to know due to red hands which seemed basically concurrent with disengagement).

Can keep an eye on the leather seam on the steering wheel to see the deviation - unfortunately too dark to see my hands moving with the wheel.

Caught it quickly but not fast enough to not freak out a nearby car (who was not in danger).

Good riddance, path planner. I'm sure the neural nets will never do this.

Autopilot Disengaged. What happened?

 

[AlanSubie4Life]

View attachment 1011790

Autopilot Disengaged. What happened?

Jerked to the right. Watch the leather seam.

Disengaged due to torque. I resisted the turn to the other lane which path planner wanted.
Before:

After (during…actually keeps turning further):

 

[enemji]

Jerked to the right. Watch the leather seam.

Disengaged due to torque. I resisted the turn to the other lane which path planner wanted.
Before

After:

I did not understand this comment "Path planner flipped out (car seemed to know due to red hands which seemed basically concurrent with disengagement)."

 

[AlanSubie4Life]

I did not understand this comment "Path planner flipped out (car seemed to know due to red hands which seemed basically concurrent with disengagement)."

Yeah you can see the blue tentacle go all wrong and then as I disengage by resisting the car’s turn to the right to follow the tentacle, the urgent red hands appear (not sure whether because of disengagement or because it had panicked immediately prior). It’s an unusual hands warning (red hands).

 

[enemji]

Yeah you can see the blue tentacle go all wrong and then as I disengage by resisting the car’s turn to the right to follow the tentacle, the urgent red hands appear (not sure whether because of disengagement or because it had panicked immediately prior). It’s an unusual hands warning (red hands).

ah... it was too quick. Got it now

 

[sleepydoc]

You can see it whip right to follow path planner. No warning. Leather seam.

Caught by disengagement. Torque.

Probably would have corrected on its own but who knows. Never will know.

I don’t think one’s hand should be through the wheel.

Sorry, still not seeing it - I can tell when you disengage but there's nothing distinguishable from all the camera shaking. Regardless, like I said - I catch the car before it leaves the lane so my hand position isn't an issue, IMO.

 

[AlanSubie4Life]

Sorry, still not seeing it - I can tell when you disengage but there's nothing distinguishable from all the camera shaking. Regardless, like I said - I catch the car before it leaves the lane so my hand position isn't an issue, IMO.

You can see the steering wheel in the attached pictures.

You can also see the camera angle relative to the screen change suddenly in response to movement.

I have no idea how far this would have gone without intervention. Probably not too much further? We’ll never know.

 

[Therealm80]

When I disconnect auto pilot, it goes to full manual driving when before it would keep the speed but let me steer. How do I get it to keep the speed and let me steer in times such as rain when it wants to slow me down so I don’t have to keep my foot on the accelerator.

 

[AlanSubie4Life]

When I disconnect auto pilot, it goes to full manual driving when before it would keep the speed but let me steer. How do I get it to keep the speed and let me steer in times such as rain when it wants to slow me down so I don’t have to keep my foot on the accelerator.

Go back to double pull.

https://www.tesla.com/ownersmanual/modely/en_us/GUID-20F2262F-CDF6-408E-A752-2AD9B0CC2FD6.html#:~:text=Autopilot%20Activation%3A%20Choose%20how%20to,quick%20succession%20to%20engage%20Autosteer.

 

[Therealm80]

Go back to double pull.

https://www.tesla.com/ownersmanual/modely/en_us/GUID-20F2262F-CDF6-408E-A752-2AD9B0CC2FD6.html#:~:text=Autopilot%20Activation%3A%20Choose%20how%20to,quick%20succession%20to%20engage%20Autosteer.

Awesome, thanks man

 

[rlsd]

2023.44.30.13/V11.4.9 is out.
No chance for 2023.44.30.12/V12 1.2

 

[jebinc]

2023.44.30.13/V11.4.9 is out.
No chance for 2023.44.30.12/V12 1.2

Nope.

 

[Twiglett]

Great fun on the drive home with 2023.44.30.8 FSD(wtf) 11.4.9
Everything seemed to be going swimmingly until the car decided the lane that will turn off the highway in 2.5 miles isn't correct so it executes a signal and move "changing lanes to follow route" - except there's another car in that lane. I thought it would stop because the bright red warning was showing, but no, it decided it was vital that we be in that lane.
Conversely, near the end of the highway part of the commute, we are 800 yards from taking our exit on the right when it decides we really, really need to be in the left lane In stop and go rush hour traffic.
I don't know why it surprises me that it can make two radically different decisions to the same type of scenario. Especially when I have minimal lane changes set.

 

[lascavarian]

Running a v11 release on v3 hw. Using single pull activation.

Encountered ground fog related to warming temperatures over very frozen ground & snow.

Traffic was moving well as fog was light and visibility good. Wipers in auto and not wiping. Light mist on glass.

FSD just went away with a message about conditions. Conditions were not bad but light fog was evident.

First time I have seen this cautious behavior. I am guessing that FSD got a weather advisory about fog.

A couple hours later, perhaps sun burning off fog and operation restored.

 

[mgs333]

As can be expected, this thread is slowly dying, to be replaced by the V12 thread(s). So not sure what the purpose is to keep harping on the failings of V11 at this point but this morning, my car tries to drive straight into an oncoming pickup truck before I took over.

This middle aged guy even made my first Youtube short, thereby making a horizontal video into a vertical one.

I feel it's related to this issue I see almost every day.

 

[AlanSubie4Life]

As can be expected, this thread is slowly dying, to be replaced by the V12 thread(s). So not sure what the purpose is to keep harping on the failings of V11 at this point but this morning, my car tries to drive straight into an oncoming pickup truck before I took over.

This middle aged guy even made my first Youtube short, thereby making a horizontal video into a vertical one.

I feel it's related to this issue I see almost every day.

Yeah just bad planner stuff. Pretty routine.

Kind of a debacle. It’ll be good to get rid of the planner, but I don’t expect much immediate progress with the replacement.

I’ll be surprised if it doesn’t bring out new problems that did not exist before but I guess that is what regression testing with extensive simulation is for.

And we know Tesla would not release v12 with regressions.

This middle aged guy even made my first Youtube short, thereby making a horizontal video into a vertical one.

This is the way.

 

[JulienW]

.....This middle aged guy even made my first Youtube short, thereby making a horizontal video into a vertical one.....

OT but I'll die on this hill.

 

[mgs333]

OT but I'll die on this hill.

View attachment 1012262

I'm with you, while uploading my video to YT, it asked me to edit it into a YT short so I gave it a try out of curiosity.

 

[jebinc]

As can be expected, this thread is slowly dying, to be replaced by the V12 thread(s). So not sure what the purpose is to keep harping on the failings of V11 at this point but this morning, my car tries to drive straight into an oncoming pickup truck before I took over.

This middle aged guy even made my first Youtube short, thereby making a horizontal video into a vertical one.

I feel it's related to this issue I see almost every day.

R.I.P.

 

[jebinc]

Yeah just bad planner stuff. Pretty routine.

Kind of a debacle. It’ll be good to get rid of the planner, but I don’t expect much immediate progress with the replacement.

I’ll be surprised if it doesn’t bring out new problems that did not exist before but I guess that is what regression testing with extensive simulation is for.

And we know Tesla would not release v12 with regressions.


This is the way.

J