FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[EVNow]

Ideally they merge Main and FSDb into like “2023.11.x” or something and go from there with the only special versions being factory builds and very specific vehicles.

For that to work, FSD releases have to keep up with main releases i.e. they have to synchronize the changes. Main releases are generally a lot more easy and more definitive than FSD releases - so can be difficult to manage this.

 

[kabin]

Yowza! The video is sped up so probably best to run it at 0.25x or 0.5x speed.

 

[TK211X]

For that to work, FSD releases have to keep up with main releases i.e. they have to synchronize the changes. Main releases are generally a lot more easy and more definitive than FSD releases - so can be difficult to manage this.

I get it, we have to go on the furry forums and start dropping hints that FSDb development should speed up. All the devs from Tesla will see it and get the message.

 

[cowleggies]

One person outside of the 11.3.2 cohort has gotten 11.3.3….

 

[kabin]

I get it, we have to go on the furry forums and start dropping hints that FSDb development should speed up. All the devs from Tesla will see it and get the message.

I get the tone but it probably makes more sense to slow down or bundle the main releases so they have a chance of being in phase with FSD releases.

 

[JulienW]

Chuck 11.3.3 Memorial Park and doing qa little parking lot playing.

 

[kabin]

V11.3.3 might be addressing V11.3.2 sluggish UPL acceleration with increased accel jerk. If so Crazy Ivan turns are back. Hold on to your coffee cups.

 

[kabin]

Here's a delayed hard braking for orange cones. There was another hard brake instance earlier where the orange cone was rendered on the UI but wasn't anywhere to be seen on road. Overall V11.3.3 looks a bit rough on Chuck's Memorial route.

 

[kabin]

The lead vehicle slows as it pulls into the left lane. The UI shows the lead vehicle lane location and brake light illuminate close to real time but the ego's brakes respond to a much different scenario with an unnecessary hard brake action. It's as if brake control function is processing lead vehicle position from ~2 seconds earlier. Maybe an increased delayed response to emergent braking scenarios with V11.3.3?

I thought accel/braking functions were expected to improve with NNs.

 

[DarkForest]

I get it, we have to go on the furry forums and start dropping hints that FSDb development should speed up. All the devs from Tesla will see it and get the message.

I didn’t know FSDb was written in Rust…

 

[jebinc]

Here's a delayed hard braking for orange cones. There was another hard brake instance earlier where the orange cone was rendered on the UI but wasn't anywhere to be seen on road. Overall V11.3.3 looks a bit rough on Chuck's Memorial route.

More junk and more delays. I suspect 11.3.3 is not long for this world based on early reports.

 

[LowlyOilBurner]

When 11.3.3 will brake for a rabbit, I'd like to see Dan O'Dowd try and fool it with a baby doll:

https://twitter.com/x/status/1639551347988090881

Well, it may brake for a rabbit (I’ve had 10.69.25.X do this), it completely ignored a huge fu%cking tree that was down across a road near me this morning. If I didn’t take over, I’m pretty sure it would have run right into it. Maybe increase that % of the occupancy network, huh?

 

[Dewg]

Well, it may brake for a rabbit (I’ve had 10.69.25.X do this), it completely ignored a huge fu%cking tree that was down across a road near me this morning. If I didn’t take over, I’m pretty sure it would have run right into it. Maybe increase that % of the occupancy network, huh?

Please people, do NOT let the system drive towards a downed tree. It's L2 not L4+, and you must intervene and drive in those circumstances. It's designed to handle boring drives like cruise control, and is not designed to handle all edge cases or even complicated maneuvers.

Even the high L4 AVs loaded with sensors can't handle all cases. Please be safe out there.

 

[jebinc]

Well, it may brake for a rabbit (I’ve had 10.69.25.X do this), it completely ignored a huge fu%cking tree that was down across a road near me this morning. If I didn’t take over, I’m pretty sure it would have run right into it. Maybe increase that % of the occupancy network, huh?

11.3.3 - DOA. Maybe 11.3.4?

 

[LowlyOilBurner]

Please people, do NOT let the system drive towards a downed tree. It's L2 not L4+, and you must intervene and drive in those circumstances. It's designed to handle boring drives like cruise control, and is not designed to handle all edge cases or even complicated maneuvers.

Even the high L4 AVs loaded with sensors can't handle all cases. Please be safe out there.

Trust me, I wouldn’t let it proceed. I reported it, obviously.

 

[kylinblue]

Any NC resident can test v11 for me at this spot to see if it fails there?

 

[kylinblue]

View attachment 921240
One person outside of the 11.3.2 cohort has gotten 11.3.3….

View attachment 921241

Could very well be someone waited too long to install the 1132

 

[LowlyOilBurner]

Initial thoughts are, they dialed back the “assertiveness” of 11.3.2. Lot more “creeping to check for visibility” where it wouldn’t on 11.3.2. Had a couple dopey drives on 11.3.3 this afternoon where it dilly dallied around otherwise simple maneuvers. Frustrating, and reported each instance. Kind of a bummer, if they “dumb” it down for a wide release, folks are gonna be unhappy. Leads me to think this is more than simple “bug” fixes. Maybe an A-B comparo? 70 miles today, and I’d say 11.3.2 was better, “city” wise. Highway seems about the same. Maybe moves out of the passing lane a little better?

 

[TK211X]

 

[nvx1977]

Initial thoughts are, they dialed back the “assertiveness” of 11.3.2. Lot more “creeping to check for visibility” where it wouldn’t on 11.3.2. Had a couple dopey drives on 11.3.3 this afternoon where it dilly dallied around otherwise simple maneuvers. Frustrating, and reported each instance. Kind of a bummer, if they “dumb” it down for a wide release, folks are gonna be unhappy. Leads me to think this is more than simple “bug” fixes. Maybe an A-B comparo? 70 miles today, and I’d say 11.3.2 was better, “city” wise. Highway seems about the same. Maybe moves out of the passing lane a little better?

Unlikely to be A/B if they're just going to upgrade all the "A" cohort to "B".