FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Mardak]

My observation is that the lead car is always darker in color

Yeah, sometimes there's multiple lead cars such as during a lane change. Additionally, slower adjacent vehicles are highlighted the same way, so more generally, we can say Autopilot does this type of highlighting for vehicles it's actively adjusting its speed for. This highlighting for "Adjacent Lane Speed Adjustments" was added in 2019.40.2 with an old Dirty Tesla video showing off the initial implementation.

Here's a clearer view using some examples from 2021 Autopilot and FSD Beta 11.3.1 both left and right sides:

Looks like the adjacent lane chevrons are gone with FSD Beta 11.x single stack. I've experienced both sides getting chevrons and highlights at the same time near an interchange, and I estimate the highlighting thresholds for right side was 30mph slower while the left side was 10mph slower.

 

[Dewg]

Interesting. Have we seen blue highlighted cars on freeways with V11? If not, could be that dark cars are the equivalent of blue cars on freeways. Blue might only show up on city streets?

 

[willow_hiller]

Not sure what the double green light bug is:

https://twitter.com/x/status/1633994781725544448

Is it where FSDb proceeds on red because there's another set of lights a short distance down the road?

 

[powertoold]

Not sure what the double green light bug is:

https://twitter.com/x/status/1633994781725544448

Is it where FSDb proceeds on red because there's another set of lights a short distance down the road?

I believe it's where fsd slows down when there are two sets of green lights near each other in front of the car.

 

[AlanSubie4Life]

Have we seen blue highlighted cars on freeways with V11?

Yes. They show up when the car is thinking about changing lanes.

I believe it's where fsd slows down when there are two sets of green lights near each other in front of the car.

Yeah, I’ve seen this. If the light pair is susceptible to the green light slowing in the case of two greens, then in the event the second light is red, it seems to lead to potential red light running attempts as well (though it is hard to duplicate). Keeps you guessing!

This will do it (visibility of the lights is not an issue), and you can see the video of the red light running here, posted elsewhere:

Google Maps

Find local businesses, view maps and get driving directions in Google Maps.

maps.app.goo.gl

It’s not really surprising since 11.3 appears to be basically the same as 10.69 on surface streets - probably just different slider settings and a few patches. Just going off the release notes, anyway.

So we should expect things to basically be fundamentally the same, but with different thresholds.

 

[Dewg]

Yes. They show up when the car is thinking about changing lanes.

Now I'm curious what the dark colored cars represent vs blue. Which is higher priority, or are they situational but have the same priority to the AI?

 

[willow_hiller]

Now I'm curious what the dark colored cars represent vs blue. Which is higher priority, or are they situational but have the same priority to the AI?

If it's the same highlighting as is used on AP and NoA, it signifies the "lead vehicle." So most of the time it's simply the car in front of you, but in the case of slowing for adjacent traffic, the speed of your car is being governed by multiple lead vehicles.

Again, if it's the same as NoA, in light screen mode the highlighting is a darker shade, and dark screen mode the highlighting is a lighter shade. Have we seen any V11.3.1 videos in dark mode, yet?

 

[EVNow]

https://twitter.com/x/status/1633986937529815042

Spoiler

warning: Deep Fake

 

[aerodyne]

MX at the Santa Barbara store today, was told delivered yesterday. No USS, no new cameras, but appears HW 4.0 with latest FSD update:

 

[MP3Mike]

but appears HW 4.0 with latest FSD update:

That is not a FSD Beta firmware version.

 

[powertoold]

This video is very impressive. It shows 11.3.1 handling everything, cautiously but smoothly. You can begin to imagine the robotaxi dream becoming reality. It is a night and day improvement vs a year ago in downtown Chicago:

 

[thewishmaster]

Mine will make an attempt once a month. In fact it just tried maybe last week. You can tell it’s up to no good when you pull up to a red light and the on-screen cross traffic is all blue

Tried again this evening - car was following another that ran a very stale yellow. The traffic across the intersection was backed up to the edge of the road, so FSDb stayed behind the crosswalk on the correct side, waiting to get enough space across the intersection. After some hesitation it decided to go on full-on red, after waiting out the first few crossing cars.

Can’t imagine transitioning more of this logic to neural nets will help reduce these cases…

 

[scottf200]

Wonder which one is being displayed
If the "regular" walking human wasn't there would it "do further damage"

Via DirtyTesla video

 

[aerodyne]

That is not a FSD Beta firmware version.

My bad. I saw V11.x and did not verify what the SA told me.

 

[Gilliland]

Did the max speed on the screen change ? If Tesla doesn't think there is a change, its not going to slow down ...

Yes. The speed limit and max speed changed at the sign, and the car did begin to slow down (just as it did in earlier versions). But the deceleration was very gradual, and the driver could have easily received a ticket before the car reached the speed limit.

 

[powertoold]

There's a ladder on the left lane here that is rendered on the visualization as a blob after it passed it.

 

[Ramphex]

Just want to mention how much I’m impressed by this huge rollout.

 

[JulienW]

Just want to mention how much I’m impressed by this huge rollout.

Not sure if that is supposed to be sarcasm or true but either are incorrect. So far looks like a stereotypical rollout of a major point release. We should expect it to go a little slower since it is a full version release and not just a point release. Elon even said it "needs more polish" before going wider.

 

[JulienW]

This video is very impressive. It shows 11.3.1 handling everything, cautiously but smoothly. You can begin to imagine the robotaxi dream becoming reality. It is a night and day improvement vs a year ago in downtown Chicago:

Watch at 8 minutes at a 4 Way Stop. It hesitates and 4 cars cut in front of it.

 

[JulienW]

Don't remember seeing this "influencer" before.