FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Supcom]

I can't remember exactly where it was but I definitely remember clicking 'ok' to a data sharing agreement.

FSD beta used to require data sharing with Tesla. However, recent software releases on both the production and development branches no longer require this. If you disable all the data sharing controls in the software panel, FSD beta remains enabled.

 

[EVNow]

I think the plan was for you to meet with @wiliamg, as you both live in Seattle. I’m down in Covington.

I'm on the Eastside. West Seattle is 24 miles, 40 minutes. Covington is 23 miles, 40 minutes

ps : So, yes I & @WilliamG can meetup in West Seattle. I can also come down to Covington.

 

[MP3Mike]

FSD beta used to require data sharing with Tesla. However, recent software releases on both the production and development branches no longer require this. If you disable all the data sharing controls in the software panel, FSD beta remains enabled.

I'm pretty sure it still requires it, just that it is agreed to separately, and they only way to stop it is to turn FSD off. (Recently, some people have complained that after a software update FSDb was disabled, and it required them to agree to data sharing to turn it back on.)

 

[FSDtester#1]

I wondered if FSDb is already checking on and considering following vehicles before braking? I have not heard of rear end collisions of moving Teslas on FSD due to PB.

I apologize if this was already posted., did anyone see Wham Baam Teslacam this Sunday?.... Beta avoided a collision by hopping 2 lanes to the left when some
a hole jumped onto the road from a parking lot. Not even I, the #1 tester could have pulled off that maneuver on manual driving...See below.

Skip to 7:25

 

[Yelobird]

Sure.

According to reports to the CA DMV Waymo has one disengagement per 17k miles driven (2022). In city driving Tesla has a disengagement ever 5-15 miles according to FSD Community tracker 2023. That's 1000-2000x.

If we only count critical* DE:s for Tesla they are at about 35 miles per DE, which would only be 500x worse than Waymo, but 1000x worse than the top reporters 2022.

So I'd say "three orders of magnitude" (1000x) is a pretty fair ballpark estimate all things considerered.

*) Avoid accident, taking red light/stop sign, wrong side of the road, unsafe action

Sources:

2022 Disengagement Report from California

It’s February again, and that’s when the California DMV releases its annual Disengagement Report, or more accurately, the data that goes with it. A so-called disengagement report is req…

thelastdriverlicenseholder.com

Home

FSD Community Tracker

www.teslafsdtracker.com

Think we assumed you were referring to a car we could actually BUY not a Wamo taxi cab that can safely traverse a 3 block circle with 4 R2D2 robots strapped to the hood.

 

[KLam]

I apologize if this was already posted., did anyone see Wham Baam Teslacam this Sunday?.... Beta avoided a collision by hopping 2 lanes to the left when some
a hole jumped onto the road from a parking lot. Not even I, the #1 tester could have pulled off that maneuver on manual driving...See below.

Skip to 7:25

Wow, that makes you want to engage FSDb all the time!

 

[Supcom]

I'm pretty sure it still requires it, just that it is agreed to separately, and they only way to stop it is to turn FSD off. (Recently, some people have complained that after a software update FSDb was disabled, and it required them to agree to data sharing to turn it back on.)

Only for safety incidents and accidents. Previously, there was no limitation.

 

[Twiglett]

I did the painful drive from Austin to Plantersville and back over the weekend.
Gotta say that (apart from the expected stupidity of 11.4.7.3, like wipers and trying to leave the road at 70mph) it actually did really well, even in pouring rain. It actually made the drive much less painful, especially compared to driving it myself.
Of note for me was that the car was visualizing objects on the display that I was having trouble seeing when in poor light or heavy rain. At one point it was raining so hard that I was doing 40mph but the car was showing road markings and other cars, even with the FSD degraded warnings.
The other plus was while driving on a series of country roads, with lots of fast, long sweeping bends and the occasional sharp corner and junctions. The car did extremely well here with no excessive slowdowns and a confident drive even with the occasional rain.
Overall, I came away much more impressed with the range of its ability.

 

[FSDtester#1]

Wow, that makes you want to engage FSDb all the time!

Like I said many times, FSDWTF is a kid to me, that I babysit whiles it drives. I keep it on 100% of the time. Unless some Ice car wants to get smoked. That is manual and fun.
If it messes up, it goes to time out. If it behaves, I give it a cookie !
For me it's safer to watch the car and expect the unexpected, and by now most of us know when it will likely fail. So that's the time I get ready to intervene and send it to timeout...The rest of the time I take it easy with a light hand on the yoke and look around for potential unsafe scenarios. How many of us has had an accident on FSDWTF since we started testing? Ok, the real question is how many times did you change your shorts when you got home ??

 

[bhakan]

Nightmare Before Christmas question: What will the Holliday update include?

1. 11.4.7.1.1.1.1.1.1/11.4.4.1.1.1.1.1.1
2. 11.5.x
3. 11.69.x
4. 11.420.x
5. 12.x (the mythical)

Maybe 12.25. Easy for Tesla to remember. @jebinc may want to add 12.25.j

 

[AlanSubie4Life]

Not even I, the #1 tester could have pulled off that maneuver on manual driving...See below.

Once again, people give humans way too little credit! Humans frequently avoid this sort of collision.

Still - it actually does seem likely that FSD avoided an accident in this case, assuming this was all FSD. It looks like there was an initial slowdown and swerve (pretty late), followed by a more extreme swerve. Hard to tell what FSD was responsible for, the second maneuver seems a bit unnatural for FSD while the first seems to be a typical somewhat late reaction.

In the end it was a dangerous maneuver (relied on no one being in the turn lane) and it would probably be better to see the traffic coming from the right (it was probably quite apparent to an attentive driver) and do an emergency stop without departing the lane.

But this was a tough one! With visibility limited, a swerve may have been the only option. Hard to tell without an in-cabin camera showing a wider view from the driver’s perspective.

I’d say this might be the first example I have seen where it is possible that FSD actually did avoid a collision with reasonably fast reaction. However, without cabin footage, we will never know.

 

[WilliamG]

Once again, people give humans way too little credit! Humans frequently avoid this sort of collision.

Still - it actually does seem likely that FSD avoided an accident in this case, assuming this was all FSD. It looks like there was an initial slowdown and swerve (pretty late), followed by a more extreme swerve. Hard to tell what FSD was responsible for, the second maneuver seems a bit unnatural for FSD while the first seems to be a typical somewhat late reaction.

In the end it was a dangerous maneuver (relied on no one being in the turn lane) and it would probably be better to see the traffic coming from the right (it was probably quite apparent to an attentive driver) and do an emergency stop without departing the lane.

But this was a tough one! With visibility limited, a swerve may have been the only option. Hard to tell without an in-cabin camera showing a wider view from the driver’s perspective.

I’d say this might be the first example I have seen where it is possible that FSD actually did avoid a collision with reasonably fast reaction. However, without cabin footage, we will never know.

I have to say, I don’t believe for a second that was all FSD. No human being is going to sit there and allow all of that to happen, for one thing.

 

[bhakan]

Once again, people give humans way too little credit! Humans frequently avoid this sort of collision.

Still - it actually does seem likely that FSD avoided an accident in this case, assuming this was all FSD. It looks like there was an initial slowdown and swerve (pretty late), followed by a more extreme swerve. Hard to tell what FSD was responsible for, the second maneuver seems a bit unnatural for FSD while the first seems to be a typical somewhat late reaction.

In the end it was a dangerous maneuver (relied on no one being in the turn lane) and it would probably be better to see the traffic coming from the right (it was probably quite apparent to an attentive driver) and do an emergency stop without departing the lane.

But this was a tough one! With visibility limited, a swerve may have been the only option. Hard to tell without an in-cabin camera showing a wider view from the driver’s perspective.

I’d say this might be the first example I have seen where it is possible that FSD actually did avoid a collision with reasonably fast reaction. However, without cabin footage, we will never know.

I would have disengaged the FSD and not sure if it was on FSD. I have seen situations where the car will move to an opening on the left lane in bumper to bumper traffic ( see my post #12802 above with a pic for second bullet) and it will say something like "moving to a faster lane." In my cases I disengage as it is an exit lane and there are construction cones and it cannot come back to the travel lane. I think that video probably needed more context.

 

[RickyFish89]

Nightmare Before Christmas question: What will the Holliday update include?

1. 11.4.7.1.1.1.1.1.1/11.4.4.1.1.1.1.1.1
2. 11.5.x
3. 11.69.x
4. 11.420.x
5. 12.x (the mythical)

Goal will be 12.x on main branch, including the updated visuals for all

Reality will be 11.4.20

 

[AlanSubie4Life]

I have to say, I don’t believe for a second that was all FSD. No human being is going to sit there and allow all of that to happen, for one thing.

Yes, as I said we’ll never know.

I’ve never seen verifiable footage of FSD even coming close to avoiding an accident, other than trivial easy cases extremely easily avoided by an attentive driver. (E.g. someone turns on their signal and then moves into your lane, with plenty of room to spare. This typically results in FSD slamming the brakes “avoiding” contact, while a human would just ease off smoothly as soon as the signal appears.)

 

[jebinc]

I'm on the Eastside. West Seattle is 24 miles, 40 minutes. Covington is 23 miles, 40 minutes

ps : So, yes I & @WilliamG can meetup in West Seattle. I can also come down to Covington.

For some reason, I thought you were in Seattle!

 

[jebinc]

I apologize if this was already posted., did anyone see Wham Baam Teslacam this Sunday?.... Beta avoided a collision by hopping 2 lanes to the left when some
a hole jumped onto the road from a parking lot. Not even I, the #1 tester could have pulled off that maneuver on manual driving...See below.

Skip to 7:25

Nice.

 

[kabin]

I have to say, I don’t believe for a second that was all FSD. No human being is going to sit there and allow all of that to happen, for one thing.

That was my impression too. Plus there's too much going on in that intersection for FSD to instantly slice though a traffic break (FSD path) with human like precise steering. FSD on my car has never worked anything like that. Most likely FSD alerted the driver to take over well after the danger passed.

 

[WilliamG]

That was my impression too. Plus there's too much going on in that intersection for FSD to instantly slice though a traffic break (FSD path) with human like precise steering. FSD on my car has never worked anything like that. Most likely FSD alerted the driver to take over well after the danger passed.

Yup. 100%.

 

[AlanSubie4Life]

for FSD to instantly slice though a traffic break (FSD path) with human like precise steering.

Plus it was very unsafe and completely unnecessary to avoid the collision. Looks like pure human panic to me (albeit capably avoiding running directly into any stationary objects) - for sure the second movement, anyway.