FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Hiline]

Yeah if the move is unexpected it is not favorable. It is not like the car is going to notice something you did not! At least, not on any timescale that would be relevant.

Just disengage. Always be disengaging, whenever the car deviates at all from the path or manner of driving that you have planned.

Well I assumed the car noticed something I didn't. I could only look one way at a time, but the car had multiple cameras so could look in multiple directions at once. Maybe the car noticed that the left lane was clear so it moved over, while I wasn't paying attention to the left lane. Just that, in this case, this observation was wrong.

Do you always check whether a lane change is warranted when the turn signal gets turned on?

 

[AlanSubie4Life]

Do you always check whether a lane change is warranted when the turn signal gets turned on?

Yes! I try to anticipate when the car will change lanes, and I’m always keeping an eye on which lanes are clear and what traffic is doing, just in case I have to make an emergency lane change, and just for general situational awareness.

Also I routinely request lane changes when the car isn’t taking advantage of appropriate opportunities, or will be in the wrong lane (for myriad reasons).

I often cancel lane changes the car tries to make, without disengaging. For example, it will persistently try to get over to the right lane because it is going to make a right turn in a mile or so. I will make sure it can’t get over (repeatedly declining lane changes) until it makes sense to me (right lane is often a terribly hazardous place to be). Then I often request the required changes myself (the car is incredibly slow at deciding to make lane changes even with obvious gaps, so usually this is necessary when things are on a tight schedule).

It requires anticipating, planning and executing the plan! A fair amount of work (especially on busy roads with three lanes like Mira Mesa, where optimal lane choice is crucial), keeping the mind fully engaged in the task of driving the car. Kind of like…driving a car!

I could only look one way at a time, but the car had multiple cameras so could look in multiple directions at once.

Remember that you can effectively see a LOT more than the car can, most of the time. You can anticipate, and construct a map.

Occasionally the car might notice something before you do, but this is not the norm - and typically would involve a sudden movement of a vehicle in your “blind” spot (typically a normal driver avoids driving in this position for a significant period of time, but unfortunately FSD does not). As another example, it’s routine to notice lane-splitting motorcycles WAY before the car does.

 

[AlanSubie4Life]

You're looking forward. The turn signal goes on. The car starts moving over. Then you do what?

If you don’t know the situation in the target lane for some reason, cancel the signal immediately when it goes on. (This is the point where I usually turn on Minimal Lane Changes AGAIN, .)

The wheel will usually vibrate just as the lane change begins, so if it gets to that point, and canceling does not work (it should), just disengage.

If the car starts to change lanes in spite of above two opportunities, just disengage with steering input (making sure no one is planning to occupy the space you signaled you were leaving, and evading them if necessary!).

As long as you are always driving, not the car, it will be fine.

 

[Pdubs]

Yeah if the move is unexpected it is not favorable. It is not like the car is going to notice something you did not! At least, not on any timescale that would be relevant.

Just disengage. Always be disengaging, whenever the car deviates at all from the path or manner of driving that you have planned.

ABD

 

[sleepydoc]

Well I assumed the car noticed something I didn't. I could only look one way at a time, but the car had multiple cameras so could look in multiple directions at once. Maybe the car noticed that the left lane was clear so it moved over, while I wasn't paying attention to the left lane. Just that, in this case, this observation was wrong.

Do you always check whether a lane change is warranted when the turn signal gets turned on?

Typically FSD gives more notice when it decides to change lanes. Typically, but not always, and I’ve had it do similar things with very short notice.

You always need to be vigilant - this includes knowing where cars are around you, if for no other reason than to take evasive action when some idiot Tesla using FSD cuts you off!

The more you use FSD, the more you get to know how it behaves, what its shortcomings are and where it’s likely to flake out but even with a couple years of experience I still get surprised.

 

[sleepydoc]

If you don’t know the situation in the target lane for some reason, cancel the signal immediately when it goes on. (This is the point where I usually turn on Minimal Lane Changes AGAIN, .)

The wheel will usually vibrate just as the lane change begins, so if it gets to that point, and canceling does not work (it should), just disengage.

If the car starts to change lanes in spite of above two opportunities, just disengage with steering input (making sure no one is planning to occupy the space you signaled you were leaving, and evading them if necessary!).

As long as you are always driving, not the car, it will be fine.

I routinely tap the turn signal to cancel the lane change. Like I said above, it doesn’t always give you a lot of notice, though.

 

[Knightshade]

Typically FSD gives more notice when it decides to change lanes. Typically, but not always, and I’ve had it do similar things with very short notice.

FWIW I most notice this when a new lane appears- the car often seems to immediately dive into a new lane (even if it's the wrong lane, as it so often is because the car needs to go straight but it's a temp turn lane)

By far the most annoying, and most disengagement causing, behavior in recent times for me.

 

[Supcom]

Typically FSD gives more notice when it decides to change lanes. Typically, but not always, and I’ve had it do similar things with very short notice.

You always need to be vigilant - this includes knowing where cars are around you, if for no other reason than to take evasive action when some idiot Tesla using FSD cuts you off!

The more you use FSD, the more you get to know how it behaves, what its shortcomings are and where it’s likely to flake out but even with a couple years of experience I still get surprised.

Yes, FSD can sometimes make a quick move into another lane. It's a bit scary how quickly it will do this when it gets the notion. It's easy to be caught off-guard.

 

[Hiline]

If you don’t know the situation in the target lane for some reason, cancel the signal immediately when it goes on. (This is the point where I usually turn on Minimal Lane Changes AGAIN, .)

The wheel will usually vibrate just as the lane change begins, so if it gets to that point, and canceling does not work (it should), just disengage.

If the car starts to change lanes in spite of above two opportunities, just disengage with steering input (making sure no one is planning to occupy the space you signaled you were leaving, and evading them if necessary!).

As long as you are always driving, not the car, it will be fine.

See the problems with this approach are:
1. The driver of the car behind you may already be speeding up since they expect you to complete the lane change
2. Having to remember to do cancel the signal immediately each time you don't want a lane change is tiring - it may be safe in theory, but it's tiring
3. Physically yanking the steering wheel can lead to the car bouncing and swinging on the suspension, and this can be as dangerous as cutting another car off

 

[AlanSubie4Life]

See the problems with this approach are:
1. The driver of the car behind you may already be speeding up since they expect you to complete the lane change
2. Having to remember to do cancel the signal immediately each time you don't want a lane change is tiring - it may be safe in theory, but it's tiring
3. Physically yanking the steering wheel can lead to the car bouncing and swinging on the suspension, and this can be as dangerous as cutting another car off

Early intervention is key. When driving it is best to not be swerving around.

The remaining downside is the spurious signaling which is unavoidable.

Turn on minimal lane changes and vibrate. It’s not perfect but have to use the tools and react in a fraction of a second. Vibrate does not work for turn-lane entry.

 

[Hiline]

Early intervention is key. When driving it is best to not be swerving around.

The remaining downside is the spurious signaling which is unavoidable.

Somewhat related to this. FSD not executing a lane change as I turn, and keep, the blinker on is actually the most annoying, though mostly harmless, part of the software to me.

 

[kabin]

Maybe it's status quo but I'm still experiencing a bad case of dry wipes with the latest version of v11.

 

[sleepydoc]

Maybe it's status quo but I'm still experiencing a bad case of dry wipes with the latest version of v11.

that sounds like a personal issue.

 

[kabin]

There might be some hope. Maybe the issue is broader than FSD. 2 weeks!

 

[kabin]

that sounds like a personal issue.

Maybe a DME script for a bidet?

 

[Scott Chapman]

There might be some hope. Maybe the issue is broader than FSD. 2 weeks!


View attachment 1019906

Does not making it worse count as an improvement?

 

[AlanSubie4Life]

Somewhat related to this. FSD not executing a lane change as I turn, and keep, the blinker on is actually the most annoying, though mostly harmless, part of the software to me.

Yes it is sometimes remarkably slow, even in the absence of traffic. But not always. It’s annoying!

1) Sometime it is due to perceived conflict with other vehicles.
2) Sometimes it just doesn’t seem to get it and just have to cancel and try again.
3) Sometimes pressing the accelerator helps (usually in conjunction with situation 1 above).
4) Sometimes it is just slow for no apparent reason.

Very frustrating because you want crisp reliable lane changes on demand, as it would make it easier to use FSD as a tool, as you did the planning and execution of the drive. But it just dawdles too much and is often not good about slipping into obvious safe gaps.

 

[pimfram]

A bunch of folks on Reddit are getting 2023.44.30.20 right now.

 

[hybridbear]

A bunch of folks on Reddit are getting 2023.44.30.20 right now.

Anyone who is outside of California? The few commenters I’ve seen saying that they’re getting it are all in California.

 

[kabin]

Anyone who is outside of California? The few commenters I’ve seen saying that they’re getting it are all in California.

Yes, one TSLA X poster in Sarasota FL.