FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[EVNow]

Hopefully this early stop and creep behavior will get corrected soon, very annoying. Should be able to stop, look, and smoothly proceed - can shave 3-5 seconds off each turn easily.

Actually in this case - shouldn't stop at all. It is a free right turn, its own lane & the light was green. And clear visibility too ...

 

[SSDTester#1]

Looks like nothing is rolling out this weekend, eh?

 

[FSDtester#1]

Looks like nothing is rolling out this weekend, eh?

Yes, your car in fsdb, rolling out into the path of a moving semi

 

[SSDTester#1]

Yes, your car in fsdb, rolling out into the path of a moving semi

Make my dream come true

 

[FSDtester#1]

Make my dream come true

You better have gap insurance if you financed. They dropped prices again last night , highest on X and S, added AWD MY to configurator, and price cuts , smaller ones, on Y and 3

 

[SSDTester#1]

You better have gap insurance if you financed. They dropped prices again last night , highest on X and S, added AWD MY to configurator, and price cuts , smaller ones, on Y and 3

I’m an OG, ordered at $79,999. About the only thing I can brag about being a Tesla OG for something. Rest I got screwed on, tilting screen, Steam, headlights, taillights, etc.

 

[FSDtester#1]

I’m an OG, ordered at $79,999. About the only thing I can brag about being a Tesla OG for something. Rest I got screwed on, tilting screen, Steam, headlights, taillights, etc.

Shoot, if they get that price on X, it would qualify for the $7500 credit.

 

[AlanSubie4Life]

11.3.4

Freeways:

Cannot do merges as discussed

• No signal in non-optional merges.
• Merges too close behind.
• Doesn’t merge

Cannot handle cut-ins. Even mild cut ins result in excess slowdown in most cases.
Does not seem to be able to predict cut-ins based on signaling.

 

[TK211X]

11.3.5

Fails to route plan far enough into the future causing it to stutter and make last second corrections. To add to this, it will randomly not follow the planned route and sometimes get itself stuck in dead ends such as parking lots that just end.

Will creep inappropriately and sometimes put the vehicle into dangerous situations then stop forcing a driver takeover.

Will stop randomly at intersections where there is no stop sign and impeding the flow of traffic behind you making people livid.

DM is very sensitive.

Speed limit usually set to 25mph in parking lots causing higher speed not adequate to brake in time for children that pop out or unknowing pedestrians.

Cannot detect road hazards such as pot holes or road debris the size of a dead dog. Ask me how I know.

Phantom braking on residential streets is a thing for some reason but have not experienced it on highways oddly enough.

Sometimes loves to stop in an intersection in demonstrations of authority or something.

Edit: I’m still upset I got a strike when my passenger was waving their phone around as I was looking at the road and diligently shaking the wheel trying to get it to dismiss.

 

[SSDTester#1]

11.3.4

Freeways:

Cannot do merges as discussed

• No signal in non-optional merges.
• Merges too close behind.
• Doesn’t merge

Cannot handle cut-ins. Even mild cut ins result in excess slowdown in most cases.
Does not seem to be able to predict cut-ins based on signaling.

Sounds like it’s performing as designed.

 

[zoomer0056]

Cannot detect road hazards such as pot holes or road debris

Is FSD expected to see pot holes? I wonder how big a sink hole has to be for detection.

 

[JulienW]

Is FSD expected to see pot holes? I wonder how big a sink hole has to be for detection.

This big.

 

[powertoold]

3.4 does that thing again where if it makes a wrong decision / judgment, then it will spaz out and be much more likely to make another mistake for the next 5 seconds.

For example, if it starts to bias to the right to make a right turn and then for some reason it changes its mind about biasing and starts to swerve back into the middle of the lane, it will very likely overshoot the right turn (by doing something stupid like reaccelerating rather than continue decelerating for the turn).

 

[zoomer0056]

This big.

Ha, portal to another universe big!

 

[FSDtester#1]

I just had a 4 mile drive, no interventions, disengagements... but the weirdest part.
Not 1 nag... made 3 turns, suburban driving, city streets
Did they make a nag adjustment...

 

[Supcom]

I let the car drive on I-30 West toward Downtown Dallas today. Both good and bad results:

Bad - The car had difficulty getting onto the interstate on the way back. The onramp was the type with a combination onramp/offramp that enters the interstate at an overpass with a short merge area before the lane continues as an exit ramp on the far side of the overpass. The car didn't attempt to merge until far too late, if at all. I had to disengage to complete the maneuver by hand. If I had let the car go, it would have exited back onto the frontage road and make another attempt at the next onramp. This is the third time I have had issues with this type of onramp. NOA was able to do these acceptably, so this is a definite regression.

Good - There is eternal construction at the I-30/I-635 interchange that ahs I-30 lanes shift left and right multiple times. NOA had trouble staying in the narrow lanes as they swerve, apparently due to processing lags and I would always drive the stretch by hand. FSDb v11.3.4 handled all of the construction flawlessly. It even handled the fast lane with semi trucks on the right and a construction barrier placed just inches outside the left lane line.

Stressful - About 2 miles from my exit downtown, the car was in the leftmost lane with heavy traffic with some stop and go places. I was worried that it would never get over the five lanes needed to make the exit. But, amazingly, FSDb managed to find enough gaps and got into the correct lane just in time. Some of the lane changes were from dead stop with traffic approaching, but FSDb picked good gaps and used suitable acceleration to safely switch lanes. Still, I wish the car would start moving over a little sooner!

 

[Dennisis]

Stressful - About 2 miles from my exit downtown, the car was in the leftmost lane with heavy traffic with some stop and go places. I was worried that it would never get over the five lanes needed to make the exit. But, amazingly, FSDb managed to find enough gaps and got into the correct lane just in time. Some of the lane changes were from dead stop with traffic approaching, but FSDb picked good gaps and used suitable acceleration to safely switch lanes. Still, I wish the car would start moving over a little sooner!

In my limited testing (3 lanes in one direction) it seemed to move over from the left lane around 1 mile before an exit and then to the righthand lane half a mile before the exit - seemed pretty consistent. I agree, in heavy traffic I'd double both those distances. My car made the moves nicely but with too little room for unforeseen contingencies IMO.

 

[AlanSubie4Life]

Still, I wish the car would start moving over a little sooner!

Yeah I am pretty impressed with how quickly and competently it moves over - sometimes.

I wouldn’t worry about it moving over sooner at all - if I were confident it could reliably do it even in challenging situations.

It’s the sort of thing where trust has to be developed and not sure we are there yet. I don’t think it can reliably handle tough situations based on some of the hesitation (meaning failure) I have seen. On the other hand, it has also done some great - legitimately impressive - lane changes.

 

[Supcom]

In my limited testing (3 lanes in one direction) it seemed to move over from the left lane around 1 mile before an exit and then to the righthand lane half a mile before the exit - seemed pretty consistent. I agree, in heavy traffic I'd double both those distances. My car made the moves nicely but with too little room for unforeseen contingencies IMO.

I have generally not seen that behavior. Usually, the car starts to move toward the exit side of the highway about 2 miles before the exit. With 11.3.4, it seems to wait a bit later, but I have not had the car go in the opposite direction unless there was significantly slower traffic it wanted to pass. 10.x was worked well for me on multi lane city streets as well moving over in advance of an upcoming turn.

 

[AlanSubie4Life]

Usually, the car starts to move toward the exit side of the highway about 2 miles before the exit.

On Assertive, right hand lane at 0.5-0.6 miles. (No idea if mode dependent.)