FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Dennisis]

On Assertive, right hand lane at 0.5-0.6 miles. (No idea if mode dependent.)

True, I was on assertive with same behavior, I’ll switch and see if it changes, that would be nice.

 

[tmoz]

California is the only state in the union where lane splitting is allowed. Hopefully not much longer before that unsafe maneuver is abolished here too. They probably don't have it as a high priority to train on this data as the ODD is small.

I just learned that lane splitting and lane filtering are two different things. I knew that motorcycles could drive on the white stripe dividing lanes in Arizona under certain circumstances, but didn't know that moving faster than the traffic in the other lanes was one term and traffic moving faster in the other lanes was another term.

Lane Splitting & Filtering Are Legal in the Following States

Lane splitting can be a beneficial practice, but it’s still illegal in most states. Read on to discover the states where lane splitting is legal.

missionlegalcenter.com

 

[JHCCAZ]

Yes, your car in fsdb, rolling out into the path of a moving semi

Make my dream come true

My mother-in-law had raised her son and daughter on a tight budget and could not afford life insurance. But she had decent auto Insurance.

So she told the kids that in case she dropped dead or became incapacitated, they were to load her into the driver's seat of her car and let it roll down the street, into the main cross street at rush hour.

 

[TK211X]

My mother-in-law had raised her son and daughter on a tight budget and could not afford life insurance. But she had decent auto Insurance.

So she told the kids that in case she dropped dead or became incapacitated, they were to load her into the driver's seat of her car and let it roll down the street, into the main cross street at rush hour.

If you want a new car, just back yours into the side of a Walmart. Don’t forget the rear plate at the scene.

 

[aronth5]

Also because it waits until the end of the merge lane to merge, even when inappropriate, does not signal, and pulls into inappropriately small gaps (as you outlined). In California it is fine to use all the merge lane if no one is there but earlier opportunities should be safely used when traffic is around, to maintain as long a following distance as possible given the situation.

I don’t have much doubt that it is tracking other vehicles and will merge in behind them, but the merge is totally broken.

Signaling is most definitely required even if it is clear your lane is ending. (Apparently maybe not - if no one is around…but that would be negligent and irresponsible for obvious reasons. Know the Road with the CHP: When a lane ends and you're merging, do you have to signal? | abc30.com )

Your example is just one demonstration of the consequences.

My problem is when highway traffic is going 20-40mph during rush hour. Merging is terrible. As each car in front of me successfully merges my car speeds up since there is now space available. And after other cars merge it continues to go far too fast until it runs out of merge lane. Very predictable.

On the highway except for merging problems I haven't disengaged for almost 600 miles. Granted it's not urban highways which add a lot of complexity but that is still pretty darn good.

 

[FSDtester#1]

I just had a 4 mile drive, no interventions, disengagements... but the weirdest part.
Not 1 nag... made 3 turns, suburban driving, city streets
Did they make a nag adjustment...

I took 2 more drives after this. It did not repeat.
It must of been a glitch. Nags are back as usual.

 

[clydeiii]

Example of how 11.3.3 isn’t nearly aggressive enough when taking exits on the highway

 

[aronth5]

TeslaFi shows some people getting the map update NA-2022-44-14515. I still have NA-2022-28-14046.
Does anyone know if/how the new maps have affected FSD?

TeslaFi.com Firmware Tracker

 

[clydeiii]

TeslaFi shows some people getting the map update NA-2022-44-14515. I still have NA-2022-28-14046.
Does anyone know if/how the new maps have affected FSD?

Maps absolutely affect how Beta performs

 

[TK211X]

This is odd, according to trend data there should be a new FW Ver. like now.

 

[aronth5]

Maps absolutely affect how Beta performs

That is my assumption but people have often reported prior map updates had little or no affect on FSD. That was my experience with the last map update. So the question is what about this new map update?

 

[clydeiii]

That is my assumption but people have often reported prior map updates had little or no affect on FSD. That was my experience with the last map update. So the question is what about this new map update?

That could because their local maps didn’t get any corrections. Here in Baltimore the car randomly stops at intersections that have no stop signs, and I assume it’s because old out of date map data is convincing the car it needs to stop even if it can’t see a stop sign or light. Will this update fix those issues? Unclear.

 

[Dennisis]

I took 2 more drives after this. It did not repeat.
It must of been a glitch. Nags are back as usual.

To clarify, what do you think triggers the nags? Not holding the wheel? Not looking at the road ahead?

 

[NathanielHrnblwr]

I think I'm the last person waiting to get v11.

 

[clydeiii]

To clarify, what do you think triggers the nags? Not holding the wheel? Not looking at the road ahead?

In my experience holding a phone but looking at the road causes nags.

 

[Ramphex]

In my experience holding a phone but looking at the road causes nags.

In my experience holding the phone to record the dash on model S gives you strikes

 

[FSDtester#1]

To clarify, what do you think triggers the nags? Not holding the wheel? Not looking at the road ahead?

Can't say what triggers them, but the 1 trip I got no nags, I was thinking, how come no nags? Then I took a mental snapshot. My left hand was barely on the wheel, bottom left. My right elbow was resting on the armrest. I also was not moving around at all, almost like a statue, slightly leaning towards the armrest, but definitely more centered than usual. My eyes were directly forward and I never touched my phone or touchscreen. Of course this could be anecdotal. I will attempt the exact same route in the morning doing the exact same things and report back.

 

[Konicom04]

done some substantial Chicago land highway driving over the last few days. As you can imagine there's construction literally everywhere you turn .
So - I insisted on testing the advance AP after recent update ( went from 2023.2.12 to 2023.6.9, so not FSDb )
Very impressed.
I am not sure if that update did anything to autopilot but:
* the wipers appear to be fixed, and not running crazy like before
* braking for traffic ahead seems smoother and more human like
* flawless line following even through intricate line shifts in construction zones,

Pretty happy.
Road-trip next weekend so testing will continue

 

[Dennisis]

Can't say what triggers them, but the 1 trip I got no nags, I was thinking, how come no nags? Then I took a mental snapshot. My left hand was barely on the wheel, bottom left. My right elbow was resting on the armrest. I also was not moving around at all, almost like a statue, slightly leaning towards the armrest, but definitely more centered than usual. My eyes were directly forward and I never touched my phone or touchscreen. Of course this could be anecdotal. I will attempt the exact same route in the morning doing the exact same things and report back.

Just curious as some complain about nags. Only thing I found to get a nag was looking down at the screen for 10 secs plus without glancing up or not holding the wheel for a time, didn’t actually time that. Otherwise no issues.

 

[Supcom]

Followed up this afternoon's generally good ride with a couple crummy ones this evening. The car kept missing required lane changes on city streets.

The first was a right turn that required two lane changes to the left to enter a turnpike. The car made one lane change, then delayed too long before attempting the second. No cars around to blame. Just poor planning. This is a maneuver that was poor on 10.x, but executed correctly the first three times on 11.3.4. Had to take over to quickly make the lane change.

Leaving a restaurant parking lot. the car needed to make a simple right turn into the right lane so it could quickly get into a right turn lane. The car must have had a few beers while I was eating, as it turned right into the left lane (of two) then swung over to the right lane, but not in time to get into the right turn lane, so had to continue straight and reroute. Once again, no cars nearby, so no danger and I let the car continue without intervention.

Car turned right onto the turnpike frontage road. Simple required maneuver to change two lanes to the left to hit the onramp. Lots of room available. Car swerved around like it was drunk (maybe it had more than a few beers earlier?) then proceeded to dance with another car in the left lane and couldn't get make a space to move over. It was one of those cases where both cars slow down. The other car dropped back enough that a human would hit the accelerator and get over. But FSDb wouldn't do it, so it missed the onramp and had to continue down through the next intersection to get on the turnpike. No intervention, but I'll give 25% of the blame to the other driver, who was well meaning, but didn't help.

Then, on a surface street almost home, the car decides to slow down for a green light. Really?? A little accelerator solved the problem, but It's been through that intersection countless times without issue - until tonight.

Once again, driving on the turnpike was without issue and worked well. But the city streets needs more than a little polish.

I'm now awaiting 11.3.6.