FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Mardak]

- Added new Vision Speed network to infer the typical driving speed on a given road. This is used to limit the maximum allowed speed in environments such as parking lots and residential roads.​

​

Seems like some progress towards Smart Summon / Auto Park with this entry. I believe this is the first time any FSD Beta release notes have mentioned "parking lots."

 

[AlanSubie4Life]

- Added new Vision Speed network to infer the typical driving speed on a given road. This is used to limit the maximum allowed speed in environments such as parking lots and residential roads.​

​

Seems like some progress towards Smart Summon / Auto Park with this entry. I believe this is the first time any FSD Beta release notes have mentioned "parking lots."

You’d think for ASS they’d just limit max speed to 10-15mph or something. Presumably it won’t support surface street diversions.

To me this seems more like a safety improvement to prevent surging into parking lots at 20mph (where FSDb can’t be used at all, but I guess people try?), and to better predict appropriate speeds in residential neighborhoods when unmarked (often too slow right now so I wonder if this will lead to max speed increases in some cases).

 

[Mardak]

Your crystal ball on this turned out to be amazingly accurate. How did you know that 2023.6.15 was going to be the build that merged FSD with the 2023 branch?

Ha thanks but not perfectly accurate. It was clear Tesla would be updating to a 2023.x version, and to get the broadest reach, it should fork from the latest version 2023.6.x. I thought it could be FSD Beta 11.3.6 before that was released, but even better we'll be getting 11.4 with new FSD Beta functionality.

As for the software version number 2023.6.15, looking at previous releases, FSD Beta moving to a newer software branch has been numbered as a minor version multiple of 5 greater than the previous ones used. This is so that updates "move forward" by including/maintaining any earlier production software fixes when newly being added to FSD Beta.

 

[sandman1330]

Mitigated hydroplaning risk by making maximum allowable speed in Autopilot proportional to the severity of the detected road conditions. In extreme cases, Autopilot may use the wetness of the road, tire spray from other vehicles, rain intensity, tire wear estimation or other risk factors that indicate the vehicle is near the handling limit of the surface to warn the driver and reduce speed.

This actually worries me. I'm already fussed when TACC throws the "reducing speed due to poor visibility" considering how often it does so on a false positive.

Now I see it engaging this "safety" slowdown when it "thinks" it's necessary. But like phantom braking and TACC reduced speed, I suspect it will start doing this more frequently on false positives than when actually necessary....

Edit: I'm OK with it doing this on Autopilot for the safety aspects, but leave my cruise control alone pls....

 

[JHCCAZ]

Excellent! Looks like they're quickly addressing key issues - turns, lane changes, and bus lanes. Let's hope there are significant improvements.

Yes, bus lane recognition Improvement is mentioned twice in these notes. Sometimes I turn from southbound Craycroft onto westbound Grant, it wants to turn into the bus lane every time.

Also a lot of encouraging notes about lane competency.

I'll test it as soon as I get this Yuge New Update!

 

[Dennisis]

Sometimes I turn from southbound Craycroft onto westbound Grant, it wants to turn into the bus lane every time.

Funny, that's the worst place for me too, almost hit a bus sitting there twice - at least it's repeatable!

 

[Mr_Joe]

https://twitter.com/x/status/1647976417270054914

Huge update! I cant wait to be able to enroll myself. I'm on 2023.6.9. Hopefully I can flick the switch soon - already subbed to FSD.

 

[YNoVA]

I don't have FSD but am on 2023.6.11..

Can I subscribe to FSD and get 11.4 since it's on 2023.6.15?

 

[johnmodely]

FSD Beta on highway where there are super curvy roads and you are on the left lane seems like beta has a hard time sticking to center of lane, it keeps hiding the yellow line which in this highway is super close to the concrete divider/rail...I have to disengage on almost every curve in the highway in this case...seems like a major regression from pre FSD beta highway.

Anyone else experiencing similar?

 

[Joboo7777]

this might be a-little premature, but I’m going to ask anyway…

Is there a V12 in the pipeline that we can start complaining about?

 

[kabin]

this might be a-little premature, but I’m going to ask anyway…

Is there a V12 in the pipeline that we can start complaining about?

No worries. I hear V12 will transition to the latest version of Fortran for a 25% improved FSD arcade-feel over V11. It's gonna be the bomb!

 

[Tronguy]

FSD Beta on highway where there are super curvy roads and you are on the left lane seems like beta has a hard time sticking to center of lane, it keeps hiding the yellow line which in this highway is super close to the concrete divider/rail...I have to disengage on almost every curve in the highway in this case...seems like a major regression from pre FSD beta highway.

Anyone else experiencing similar?

Well, was chugging down the Merritt Parkway in CT heading southbound. For those who don't know, this relatively ancient freeway predates Interstates and has some foibles. First, no trucks - because there's Too Darn Many Low Bridges. And it's a Parkway, I guess.

Second: Speed limits were a lot lower back in the day. So, there's places near the CT-NY border where the road nominal speed limit drops down to 45 mph. That doesn't stop everybody from going a lot faster than that, say, 65, but, when doing that, there's some serious G forces going around the bend.

And, yeah, 11.3.4 definitely drifts towards the left line on those right turns. I did disengage a couple of times through there.

Seems that when they put in that function, "lean to the left side when passing a Semi", it also seems that they made "staying in the center" not as strong on curves.

 

[powertoold]

3.4 biases a lot within lanes, but it's also more aware of and changes behavior for cars in the way or oncoming

 

[Dewg]

Edit: I'm OK with it doing this on Autopilot for the safety aspects, but leave my cruise control alone pls....

TACC stands for "Traffic Aware Cruise Control". It's adjusting itself based on traffic around you. The most common application of this is when driving significantly faster than the lane next to you, the system will slow down.

 

[JHCCAZ]

I don't have FSD but am on 2023.6.11..

Can I subscribe to FSD and get 11.4 since it's on 2023.6.15?

Yes but you might want to time when the spending clock starts. See my comments to @snovon exactly this topic:

Post in thread '2022.40.4.1 am I screwed out of FSD beta?' 2022.40.4.1 am I screwed out of FSD beta?

 

[Supcom]

Took a long drive today with 11.3.6 from Dallas toward Wichita Falls, TX on I-35E and US287. Last summer, on ancient NOA software, I had numerous PB events as well as software crashes on US287, so I've been wanting to drive this on a warm sunny day with the newest (until 11.4.x shows up!) single-stack to see if there is any improvement.

I am pleased to report that I had NO PB events driving all the way to Henrietta Supercharger and back. And, I had NO software crashes. Previously, NOA would crash at the crest of some of the rolling hills. But, not today. Very nice ride. FSDb even handled the treacherous I-35E traffic like it was purpose-built for it.

Sadly, it wasn't without a few minor issues. Most notably, as reported earlier, FSDb wanted to slow way down for a couple flashing yellow lights strung above the road. And, when I went to exit at Henrietta, a tailgating pickup moved into the exit lane before me. FSDb was too timid to change lanes in front of it, so I had to take over to make the exit.

Still, MUCH improved over last summer's trip!

 

[JHCCAZ]

when I went to exit at Henrietta, a tailgating pickup moved into the exit lane before me. FSDb was too timid to change lanes in front of it, so I had to take over to make the exit.

Even on city streets, FSD is "good" about waiting for the actual turn ane opening, sometimes with a dashed line demarcating the entrance. It won't cross the solid line or gore.

But "good" is ultimately not good enough. The problem is that (in my experience) it also waits until that point to activate the turn signal, and it doesn't bias over to the right ahead of the exit/turn. Its external cues are indistinguishable from a car that intends to go straight past the exit/turn, until the first legal opportunity. Of course, drivers behind shouldn't jump around to the right and create a problem in case the car they're following does exit - but that's pretty much to be expected as we all know.

I'm fairly sure that earlier turn signaling (without ever false signaling!) is the proper, definable and legal approach and should be coming up in FSDb hopefully this year. More subtle and territory-claiming cues to other drivers - in this case, proactively shifting to the right while the tires stay in the legal lane, will really be needed.

 

[Supcom]

Even on city streets, FSD is "good" about waiting for the actual turn ane opening, sometimes with a dashed line demarcating the entrance. It won't cross the solid line or gore.

But the problem is that (in my experience) it also waits until this point to activate the turn signal, and it doesn't bias over to the right ahead of the exit/turn. Its external cues are indistinguishable from a car that intends to go straight past the exit/turn until the first legal opportunity. Of course, drivers behind shouldn't jump around to the right and create a problem in case the car they're following does exit - but that's pretty much to be expected as we all know.

I'm pretty sure that earlier turn signaling (without ever false signaling!) Is the proper, definable and legal approach and should be coming up in FSDb hopefully this year. More subtle and territory-claiming cues to other drivers - in this case, proactively shifting to the right while the tires stay in the legal lane, will really be needed.

I don't know the exact timing of the turn signal, but the other car was very anxious to get into the turn lane. The driver didn't try to overtake me, but sat inside FSDb's 'personal space', so it wouldn't move over. No human driver would have thought twice about getting into the lane in this scenario, but FSDb is sometimes just too cautious. It really surprised me, as it was a simple exit and I expected no issue with it. In the end, there was no safety issue. Had I not intervened, FSDb would have just continued down the highway and rerouted. But, I didn't want to deal with that, so I took over.

 

[JHCCAZ]

I don't know the exact timing of the turn signal, but the other car was very anxious to get into the turn lane. The driver didn't try to overtake me, but sat inside FSDb's 'personal space', so it wouldn't move over. No human driver would have thought twice about getting into the lane in this scenario, but FSDb is sometimes just too cautious. It really surprised me, as it was a simple exit and I expected no issue with it. In the end, there was no safety issue. Had I not intervened, FSDb would have just continued down the highway and rerouted. But, I didn't want to deal with that, so I took over.

What you said reminds me also, that FSD beta is bad about riding in or near the blind spot or at least in the "personal space" of cars in the adjacent lane. This definitely needs to be improved as well.

And, I almost had an accident a couple of weeks back because FSD beta doesn't slow enough when passing a line of backed-up cars in the adjacent lane. They've added chevrons indicating differential speed in the visualization, but FSDb dosn't sufficiently react in its driving policy.

What happened was that the leftmost and middle lanes were backed up, but drivers had helpfully allowed space for a car turning left from the opposite direction. That car took the invitation to cross between the stopped cars, but my FSD car was barreling too fast in the sparsely occupied right lane, so both FSD and myself were blind to the dangerous crossing car until it darted across my lane. I got the emergency train of beeps as I was smashing the brake pedal, and my heart rate synced with the beep rate. Not good.

 

[MP3Mike]

What happened was that the leftmost and middle lanes were backed up, but drivers had helpfully allowed space for a car turning left from the opposite direction.

Personally I won't use those gaps unless I can see all of the lanes of travel I will be crossing, as I have seen too many collisions from people doing that while someone comes barreling down the outside/turn lane.