FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[enemji]

I’ve had very inconsistent behavior with blinking red lights. Occasionally it will handle them appropriately. Most times it will simply stop and not proceed. I haven’t been able to figure out why.

FSDb does reliably seem to handle flashing yellow lights, so they must have some code to deal with them, unless all FSD doesn’t see the blinking at all and sees a solid yellow.

Which FSD version do you have? Mine is 11.3.6

 

[uscbucsfan]

I’ve had very inconsistent behavior with blinking red lights. Occasionally it will handle them appropriately. Most times it will simply stop and not proceed. I haven’t been able to figure out why.

FSDb does reliably seem to handle flashing yellow lights, so they must have some code to deal with them, unless all FSD doesn’t see the blinking at all and sees a solid yellow.

For me FSD Beta won't go on a yellow turn signal. I have to push the accelerator to make it go.

 

[enemji]

For me FSD Beta won't go on a yellow turn signal. I have to push the accelerator to make it go.

Flashing yellow turn signal?

 

[uscbucsfan]

Flashing yellow turn signal?

Yes. It just stops, they are everywhere in my area.

 

[EVNow]

For me FSD Beta won't go on a yellow turn signal. I have to push the accelerator to make it go.

Is this for solid yellow or flashing yellow ?

Flashing yellow turn signals are difficult. First FSD will stop, wait for a few seconds and then go forward and stand until there is a clear gap. It won't take small gaps like we might do. Also once there is a long gap, it will take a few seconds to get going.

Basically, if the intersection is busy I usually disengage on flashing yellow. Otherwise it will take a few more seconds but will work.

 

[SSDTester#1]

I think the Junk complains more than I do.

 

[MP3Mike]

I think the Junk complains more than I do.

View attachment 944500

I wonder if that is why you get such poor FSDb performance. Does the pointer for your vehicle normally show up in the correct spot on the nav map?

 

[Yelobird]

I think the Junk complains more than I do.

View attachment 944500

That is one competition FSD will Never win. lol

 

[SSDTester#1]

I wonder if that is why you get such poor performance. Does the pointer for your vehicle normally show up in the correct spot on the nav map?

Yea it’s in the right spot and follows along appropriately. I have not seen this message before 11.4.x

 

[Dewg]

First drive down to SD today on 11.4.2. I really like what they've done with cut-ins and cut-outs. Car didn't overreact at all the entire drive. Smooth slow down for people cutting in, and maintained speed with cut-outs instead of slowing down. Also like the smoother lane shift when passing a semi. 11.3.6 felt a little abrupt when shifting in the lane to give more room for the semi.

I think people with basic AP having harsh slowdowns for cut-ins and cut-outs will be very pleased when the new single-stack makes it way down to AP.

 

[JHCCAZ]

I think the Junk complains more than I do.

View attachment 944500

Thanks for the photo.
it's so obvious now, why you have so many problems with FSD.
The roads in your area are covered in dust!

 

[fasteddie7]

Went for a drive today and suddenly it merges correctly (no longer going all the way to the merge point), stops are way smoother, and left turns are less jittery and more confident. Don’t know what happened. I didn’t get an update, it just started doing the right thing all of the sudden. It did start doing other things incorrectly in places it never did before. Oh well, you win some you lose some .

 

[JHCCAZ]

Went for a drive today and suddenly it merges correctly (no longer going all the way to the merge point), stops are way smoother, and left turns are less jittery and more confident. Don’t know what happened. I didn’t get an update, it just started doing the right thing all of the sudden. It did start doing other things incorrectly in places it never did before. Oh well, you win some you lose some

Interesting, because all of a sudden today my car started doing wrong things it used to do back in March and April. Lane selection errors that got fixed in late April while on 11.3.6, today reverted to that kind of behavior along with a couple of new foibles.

Something to do with the real-tine map hints database I believe. I've been pretty enthusiastic about the improvements from that, but today it went the other direction. As you said, win some and lose some!

I am now on the latest NA-2022.44-14515 map, but I don't actually know when I got it - maybe yesterday? (I subscribed to TeslaFi recently, but I can't find any way to track map updates for my own vehicle, even though there is a section in the fleet software tracker that deals with map updates.)

 

[aronth5]

Yes, same experience (though never encountered a true flashing red as I said). I have only seen it proceed on solid red (as documented on video posted - been a few months), just a regular red, normal, unconfusing intersection, but second in a series (location provided previously).

But flashing red (disabled traffic light) - nothing. Always stopped. And remained stopped.

This used to be my behavior but with V11 FSD treats the flashing red light as a Stop sign and works fine.
There is also an actual Stop sign but FSD used to just sit forever pre V11 so that doesn't seem to matter.

Now flashing yellow lights are another story. No consistency. Sometime it works well other times it's a mess.

 

[sleepydoc]

Interesting, because all of a sudden today my car started doing wrong things it used to do back in March and April. Lane selection errors that got fixed in late April while on 11.3.6, today reverted to that kind of behavior along with a couple of new foibles.

Something to do with the real-tine map hints database I believe. I've been pretty enthusiastic about the improvements from that, but today it went the other direction. As you said, win some and lose some!

I am now on the latest NA-2022.44-14515 map, but I don't actually know when I got it - maybe yesterday? (I subscribed to TeslaFi recently, but I can't find any way to track map updates for my own vehicle, even though there is a section in the fleet software tracker that deals with map updates.)

I noticed the same thing this afternoon. wild lane selection errors in places that had been perfect for the last several releases.

 

[D Good]

I noticed the same thing this afternoon. wild lane selection errors in places that had been perfect for the last several releases.

THIS! I have new lane selection errors, old, resurfaced lane selection errors; 11.4.2 is a complete lane selection mess. This is the worst update in terms of lane selection since 10.2. Wow that was a long time ago, and I can’t really say my car has fixed any lane selection errors since then. Not one. In fact, they have gotten worse. The arrows on the road are clearly visible (sometimes) and the car just completely ignores them. Why can’t they write literally one line of hard code “if you see a right turn arrow and you want to go straight, then get out of that lane.” Why why why? It’s so frustrating considering how capable the car is in other areas. Makes it practically useless unless you want to constantly intervene with the turn signal.
P.S. why can the car not “remember” when it’s wrong and upload the correct lane topology to the mothership? Seems simple enough.

 

[kabin]

Gotta agree with the V11.4.2 lane selection hot mess.

I left home this morning and within 2 block had 3 disengagements and by 2 miles it was a half dozen. #FSD WINNING!

 

[enemji]

My 11.3.6 FSD is getting better everyday. It has learned to plan my routes better based upon my corrections applied during the last 2 weeks on those routes.

 

[LSDTester#1]

I think the Junk complains more than I do.

View attachment 944500

I had that flag a while ago. I had a wire in the harness in the rear view mirror housing have some of its sheathing come up and expose the wire. Mobile identified it but had to make a service appointment as he didn’t have the part. Took it to svc and they fixed it up. Granted they had it for a week but they gave me a loaner MY. It was during that week driving the MY I realized that no matter how much I complained about FSD, I actually missed it.

edit: he said the exposed wire was causing interference

 

[JHCCAZ]

THIS! I have new lane selection errors, old, resurfaced lane selection errors; 11.4.2 is a complete lane selection mess. This is the worst update in terms of lane selection since 10.2. Wow that was a long time ago, and I can’t really say my car has fixed any lane selection errors since then. Not one. In fact, they have gotten worse. The arrows on the road are clearly visible (sometimes) and the car just completely ignores them. Why can’t they write literally one line of hard code “if you see a right turn arrow and you want to go straight, then get out of that lane.” Why why why? It’s so frustrating considering how capable the car is in other areas. Makes it practically useless unless you want to constantly intervene with the turn signal.
P.S. why can the car not “remember” when it’s wrong and upload the correct lane topology to the mothership? Seems simple enough.

I hear you, but are you saying that 11.4.2 misbehaved immediately for you? For me, it was initially somewhat better than 11.3.6, which itself had improved noticeably in late April. Suddenly today, several of those improvements were lost and I'm approximately back to the mid-April collection of errors.

Hopefully these regressions will get repaired shortly.