FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Ramphex]

Do you use FSD in both cars?

Model Y is non-FSD but I use AP in it. Only when driving alone though, the woman hates it as much as she hates FSDj and she never uses it when she’s driving (Model Y is her car)

 

[jebinc]

Don't believe that is possible since all HW4 cars are on 23 software (unless I'm mistaken). Never know for sure but if history is a guide it will likly be at least several months before the software is ready for HW4. Unfortunately Tesla is notoriously bad at releasing or removing hardware LONG before the software is ready. Tesla is soooooo aspirational on their ability to provided software that they never let it stop them from making hardware changes.

Has HE4 made it to the MY yet?

 

[jebinc]

Model Y is non-FSD but I use AP in it. Only when driving alone though, the woman hates it as much as she hates FSDj and she never uses it when she’s driving (Model Y is her car)

My women just purchased a MY 4680 AWD, and NO FSD(j) or FSD(r)!

 

[JHCCAZ]

My women just purchased a MY 4680 AWD, and NO FSD(j) or FSD(r)!

Your women? Don't know whether to envy or pity you.
Maybe it's a religious thing, no judgment in that case...

 

[Ramphex]

Your women? Don't know whether to envy or pity you.
Maybe it's a religious thing, no judgment in that case...

You didn’t know Jeb is a prince from UAE?

 

[JHCCAZ]

You didn’t know Jeb is a prince from UAE?

News to me. But I was wondering if there were enough of them to require the seven-seat version.

 

[Ramphex]

News to me. But I was wondering if there were enough of them to require the seven-seat version.

Good question. You think he’s got 6 of them? I feel like it’d have to be an odd number if anything, 5 or 7. 7 wouldn’t fit in the car with him, 5 would leave one empty seat…. Someone would pretty much have to draw the short stick to sit all the way in the back. This could get pretty complicated.

 

[arnolddeleon]

My observations after a few more drives, focusing on freeways.

The car is no longer super eager to move to right lane for turns. So much so that several of my interventions were to get over so I wouldn't miss an exit. I've seen similar behavior on surface streets. I'm guessing this a heuristic to avoid the right lane (I have seen it flash the message about moving out of the right lane).

I think the car is less jumpy with cut ins (noticed this in the first few drives and continues to be that way). It seems to be "ok" with ending up "too close" to cars and the slowly increasing distance (versus over reacting). It feels drives more like me in this regard. This feels like a significant improvement.

It will confidently pass a car that is close to the lane line in an adjacent lane. A human (at least me) would slide over a little bit. The car also will also sit next to cars (something I avoid doing, I will pass or drop back). This is not new behavior.

Overall, I like it better on the freeway than old Navigate On Autopilot.

On surface streets it will cut a corner on left turns if there are no cars in other lane but correctly turns wider if there is one. My guess here is they have the car follow a "good enough" path when there are several options that are close (at least with the cost function they are using). This will sometimes pick a less than ideal path. The benefit here is that the car doesn't lurch back and forth between the alternate options.

I still think the car needs a "slow" and "long" planner. This is the planner than thinks about upcoming lane positioning and things like defensive driving. If it has one, it needs to look out further in time. I know they did make some improvements in planning for future lane changes but I think they will need to do more. My suspicion continues to be that this caused by being compute limited in HW3. They either need significant improvement in resource utilization or more hardware.

 

[jebinc]

Your women? Don't know whether to envy or pity you.
Maybe it's a religious thing, no judgment in that case...

I was just following @Ramphex dialog and use of the term “women”! It’s actually my wife!

 

[Ramphex]

I was just following @Ramphex dialog and use of the term “women”! It’s actually my wife!

Women is plural

 

[JHCCAZ]

I was just following @Ramphex dialog and use of the term “women”! It’s actually my wife!

Uh huh...

Look up "Freudian Slip".

 

[JHCCAZ]

Women is plural

So is "troubles".
Which is what @jebinc will have if his wife reads that post.

 

[JHCCAZ]

My observations after a few more drives, focusing on freeways.
...The car also will also sit next to cars (something I avoid doing, I will pass or drop back). This is not new behavior.

It does this on city streets also. I've noted that it likes to ride in the blind spot area.

On surface streets...I still think the car needs a "slow" and "long" planner. This is the planner than thinks about upcoming lane positioning and things like defensive driving. If it has one, it needs to look out further in time. I know they did make some improvements in planning for future lane changes but I think they will need to do more. My suspicion continues to be that this caused by being compute limited in HW3. They either need significant improvement in resource utilization or more hardware.

Have you read the release notes for 11.4? It sounds like there should be further improvements in the lane selection. However, they've claimed such improvements before, so I think we should expect "better but not solved"

Regarding compute limitations, Elon tweeted about AI Diffusion being more efficient than Transformers, which were more efficient and powerful than prior architectures. There have also been some recent reports about training on very large models and then executing on relatively small ones, while retaining most of the trained characteristics.

So I think it's not unreasonable to think that significant compute efficiency improvements are possible, and that Tesla will leverage new AI learning over time. I don't think there will ever be a hard and clear limit where HW3 runs out of compute.

But more is obviously better, and the concern is that they might not have enough engineering resources to keep
a) maximizing performance and learning onhwlHW4,
b) applying great effort into refining the HW3 version as it gets older,
c) expand FSD to China, Europe and rest of world, and on both platforms.

 

[EVNow]

I was just following @Ramphex dialog and use of the term “women”! It’s actually my wife!

You two are just a post away from describing your household as a harem

Good question. You think he’s got 6 of them? I feel like it’d have to be an odd number if anything, 5 or 7. 7 wouldn’t fit in the car with him, 5 would leave one empty seat…. Someone would pretty much have to draw the short stick to sit all the way in the back. This could get pretty complicated.

The official max for a UAE prince would be 4.

 

[JB47394]

At times still runs wide on turns. You can hear the tires drive over the reflectors.

I watched that clip a dozen times and I don't see any problem with that highway exit. The car was a bit off center, but certainly not onto the lane markers.

Cutting the corner on a left turn again.

The linked video shows how FSDb badly handled a truck pulling out of the corner gas station while the Tesla was trying to make a left turn through an intersection. Watch the planner. FSDb sees the truck, considers it an obstruction, tries to route around to the right of it, realizes that there is a car to FSDb's right, and decides to go straight ahead a bit and stop in the intersection. Definitely not the right behavior, but I didn't see anything about trying to cut a corner.

 

[Scott Chapman]

So which is going to happen first?

1. New FSDb appears on TeslaFi downloads page?
2. No FSDb versions appear on TeslaFi Downloads page?

 

[aronth5]

So which is going to happen first?

1. New FSDb appears on TeslaFi downloads page?
2. No FSDb versions appear on TeslaFi Downloads page?

Per usual YouTube influencers will report they received the 11.4.x release. Whether any "influencers" are on TeslaFi will answer your question.

 

[Mardak]

I'm guessing this a heuristic to avoid the right lane (I have seen it flash the message about moving out of the right lane)

You've focused on freeways, and indeed I've seen that message on interstates just before onramp merges, and I actually toggle on "Minimal Lane Changes" and force FSD Beta into the right-most lane to get some extra testing of merges. I'm guessing this moving out behavior is primarily to smooth out the experience by avoiding the need to slow down or get "too close" to the current lead vehicle or the merging vehicle, but so far 11.x seems acceptable in its ability to handle these.

I've also noticed the same message on higher speed local highways that have a lot more intersections. I believe the speed limit where this behavior kicks in is around 55mph, so that makes sense why it's active on interstates as well. However, more often than not, it's not actually necessary to get out of the right lane as there's rarely any vehicle getting on/off the highway whereas there is much more likely a vehicle in the left blindspot when the "get out" logic activates.

Do people think this is a temporary workaround to try to hide/lessen some undesired current FSD Beta behavior or that it'll stick around for a while?

 

[JB47394]

Do people think this is a temporary workaround to try to hide/lessen some undesired current FSD Beta behavior or that it'll stick around for a while?

I consider this to be an accident mitigation strategy. By staying out of the right lane, there will be fewer interactions with cars that are changing lanes, speeding up, slowing down, etc. All of those are opportunities for FSDb to make a mistake - or for another driver to do something idiotic against which few could defend.

You mentioned a lack of need to stay out of the right lane. I doubt their code worries about context that much. I think it just says Right Lane Bad and goes from there.

 

[thewishmaster]

Observed what seems like new, or at least newly mentioned in the UI "Merging out of the rightmost lane" behavior today; it was a stretch of freeway with no onramps / offramps on the right side. The nav actually needed the car to stay in the right lane due to me needing to exit at the next exit.

It did end up merging back to the right lane eventually, but not after 2 attempts to get out of the right lane. First one I wasn't paying attention to the screen and disengaged when I felt the car suddenly start to leave the lane (the blinker activated late), and the second time I let it merge to see how soon it would merge back.

Also it seems like the "sound / vibrate" feature to warn before changing lanes doesn't work for FSDb because I have both enabled

Saw this behavior again… no reason to merge left, car began to leave the lane without a blinker, message, or even visualization of the lane change. At first I thought it was confused by the lane lines, but after disengaging and recording a voice note I realized the left blinker was on, presumably turned on after a delay similar to last time.