FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[ItsNotAboutTheMoney]

Trying to envision this. Which lane were you in and which direction were you turning?

I read it as being in the left lane of the 2, and then it turned and instead of going left lane to left lane, it went left lane to right lane.

 

[aronth5]

Very few posts now mention phantom braking. For me I wouldn't say phantom braking is completely solved but the reduction is significant. Better description is "unnecessary braking" when FSD overreacts to a car near me. Would others agree?

 

[Scott Chapman]

Very few posts now mention phantom braking. For me I wouldn't say phantom braking is completely solved but the reduction is significant. Better description is "unnecessary braking" when FSD overreacts to a car near me. Would others agree?

Yea, I feel like it is being extra cautious. I noticed it a couple of times when a road joins from the right and a car is rapidly approaching a stop sign. My guess is that the trajectory says there is higher probability of collision, until the car increases stopping for the stop sign.

I notice something similar (less frequently) on rolling curvy rural road when all of a sudden a car appears in the opposite lane. Especially on a curve I again suspect that the initial trajectory has a lot of uncertainty which eventually improves.

But it isn't so much that it is braking, more like it is lifting off the accelerator slowing the car down.

 

[Scott Chapman]

I do wonder how they will start leveraging the new computer (I also don't know how long before it becomes useful). I sort of expected that it would be dedicated to V12, leaving previous gen computer working on V11 or something like that.

 

[willow_hiller]

Very few posts now mention phantom braking. For me I wouldn't say phantom braking is completely solved but the reduction is significant. Better description is "unnecessary braking" when FSD overreacts to a car near me. Would others agree?

It's rare for me to experience a harsh braking event without knowing the reason why. But I'm often glancing at the visualization. Anyone not looking at the visual would still probably call it phantom braking.

One scenario where FSD Beta unnecessarily brakes most often is for vehicles in the on-coming traffic lane positioning themselves for a left turn across my lane. It will always highlight them in blue when it brakes.

 

[Joboo7777]

It's rare for me to experience a harsh braking event without knowing the reason why. But I'm often glancing at the visualization. Anyone not looking at the visual would still probably call it phantom braking.

One scenario where FSD Beta unnecessarily brakes most often is for vehicles in the on-coming traffic lane positioning themselves for a left turn across my lane. It will always highlight them in blue when it brakes.

This is the cause of my abrupt breaking events. Even when I’m in the far right lane and the on coming car making the left is no where near impeding on the left lane in my direction. Sometimes it happens even after I have passed the car trying to make the left.

 

[JohnB007]

Trying to envision this. Which lane were you in and which direction were you turning?

Sorry for the messy explanation.

I was waiting on the left most lane, trying to turn left at the intersection. The road I'm about to turn is a 4-lane road. The car went directly from the left most lane to the right most lane (without touching the left lane) while turning left, all in one motion.

Hopefully that's more clear...

 

[Twiglett]

Seems like if you install and then reboot and let the car go into deep sleep (or full power off) things improve. I just got offered 4.7 so I'll check out as well.

in for a penny...
Here goes - how much worse can it be?
Says all of us who notice such ridiculous regressions

 

[JulienW]

I do wonder how they will start leveraging the new computer (I also don't know how long before it becomes useful). I sort of expected that it would be dedicated to V12, leaving previous gen computer working on V11 or something like that.

Obviously you missed or are oblivious to Elon's V12demo drive.

 

[EVNow]

Finally got 4.7 yesterday. Yet to drive, though. Haven't seen much positive commentary about 4.7 (compared to 4.4) on YT.

 

[SSDTester#1]

I didn’t go out when you came to town but had to yesterday. I fully expected them to not work at all but they not only worked, they slowed down and went intermittent when needed. First time I’ve used them and not had to manually shut them off or put them on intermittent myself.

I was surprised

Video Aug 30 2023, 08 58 36.mov

Shared with Dropbox

www.dropbox.com

The only time the wipers came on is when I pressed the wiper button lol

 

[MP3Mike]

The only time the wipers came on is when I pressed the wiper button lol

Yeah, I had that with my first drive on 11.4.7 last night as well. (But yet it was displaying the FSD possibly degraded because of weather message, so it knew it needed the wipers.)

Other people have reported that after you have driven and the wipers didn't work, that once the vehicle goes to a deep sleep, i.e. no Sentry mode, they start working properly again. I'm not sure when I will get the next chance to drive in rain...

 

[SSDTester#1]

Yeah, I had that with my first drive on 11.4.7 last night as well. (But yet it was displaying the FSD possibly degraded because of weather message, so it knew it needed the wipers.)

Other people have reported that after you have driven and the wipers didn't work, that once the vehicle goes to a deep sleep, i.e. no Sentry mode, they start working properly again. I'm not sure when I will get the next chance to drive in rain...

Interesting. I’ll keep an eye on it for the next drive.

 

[FSDtester#1]

After a deep sleep and a 2 button reset. Auto wipers are working again for me.

 

[FOOLSLFDRVNG]

Yup. It's actually worse now than it was on 11.4.6.

I only had up to 11.4.4 now 11.4.7 and just had a random lane change to the right lane when I need to make a left less than a mile. I zoom in on the map and notice the path is not straight but a bit crooked. This is probably the culprit.

 

[JHCCAZ]

After a deep sleep and a 2 button reset. Auto wipers are working again for me.

They say humans can improve their athletic performance by practicing in their minds, and even dreaming over and over about shooting foul shots or hitting a curveball. The advice is to fall asleep thinking about the task.

Elon told us the car has a mind; this is just one more example!

 

[Dewg]

After a deep sleep and a 2 button reset. Auto wipers are working again for me.

And I got laughed at for suggesting that could be the solution.

 

[LSDTester#1]

After a deep sleep and a 2 button reset. Auto wipers are working again for me.

This would make sense. My car was off a day longer than @Ramphex and I did a reset for another issue. Blows my theory of it just likes me better than the rest of yous guys

 

[SSDTester#1]

This would make sense. My car was off a day longer than @Ramphex and I did a reset for another issue. Blows my theory of it just likes me better than the rest of yous guys

I did the two wheel reset, just haven’t tried to get it to also deep sleep. Well it’s been sleeping for a couple of days now so hopefully it’s deep enough

 

[rlsd]

And I got laughed at for suggesting that could be the solution.

Reset button press:
1 time fixes wipers
2 times fixes slowness to move at intersection
3 times fixes erratic lane changing
4 times fixes excessive nags
5 times increases range to 150 percent