FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[sleepydoc]

You are too picky. I drive on the same road like you do but don't notice much. The most annoying break happens when some car on the left or right lane comes close to my car - it looks like FSD wants to yield the lane to those cars - bad.

Let Tesla fix my right turns first. Please, please.

Yes, Alan tends to perseverate on making the acceleration/deceleration curves perfectly smooth, then gets full blown OCD over the regen bar. The regen bar correlates very poorly with the actual drive so focusing on how much it moves in one direction or another doesn’t tell you much.

 

[KLam]

I really don't care whether Tesla uses regen or brake to slow down, as long as it is smooth and on time. I would rather they fixed the steering wheel pointing to the left when stopped to take a right turn and pointing to the right when stopped to take a left turn! Can they make it point straight? Wide turns are unsafe.

 

[Twiglett]

I really don't care whether Tesla uses regen or brake to slow down, as long as it is smooth and on time. I would rather they fixed the steering wheel pointing to the left when stopped to take a right turn and pointing to the right when stopped to take a left turn! Can they make it point straight? Wide turns are unsafe.

with you on that one. Wide turns are for people who can't figure out how to steer their car. FSD shouldn't need to drive like a learner.

 

[Tronguy]

So, something new, different, and better with 11.4.9 and 2023.44.30.8/2023.44.30.7.

With every version of FSD I've had the two following problems:

• Say that one is on a two-lane road, one each direction. In front of one is a vehicle attempting an unprotected left turn onto a road that "T's" onto the road one is on. The lane that one is in is wide, probably on purpose, so that cars that are going straight ahead can casually steer a little to the right and go around the car waiting for traffic to clear. Until a couple of days ago, on FSD, invariably, the Tesla would line up behind the turning car, sometimes shifted to the right somewhat, so that traffic behind one would be blocked, waiting for the turning car to go left and get out of the way. (This leads to a certain amount of cursing and an intervention because cars behind one going straight are going to try and squeeze by the right-shifted Tesla, and that's dangerous.) After the car in front of one would turn, the Tesla would continue on.
• In a related scenario: One wants to do an unprotected left onto an on-ramp, there's no lights or signage, and there's on-coming traffic that has to clear before one does so. Further, there's traffic behind one that would legitimately go around one's right, and there's Plenty Of Room to the right. Depending upon the Tesla FSD-b vintage, the car has:
  • Swerved clear over to the right hand curb, then swerved back, seemingly trying to block the road from cars trying to go straight. And I mean these swerves are severe - jerky wheel motions, BIG car motions, honking people behind one, etc. In one notable instance cars passed me on the left as the car swerved right, and then the Tesla acted like it wanted to move back into the left and hit the car passing on the left. It might not have hit that guy passing me.. but it was close, and it was dangerous.
  • In general, then, when approaching this intersection, the car had a nasty habit of swerving to the right before a left turn, attempting to center itself between the double-yellow line and the curb. When there was no traffic out, or at least none on my side, I've let the car do its thing to see what it would do. Mostly it would center itself rather abruptly instead of hugging the double-yellow, and the turn itself was always jerky, if it had to wait.
  • If there was no approaching traffic (or traffic behind me) I'd let it do its thing. Older versions were very jerky on this turn; as time has gone on, it's been a bit less jerky, but a bit less on the swerve. Still, not enough room for somebody to pass me on the right.

So, on 2023.44.30.7, was approaching, in moderate traffic, a pick-up truck waiting to turn left. The car slowed down, I got set to intervene - and the car shifted right and went around the truck. A bit slowly, but that was a first.

Then, today, on 2023.44.30.8, in moderate traffic in both directions, here came that UPL. I got set to intervene, once again - but the car, while moving a bit away from the double-yellow, stayed within six feet of it. Cars up ahead on the other side got that "blue" color, indicating that the car was waiting for traffic to clear. But it had not centered itself in the lane and, by gum, a big, old box truck passed me on the right without any trouble. When the traffic coming the other way cleared.. the car picked up and, fairly smoothly (not as smooth as I would have done it) did the left turn and got onto the ramp.

I've been complaining about these two problems for over a year. In the last three days, both are working. Once each, given. And given the vagaries of FSD-b, there's no guarantee it won't do some other cock-a-mamie thing the next time I go through these places. And that's just two intersections where one is either encountering people doing a UPL or is attempting one oneself, and there's lots of those suckers around here.

But, still, looks like an improvement.

 

[AlanSubie4Life]

The regen bar correlates very poorly with the actual drive

Lol. On a flat surface, it's perfectly correlated (assuming you have it display the braking portion using the GUI config). It's just physics. There's no other way it can be. There's literally zero way to have a debate about this. (Neglecting the 1-2kW three-pixel-wide portion which is allocated to accessory use.)

I really don't care whether Tesla uses regen or brake to slow down,

Nor do I, as I have repeatedly said.

Unfortunately, under current programming, friction brake use is correlated with poor smoothness and late braking. Everyone knows this, I think. Even if they are reluctant to agree. Haha.

It's quite possible to smoothly slow down with higher acceleration forces using friction braking, under manual control. Many aggressive and competent drivers do it. You get between A & B faster. But that's not what FSD is doing.

No one knows what FSD is attempting to do. There is definitely overcompensation for lead vehicles as @DrChaos has mentioned (this could be to ensure extra safety margin at the expense of comfort). But there are plenty of issues in the absence of any vehicles at all.

I would rather they fixed the steering wheel pointing to the left when stopped to take a right turn and pointing to the right when stopped to take a left turn! Can they make it point straight? Wide turns are unsafe.

You'd think this would be fixed on V12. It'll be something to look forward to. If they're actually training using real humans, there would be no reason to do this (except when making an unprotected left when they turn the wheel into traffic when stopped, based on the massive dataset of drivers doing this, incorrectly).

 

[EVNow]

Got updated from 2023.44.30.5.1 to 2023.44.30.8 - both 11.4.9 FSD build. Looks to be a first for me.

 

[oneday]

Just updated as well. Minor fixes and improvements.

 

[kabin]

Apparently there's a video of a UPL related crash with FSD engaged. Chuck describes it along with the usual warning.

And I should add Chuck's 2nd UPL attempt (@11:10) was a disengagement for attempting to enter a lane with a quickly approaching vehicle.

 

[AlanSubie4Life]

Apparently there's a video of a UPL related crash with FSD engaged.

Median thing is dumb. Always just rocket across.
Personally I think FSD increases risk of an accident at ULTs. It distracts me from checking for cross traffic. Not good.

 

[cbdream99]

This is a new one after installing 44.30.8. There is a big sentry alert show up in the MCU, it won’t go away until I launch the viewer, remains on during driving.

 

[Yelobird]

This is a new one after installing 44.30.8. There is a big sentry alert show up in the MCU, it won’t go away until I launch the viewer, remains on during driving.

View attachment 1008287

Remove and reinstall USB disk.

 

[Scott Chapman]

I am amazed that basically the entire tesla fleet is going to be on the same release, and that the next major update to FSD is already in the same general code line. That is amazeballs.

 

[Mardak]

I am amazed that basically the entire tesla fleet is going to be on the same release

Yeah at least the current state of FSD Beta testers and general release on the same version 2023.44.30.8 / 11.4.9, TeslaFi shows installed and pending of this version would be nearly 92% of its vehicle population. Around 3% of that population have MCU1 and still on 2022.8.10.x and maybe the rest haven't checked in yet, but it does seem like it's intended to go to all MCU2+ vehicles.

Curious what would be important enough to get basically everyone on this latest version? Perhaps there's "minor fixes" for additional data collection that we saw when FSD Beta first went out in 2020.

 

[Hiline]

I've noticed Tesla is pushing 44.30.8 to cars that aren't connected to WiFi as well. Bringing the entire fleet to the same firmware version helps streamline features. Maybe Tesla didn't do this before just because of the lack of bandwidth, and fleet-wide firmware updates is going to be the way forward.

 

[TK211X]

This is a new one after installing 44.30.8. There is a big sentry alert show up in the MCU, it won’t go away until I launch the viewer, remains on during driving.

View attachment 1008287

This is hilarious

 

[MarcG]

Maybe Tesla didn't do this before just because of the lack of bandwidth

Or because they have to fix an issue asap.. or comply with a NHSTA order

 

[MarcG]

I am amazed that basically the entire tesla fleet is going to be on the same release

This isn’t actually good for Tesla. I think they like at least two versions out in large numbers at the same time, so they can do A/B testing and gather enough data for a while to improve future versions faster than on a single release at a time (which is eventually better for us too).

 

[Twiglett]

One new thing I noticed in 11.4.9 , not sure if its been there and I just haven't noticed.
One the highway with an onramp merging from the right in slow traffic. It highlights both the merging car in front and behind as they merge into our lane.
I remember seeing it do that for the forward car, but don't recall seeing it track the one behind.

 

[AlanSubie4Life]

2023.44.30.8 (11.4.9)

Just a quick sample showing the issues with stop signs as requested. The car will only proceed when it gets to 0mph. That is fine, no issues with that - it is great. However, it does not use the brakes to get to 0mph! It brings the speed down rapidly and not too smoothly from 10mph with mostly regen (hard to see friction brake use in video, it is very slight use, mostly regen), to 1mph. Then it just coasts to 0mph, which takes significant time - over one second, from 0:22 to past 0:23 is spent between 1 and 0mph. For comparison, from :21 to :22 takes it from 7mph to 1mph (does not seem extreme, though arguably higher than needed - it is 0.27g). If friction brakes were used the halt the vehicle at the appropriate position instead, time would be saved (I would estimate 2-3 seconds in total - with ~1 second from the drift alone - with other modifications to the profile, which is a very large amount of time) and the stop would feel more perceptible.
I wonder if the issue is the car wanting to stop in exactly the right location? So it slows down to approximately correct location, then drifts? I don’t think so, but could be one reason for this bizarre behavior.

If I get a chance I’ll do a human stop at this stop sign for comparison. But I think in the video, the issue is clear.

Finally, proof. Let me know if others see better behavior (with video if possible!). I think some people have claimed they don’t see this behavior (though many have indicated it is definitely an issue so I am not alone).

 

[AlanSubie4Life]

Or because they have to fix an issue asap.. or comply with a NHSTA order

No, it is because they are getting ready to move everyone together to FSD v12.

Obligatory unfortunately: