FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Mardak]

How is this possible?

"It was not slippery outside that time of day." Proceeds to share video of him shoveling directly next to the tire that might have been at the steepest part of the driveway while rear wheels remained locked as the car slips down the driveway revealing compacted snow.

Although FSD Beta 11.4.9 and past versions are indeed quite bad at braking in snow given their poor motion profile for approaching some stops that now have stop lines covered up. Acceleration profile can also be quite dangerous when turning into slippery surfaces if you don't manually drop the set speed well in advance. The snow on road and snow banks probably also confuse perception resulting in a lot more micro-adjustments of steering that I've experienced minor fishtailing even when trying to drive straight.

 

[mgs333]

How is this possible?

Video: Nevada man stunned as parked car moves down driveway

A Sparks, Nevada man said his car rolled down his driveway Friday afternoon, and he is still trying to figure out why.

www.kcra.com

This is news?

 

[JulienW]

This is news?

....and FSD 11 news at that. Maybe it is a V12 in the snow preview.

 

[aronth5]

One new thing I noticed in 11.4.9 , not sure if its been there and I just haven't noticed.
One the highway with an onramp merging from the right in slow traffic. It highlights both the merging car in front and behind as they merge into our lane.
I remember seeing it do that for the forward car, but don't recall seeing it track the one behind.

Don't care about the visualization I'm too busy watching traffic. Does FSD slow down to match the slow moving traffic and then fit in or zoom past if there is space in the zip lane and then run out of room? That's what happens to me fairly consistently. Otherwise highway merging has improved a lot for me.

 

[rlsd]

"It was not slippery outside that time of day." Proceeds to share video of him shoveling directly next to the tire that might have been at the steepest part of the driveway while rear wheels remained locked as the car slips down the driveway revealing compacted snow.

Although FSD Beta 11.4.9 and past versions are indeed quite bad at braking in snow given their poor motion profile for approaching some stops that now have stop lines covered up. Acceleration profile can also be quite dangerous when turning into slippery surfaces if you don't manually drop the set speed well in advance. The snow on road and snow banks probably also confuse perception resulting in a lot more micro-adjustments of steering that I've experienced minor fishtailing even when trying to drive straight.

Is it possible the car was accidentally in N position? (car wash mode or towing mode can).

 

[rlsd]

....and FSD 11 news at that. Maybe it is a V12 in the snow preview.

That car did not want to leave the world behind yet.

 

[Yelobird]

"It was not slippery outside that time of day." Proceeds to share video of him shoveling directly next to the tire that might have been at the steepest part of the driveway while rear wheels remained locked as the car slips down the driveway revealing compacted snow.

Although FSD Beta 11.4.9 and past versions are indeed quite bad at braking in snow given their poor motion profile for approaching some stops that now have stop lines covered up. Acceleration profile can also be quite dangerous when turning into slippery surfaces if you don't manually drop the set speed well in advance. The snow on road and snow banks probably also confuse perception resulting in a lot more micro-adjustments of steering that I've experienced minor fishtailing even when trying to drive straight.

How is this possible? Well, Tesla was in the title, add some gratuitous kids in danger sadness, and we Got clicks!!! While not easy with the crap Abode commercial, a quick pause shows he had dam near NO treads on the tires. Looks more like a neglectful parent story endangering his kids to save a few bucks on tires……

 

[kabin]

One of my frustrations is late lane entry. It doesn't matter if it's left or right turn, if there's a long solid white line, cars in adjacent lanes, 11.4.9 will plan to enter that lane in the last 100ft or so with a late brake.

 

[kabin]

And what's the deal with roundabouts? In this case the speed limit was only 35mph and he didn't need to disengage but FSD applied hard braking at the last second.

My roundabout examples are 45mph and it's a crazy ride.

There seems to be something wrong with the planner as these left/right turns and roundabout entries are too haphazard.

 

[rlsd]

One of my frustrations is late lane entry. It doesn't matter if it's left or right turn, if there's a long solid white line, cars in adjacent lanes, 11.4.9 will plan to enter that lane in the last 100ft or so with a late brake.

There is another regression in 11.4.9: The car was on the correct lane to make right turn to the freeway ramp. But it changed to left lane although the distance to the ramp was less than than .4 mile. This was with minimal lane change setting and it always did that every time.

 

[jebinc]

There is another regression in 11.4.9: The car was on the correct lane to make right turn to the freeway ramp. But it changed to left lane although the distance to the ramp was less than than .4 mile. This was with minimal lane change setting and it always did that every time.

There have been over three years of regressions. How many years will this community put up with this junk? Next, v12 will come out and it will be underwhelming; the Elon devotionalists here will pronounce how that's to be expected as this is new tech, NNs, and other BS. Then, the Charlaton (Elon) will talk about how v13 is "mind blowing," and the loyalists will swallow that, hook, line and sinker... Can't you all see this is never going to end - especially for HW3 cars and lower? Seriously now....

 

[PACEMD]

Most humans I encounter can't either

I know, right? Humans are terrible drivers. They make FSDb look like an Indy 500 drive in the park, on half-dose steroids, hey, where'd you get your license? Sears?

 

[PACEMD]

I know most humans can handle wiper controls.

A recent study showed that the majority of human beings are not smart enough to even know what windshields are, let alone how to wipe them.......

 

[PACEMD]

How many years will this community put up with this junk?

2 and 1/2 years, and that's my final offer..........

 

[Hiline]

The way TSLA stock price is heading these days Elon will need to pull a rabbit out of the hat with a v12 release if at all possible. Earnings release is Jan 24. He'll also need the next FSD PR distraction for v12's potential fall from grace.

And why would Elon care about stock price? He never did before.

 

[Twiglett]

Don't care about the visualization I'm too busy watching traffic. Does FSD slow down to match the slow moving traffic and then fit in or zoom past if there is space in the zip lane and then run out of room? That's what happens to me fairly consistently. Otherwise highway merging has improved a lot for me.

right - this was when I was already on the highway with a merge coming in from the right. It seems to let the zipper work really well. The main thing that I noticed was it now tracking both cars, the one merging in front and the one merging behind.
When everyone is doing 5-10mph its no issue monitoring the cars and the display.

 

[sleepydoc]

Lol. On a flat surface, it's perfectly correlated (assuming you have it display the braking portion using the GUI config). It's just physics. There's no other way it can be. There's literally zero way to have a debate about this. (Neglecting the 1-2kW three-pixel-wide portion which is allocated to accessory use.)

IME the line moves around much more than the ride quality would suggest. If you feel it decelerating then you always see the line move to the left but the converse is not always true.

I will say that The latest version seems to have taken a step backwards in estimating distances and stopping. Sometimes it stops fine, other times it does the ol’ ‘let’s see how long I can wait before braking’ routine.

 

[sleepydoc]

A recent study showed that the majority of human beings are not smart enough to even know what windshields are, let alone how to wipe them.......

That study was retracted - turns out half of them were on Xanax at the time…

 

[AlanSubie4Life]

I suspect critical moments, like intersection stops, they want to minimize pixel smearing caused by brake induced suspension dive

I would think it would cause the least smearing if the vehicle were stopped. And they don't maximize the amount of time stopped.

If you feel it decelerating then you always see the line move to the left but the converse is not always true.

I think converse is swapping if & then? Or did you mean inverse? Anyway, If you see the line move to the left, you will feel deceleration (on a flat surface). Inverse would be if the car accelerates the line moves to the right. Which is also true on a flat surface.

The display may be nonlinear (not sure, never tried to determine) - but also remember that at slow speeds, a short line length has a lot more impact (in g forces) than at high speeds. The line is a measure of power applied (either stopping power or motor power), and it takes a lot more power at higher speeds to produce a given acceleration/deceleration than at low speeds. But after correcting for that, it's proportional to acceleration/deceleration & you can use it to see g forces directly (on a flat surface). Obviously it's not QUITE that simple because aero drag, rolling resistance, etc., produce an offset which is also proportional to speed (or to the third power for for power to overcome aero (consumption goes with the square)). But that's generally going to be really small on the display at typical surface street speeds (these cars roll really well in neutral).

P = F * v = m * a * v

 

[MP3Mike]

I took FSDb 11.4.9 out for a short drive today, where we are in the mid 15 degrees, and there is some snow/ice on the road in places. I don't know if it is because of the temperature, or what it saw, but it definitely changed the acceleration profile, where it accelerated much slower, and didn't always get up to the set speed. (It started stopping sooner as well.) It seemed to leave more following distance, but not always. A couple times when making a lane change or turn, it slid on a patch of snow/ice and it handled/corrected it without issue. (Note there was plenty of space between it and everyone else on the road when I let it do this.)

Insurance app only logged one braking event on each direction of the trip.

Overall, I was pretty happy with the driving performance. (But the wiper heating was severely lacking, assuming it was working at all.)

Yesterday I had a 29 mile trip with no interventions/disengagements and the insurance app only logged two acceleration events: