FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[kabin]

New release notes for FSD Beta v11.4.8:

- Added option to activate Autopilot with a single stalk depression, instead of two, to help simplify activation and disengagement.

- Introduced a new efficient video module to the vehicle detection, semantics, velocity, and attributes networks that allowed for increased performance at lower latency. This was achieved by creating a multi-layered, hierarchical video module that caches intermediate computations to dramatically reduce the amount of compute that happens at any particular time.

- Improved distant crossing object detections by an additional 6%, and improved the precision of vehicle detection by refreshing old datasets with better autolabeling and introducing the new video module.

- Improved the precision of cut-in vehicle detection by 15%, with additional data and the changes to the video architecture that improve performance and latency.

- Reduced vehicle velocity error by 3%, and reduced vehicle acceleration error by 10%, by improving autolabeled datasets, introducing the new video module, and aligning model training and inference more closely.

- Reduced the latency of the vehicle semantics network by 15% with the new video module architecture, at no cost to performance.

- Reduced the error of pedestrian and bicycle rotation by over 8% by leveraging object kinematics more extensively when jointly optimizing pedestrian and bicycle tracks in autolabeled datasets.

- Improved geometric accuracy of Vision Park Assist predictions by 16%, by leveraging 10x more HW4 data, tripling resolution, and increasing overall stability of measurements.

- Improved path blockage lane change accuracy by 10% due to updates to static object detection networks.

Nice! Long over due for a real update. We need some NFG software.

I imagine the improved latency isn't across the board. If it was they should be able to ditch the 30ft short of stop sign/crawl crutch.

 

[fcastrillo86]

any word what software version 11.4.8 is included in?

 

[JulienW]

any word what software version 11.4.8 is included in?

Full L5, Robotaxi App and free steering wheel and peddles removal included.

 

[jebinc]

Full L5, Robotaxi App and free steering wheel and peddles removal included.

And a pair of puckered lips for Elon’s @$$?

 

[fcastrillo86]

Full L5, Robotaxi App and free steering wheel and peddles removal included.

LOL. actually i was asking about software branch. the way it's been going, seems safe to assume this is on 2023.27.x, but would be pleasantly surprised if we got moved to a higher branch.

 

[jebinc]

LOL. actually i was asking about software branch. the way it's been going, seems safe to assume this is on 2023.27.x, but would be pleasantly surprised if we got moved to a higher branch.

I don’t see that happening intil the holiday release, at the earliest. Would love to be wrong.

 

[MP3Mike]

LOL. actually i was asking about software branch. the way it's been going, seems safe to assume this is on 2023.27.x, but would be pleasantly surprised if we got moved to a higher branch.

It is interesting that the single pull option for AP, mentioned in the FSDb 11.4.8 notes, was added in 2023.38.8. (which is still FSDb 11.4.4.)

That could be a hint, or not.

 

[sleepydoc]

Re: Improved distant crossing object detections by an additional 6%,
I just think like ... if it was 100 feet detection before then it is 106 feet now

I've interpreted them to me an improvement in "recall" without making "precision" worse. Which I understand to mean as it will correctly identify 6% more distant object crossings. They don't define what distant object crossings mean but I guess that to mean figuring out if the distant object is going to be relevant to planning.

This might be helpful:

Precision and recall - Wikipedia

Or does it just miss them 6% less?

 

[sleepydoc]

Full L5, Robotaxi App and free steering wheel and peddles removal included.

But if you have a yoke you’re stuck with it!

 

[LowlyOilBurner]

Wonder when I’ll be able to use summon to back my HW4 car out of my garage like I used to on my HW3 car?

 

[EVNow]

Or does it just miss them 6% less?

Exactly. We never know whether they are talking % or basis points.

It used to miss 20% of the time. Now it misses 14% of the time (600 basis points improvement).
It used to miss 20% of the time. Now it misses 18.8% of the time (120 basis points improvement).

 

[rogersmj]

Took a roadtrip last week and tried to extensively use FSD. Still on my free trial. I'm on FSD Beta v11.4.7.3. I had tried out an earlier version a couple of months ago and did not like it.

My experience: it’s still a disaster. I had to cancel it so often… riding too close to the center line on a 2-lane state road, lane changes for no reason (e.g. no one in front of me in the right lane, but changing into the passing lane anyway) on the interstate that bordered on cutting people off, veering too sharply into exit ramps, dangerously unnecessary braking (seriously, it slowed down to 25 mph *before* it took the exit ramp)…and this was all in easy, straight-road, good-weather driving. It’s atrocious in city or even just suburban traffic with parked cars, delivery trucks, roundabouts, etc.

In short, I'm far more stressed when FSD is turned on vs when it's off. I constantly expect the car's going to do something stupid, and it often does.

Don’t get me wrong, I’m thrilled someone’s working on this technology, but it is not good yet and people are insane to pay as much as they do for it.

I mean, I guess v12 is supposed to replace v11 on all cars manufactured after a certain point, right? Kind of surprised they're still putting all this effort into update for v11 when it seems like it's just fundamentally not going to get to where it needs to be (hence the work on v12).

 

[jebinc]

Took a roadtrip last week and tried to extensively use FSD. Still on my free trial. I'm on FSD Beta v11.4.7.3. I had tried out an earlier version a couple of months ago and did not like it.

My experience: it’s still a disaster. I had to cancel it so often… riding too close to the center line on a 2-lane state road, lane changes for no reason (e.g. no one in front of me in the right lane, but changing into the passing lane anyway) on the interstate that bordered on cutting people off, veering too sharply into exit ramps, dangerously unnecessary braking (seriously, it slowed down to 25 mph *before* it took the exit ramp)…and this was all in easy, straight-road, good-weather driving. It’s atrocious in city or even just suburban traffic with parked cars, delivery trucks, roundabouts, etc.

In short, I'm far more stressed when FSD is turned on vs when it's off. I constantly expect the car's going to do something stupid, and it often does.

Don’t get me wrong, I’m thrilled someone’s working on this technology, but it is not good yet and people are insane to pay as much as they do for it.

I mean, I guess v12 is supposed to replace v11 on all cars manufactured after a certain point, right? Kind of surprised they're still putting all this effort into update for v11 when it seems like it's just fundamentally not going to get to where it needs to be (hence the work on v12).

Of course it’s a “mind-blowing,” disaster.

 

[dflowerz]

Took a roadtrip last week and tried to extensively use FSD. Still on my free trial. I'm on FSD Beta v11.4.7.3. I had tried out an earlier version a couple of months ago and did not like it.

My experience: it’s still a disaster. I had to cancel it so often… riding too close to the center line on a 2-lane state road, lane changes for no reason (e.g. no one in front of me in the right lane, but changing into the passing lane anyway) on the interstate that bordered on cutting people off, veering too sharply into exit ramps, dangerously unnecessary braking (seriously, it slowed down to 25 mph *before* it took the exit ramp)…and this was all in easy, straight-road, good-weather driving. It’s atrocious in city or even just suburban traffic with parked cars, delivery trucks, roundabouts, etc.

In short, I'm far more stressed when FSD is turned on vs when it's off. I constantly expect the car's going to do something stupid, and it often does.

Don’t get me wrong, I’m thrilled someone’s working on this technology, but it is not good yet and people are insane to pay as much as they do for it.

I mean, I guess v12 is supposed to replace v11 on all cars manufactured after a certain point, right? Kind of surprised they're still putting all this effort into update for v11 when it seems like it's just fundamentally not going to get to where it needs to be (hence the work on v12).

My experience too and I paid big bucks for it. A new update will come out and I give it a try and it still has major issues so I stop using it until the next update. Autopilot is better and I will use that fairly often. In retrospect I would not have bought FSD. Now hoping one of these days it works but not optimistic.

 

[GSP]

… riding too close to the center line on a 2-lane state road, lane changes for no reason (e.g. no one in front of me in the right lane, but changing into the passing lane anyway) on the interstate that bordered on cutting people off, veering too sharply into exit ramps, dangerously unnecessary braking (seriously, it slowed down to 25 mph *before* it took the exit ramp)…and this was all in easy, straight-road, good-weather driving.

In short, I'm far more stressed when FSD is turned on vs when it's off. I constantly expect the car's going to do something stupid, and it often does.
…..

These things frustrate me as well. Riding too close to the center line on 2-lane roads is unnerving. Opposing traffic often rides the shoulder to avoid me.

The absolute worst is the forced use of auto windshield wipers. I have been forced to drive manually for hundreds of miles, not able to even use cruise control, because the auto wipers constantly ran on my completely dry windshield. I haven’t had to make a long highway drive without cruise since the 1970’s, when my old car did not have it. It makes me want my HW1 Model S back, it was great for reducing fatigue on long drives.

GSP

 

[FSDtester#1]

These things frustrate me as well. Riding too close to the center line on 2-lane roads is unnerving. Opposing traffic often rides the shoulder to avoid me.

The absolute worst is the forced use of auto windshield wipers. I have been forced to drive manually for hundreds of miles, not able to even use cruise control, because the auto wipers constantly ran on my completely dry windshield. I haven’t had to make a long highway drive without cruise since the 1970’s, when my old car did not have it. It makes me want my HW1 Model S back, it was great for reducing fatigue on long drives.

GSP

Did anyone see windshield wiper fix in new release notes??

Not me.

 

[cbdream99]

These things frustrate me as well. Riding too close to the center line on 2-lane roads is unnerving. Opposing traffic often rides the shoulder to avoid me.

The absolute worst is the forced use of auto windshield wipers. I have been forced to drive manually for hundreds of miles, not able to even use cruise control, because the auto wipers constantly ran on my completely dry windshield. I haven’t had to make a long highway drive without cruise since the 1970’s, when my old car did not have it. It makes me want my HW1 Model S back, it was great for reducing fatigue on long drives.

GSP

I rented a M3 for few days and it just a mixed bags, lots of these moving everything to the MCU is just not intuitive and takes a while to figure out. I picked up the car from the airport and discovered the steering wheel heat is too hot, how to I control it while driving? Do I search while driving, google it, I tried voice command "turn off steering wheel heat" and thankfully it worked.

Next, I need to move 2 cars in the driveway, since I will be blocking someone for few seconds, I want to turn on the hazard light, where is it? Ahh, google helped me, what if I am at an accident and the adrenaline is rushing my mind, can I still calm enough to google it ?

Then, the windshield wiper, it won't turn on intermittent and the stupid AI algorithm doesn't work, so voice command works but do I have to keep on saying turning on turning off ... Ahh RTFM, is one press or long press, stalkless is controlled by the MCU. I can pick up any ICE rental and expect them to work similarly across all brands. I have never seen a car that is so bad with windshield wipers since day one, too slow in heavy rain, intermittent pace is not good, and now completely broken, turn on randomly, hard to control while driving ...

There is one time after they changed the UI and I couldn't figure out how to defog the windshield while driving with FSD, it is 80% foggy and I have to quickly stop in a safe side road to google it. This is something that I rarely use and only find out the poison when I needed it.

For FSD, it works reasonably well on the interstate, the lane changing decisions however sometimes doesn't make sense, why changing lane when it is about to exit, why move to the slowest lane when all the lanes are similar pace and don't need to exit in the next 10 miles, why not move to the left most lane when there is a chance, the exit is only a mile or less away and lots of traffic ahead. It used to when I signaled it, it will change lane even with FSDb, but now it only works 20% of the time, other time just leave the blinker on and has to manually cancel it.

On the city, I think it depends on vision more than gps now, it always drift into turn lane, and on some roads not centering correctly, sometimes driving too fast, very often it is just dumb decision and short vision.

 

[Pdubs]

I rented a M3 for few days and it just a mixed bags, lots of these moving everything to the MCU is just not intuitive and takes a while to figure out. I picked up the car from the airport and discovered the steering wheel heat is too hot, how to I control it while driving? Do I search while driving, google it, I tried voice command "turn off steering wheel heat" and thankfully it worked.

Next, I need to move 2 cars in the driveway, since I will be blocking someone for few seconds, I want to turn on the hazard light, where is it? Ahh, google helped me, what if I am at an accident and the adrenaline is rushing my mind, can I still calm enough to google it ?

Then, the windshield wiper, it won't turn on intermittent and the stupid AI algorithm doesn't work, so voice command works but do I have to keep on saying turning on turning off ... Ahh RTFM, is one press or long press, stalkless is controlled by the MCU. I can pick up any ICE rental and expect them to work similarly across all brands. I have never seen a car that is so bad with windshield wipers since day one, too slow in heavy rain, intermittent pace is not good, and now completely broken, turn on randomly, hard to control while driving ...

There is one time after they changed the UI and I couldn't figure out how to defog the windshield while driving with FSD, it is 80% foggy and I have to quickly stop in a safe side road to google it. This is something that I rarely use and only find out the poison when I needed it.

For FSD, it works reasonably well on the interstate, the lane changing decisions however sometimes doesn't make sense, why changing lane when it is about to exit, why move to the slowest lane when all the lanes are similar pace and don't need to exit in the next 10 miles, why not move to the left most lane when there is a chance, the exit is only a mile or less away and lots of traffic ahead. It used to when I signaled it, it will change lane even with FSDb, but now it only works 20% of the time, other time just leave the blinker on and has to manually cancel it.

On the city, I think it depends on vision more than gps now, it always drift into turn lane, and on some roads not centering correctly, sometimes driving too fast, very often it is just dumb decision and short vision.

…

 

[JulienW]

Got a Notification from Stats of a new Software 23.27.10. Could this be 11.4.8? If so we still stay behind (nothing new about that).

 

[TK211X]

Anyone who wants to go go to the latest FW and has subscription can just let it expire for one hour, download latest, and resub. You’d get thrown back to FSD 11.4.4 though.