FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[SidetrackedSue]

So I had my first couple of "incidents" with 11.4.9 driving home last night

1. I got a hard brake on the highway - I was in the passing lane, and just approaching a truck on the right to pas it. It seemed to get confused and hit the brakes HARD. I saw it put up a stop line that appeared, disappeared, reappeared. I was able to hit the accelerator to "fix it" but it was very alarming to say the least.

Did the truck have any red sign on the back of it (for hazardous goods or some sort of non-standard 'stop' sign to discourage tailgaiting/draughting? Because you say the display showed a stop line, it makes me wonder what it thought it was seeing.

 

[Dewg]

Do you still have to be outside a Service Center connected to their wifi to redeploy the software? We had to do this once back in Jan 2022. It wasn't as bad as it could have been for us because the nearest Service Center is only 15 mins away. But it still took over 60 mins round trip to redeploy the software update that had issues. This post explains how to redeploy the firmware.

Right now we get frequent "autosteer aborted for system error" messages while using FSD Beta. Often the option to use TACC or FSD Beta will not return until the USB for Sentry Mode/TeslaCam is removed for some time. Sometimes a two scroll wheel reboot is required. So there is something about pulling the USB that affects the car.

No, you can reinstall at home now. No need to be near a service center.

 

[sleepydoc]

Was the truck infringing on your lane or moving closer to the line diving you lane from its lane at the time of the hard brake? I've experienced that FSD Beta can be alongside a semi truck/trailer & if the semi angles toward its lane FSD Beta will slam on the brakes even though the truck is still in its own lane.

I’ve noticed with the last few versions that FSD seems to have more difficulty determining if cars are in its lane or in the adjacent lane. There’s a lot more slowing down when approaching an intersection with cars waiting in the left turn lane and the like.

 

[PR0FESSOR]

A few notes on 2023.44.30.6 on HW4 MSP running FSD 11.4.9

- Auto wipers do not work normally. Sometimes in pouring rain they do not turn on at all, and sometimes, if I toggle back and forth from manual mode to auto mode, I can get them to work… and sometimes not.

- It blew right through the stop sign (completely normally placed) that it usually (but not always) blows through.

- I was in the high speed lane and a car was passing me in the slow lane, and just as the car was past even (a bit ahead) it applied the brakes. Thank goodness there was no vehicle behind me.

- I am getting nagged, on average, every 30 seconds, even if I am looking straight forward all the time. Nags are a function of velocity; the faster I am going, the more frequently I am nagged.

- I get nagged to touch the steering wheel, even when my hand is resting on it, just like it has alway been. I literally have to pull down on the opposite side (the side I am not resting my hand on) to get the nag to go away. I have never seen this behavior before.

- Disney Plus Program is gone.

+ Park Assist

+ Dry wipes are gone (so far)

Joe

 

[Dewg]

It blew right through the stop sign (completely normally placed) that it usually (but not always) blows through.

Does the stop sign show up on visualization?

 

[Pdubs]

So I had my first couple of "incidents" with 11.4.9 driving home last night

1. I got a hard brake on the highway - I was in the passing lane, and just approaching a truck on the right to pas it. It seemed to get confused and hit the brakes HARD. I saw it put up a stop line that appeared, disappeared, reappeared. I was able to hit the accelerator to "fix it" but it was very alarming to say the least.
2. it had a difficult time figuring out when to merge into the off ramp lane a couple of times. Basically it did a hard adjustment either early or late. Wasn't smooth.

Anyway, it was at night, and there was a lot of fog. I suspect that was related.

Yeah man. I’ve been dealing with your #1 since the last update. FSDWTF ftw. Every time I come up on a big truck on my right, my foot is hovering over the accelerator.
Not related to night or fog. It just fails hard in that circumstance.

 

[Pdubs]

A few notes on 2023.44.30.6 on HW4 MSP running FSD 11.4.9

- Auto wipers do not work normally. Sometimes in pouring rain they do not turn on at all, and sometimes, if I toggle back and forth from manual mode to auto mode, I can get them to work… and sometimes not.

- It blew right through the stop sign (completely normally placed) that it usually (but not always) blows through.

- I was in the high speed lane and a car was passing me in the slow lane, and just as the car was past even (a bit ahead) it applied the brakes. Thank goodness there was no vehicle behind me.

- I am getting nagged, on average, every 30 seconds, even if I am looking straight forward all the time. Nags are a function of velocity; the faster I am going, the more frequently I am nagged.

- I get nagged to touch the steering wheel, even when my hand is resting on it, just like it has alway been. I literally have to pull down on the opposite side (the side I am not resting my hand on) to get the nag to go away. I have never seen this behavior before.

- Disney Plus Program is gone.

+ Park Assist

+ Dry wipes are gone (so far)

Joe

I can repeat the cancellation of a nag by hovering both my hands directly over the yoke on the top corners of it. Not sure why that cancels the nag. I mean, if the cameras see that, why can’t it see your eyes or hands on other parts?

 

[sleepydoc]

- I get nagged to touch the steering wheel, even when my hand is resting on it, just like it has alway been. I literally have to pull down on the opposite side (the side I am not resting my hand on) to get the nag to go away. I have never seen this behavior before.

In my limited driving the torque requirement hasn't changed but the have changed the sensitivity in the past so it's possible they did so again.

- Disney Plus Program is gone.

reportedly there's a bug that hides it if you haven't used it in a while. Logging in to Disney Plus via the browser reportedly fixes the issue.

I can repeat the cancellation of a nag by hovering both my hands directly over the yoke on the top corners of it. Not sure why that cancels the nag. I mean, if the cameras see that, why can’t it see your eyes or hands on other parts?

Does it work if you only hover your right hand? That's potentially visible to the cabin camera. I'd be surprised if the left hand is and I haven't heard of any of the wheels/jokes having touch/proximity sensors.

 

[Pdubs]

In my limited driving the torque requirement hasn't changed but the have changed the sensitivity in the past so it's possible they did so again.

reportedly there's a bug that hides it if you haven't used it in a while. Logging in to Disney Plus via the browser reportedly fixes the issue.

Does it work if you only hover your right hand? That's potentially visible to the cabin camera. I'd be surprised if the left hand is and I haven't heard of any of the wheels/jokes having touch/proximity sensors.

I’ve tried it with just my right hand and it was a no go. Worked with both, but if it was late in the warning and it was blinking fast ready to cut off, seemed only a torque would work.

 

[Twiglett]

Do you still have to be outside a Service Center connected to their wifi to redeploy the software? We had to do this once back in Jan 2022. It wasn't as bad as it could have been for us because the nearest Service Center is only 15 mins away. But it still took over 60 mins round trip to redeploy the software update that had issues. This post explains how to redeploy the firmware.

Right now we get frequent "autosteer aborted for system error" messages while using FSD Beta. Often the option to use TACC or FSD Beta will not return until the USB for Sentry Mode/TeslaCam is removed for some time. Sometimes a two scroll wheel reboot is required. So there is something about pulling the USB that affects the car.

It's worth checking out the official video, lots of useful info.

Service Mode

 

[Tronguy]

So, regarding Autopark, and who's got it:
Two cars in the garage:

• 2021 MY AWD LR with (paid for) EAP, SW 2023.44.5.1. It appears to have USS all over, and they show up when one drives about.
• 2023 M3 AWD LR with (paid for and transferred to this car) FSD 11.4.9, SW 2023.44.5.1. No USS.

The MY was taken to a local parking lot. Driving past both just naked parking lines or with cars on either side, flashes up the "P" symbol, so long as one is driving slow. Hit the P, put it in Reverse as requested, and it does it's pretty-much-normal thing as regards parking. Slower than a human.
On the screen, Summon appears to be present and accounted for. In the app, Summon is present. Didn't see "Smart Summon", but I'm not right next to the car at the moment. Summon appears in the NAV tab of the car. The screen does not show the fancy visualization with all the funky colors and shapes as one gets closer and farther from things.

The M3 was also taken to another parking lot. It does have the fancy visualization, both in and outside the garage, and shows the stripy lines, light posts, and all that. No "P" symbol anywhere, no Summon menus whatsoever in the car or on the app.

The SO is pretty sure that the parking stuff disappeared at some point; it's back, now.

Conclusion: Those that have USS and 44.5.1 get Summon in all its glory, including the parking.

Those that don't have USS and are vision, only, get the fancy graphics and no summon.

 

[hybridbear]

It's worth checking out the official video, lots of useful info.

Service Mode

Thanks, I had not seen the video before. Sadly, almost not of the features shown in the video are available for our S. They all seem to be for newer cars only.

 

[Mardak]

Yes, I've noticed on 11.4.9 (coming from 11.4.7.3) that rear cars are highlighted blue as well now, especially during merges

Got a couple examples of merges on left and right sides with 11.4.9:

 

[powertoold]

11.4.9 is amazing imo, it's by far the best version for me, in terms of disengagement-free drives, it almost changes lanes more consistently and safely than I can on highway, it now makes better merging decisions, it gives me much more confidence in its decision-making in every way, there are fewer puzzling maneuvers

There are still a couple annoying things, but vision-based fsd seems close

 

[Twiglett]

I've found the best for our S is a combination of the mirrors moving down to positions we've saved along with the cameras. I like the fender cameras to confirm spacing to the garage pillar we park next to at our apartment building. But the mirrors are the best for avoiding curbs when parallel parking or parking at the edge of a parking lot. And the USS are unbeatable for parallel parking & getting close without hitting any other vehicles or obstacles.

With you on that one. Only once did I make the mistake of relying on the cameras for curbs, managed to drive the wheel covers into the curb twice in one night. You'd think I'd learned the lesson the first time in one day, but apparently not

 

[rlsd]

The show is completely downloaded and formatted on the USB. I did a dry run in the closed garage last night... went without a hitch.
Just to piss more people off, I have included such shows as Thriller, James Bond, and Kitt from Knight Rider. Not very festive, but that’s the point!

Didn't think about Bobbit protection, thanks for the pro tip!
Pun intended...

When you make your own light show song, do the doors roll up and down and does the trunk flap accordingly to the music?

 

[MP3Mike]

When you make your own light show song, do the doors roll up and down and does the trunk flap accordingly to the music?

If you program them to.

 

[old pilot]

By 2027, Ark Invest believes Tesla will be generating over $1 trillion in annual revenue, with 44% coming from the robotaxi business. That will translate into $354 billion in earnings before interest, tax, depreciation, and amortization (EBITDA), of which 64% will come from robotaxis and a further 3% from human-driven ride-hailing within Tesla's network.

Tesla Makes Money Selling Electric Vehicles, But 67% of Its Earnings Might Soon Come From This Instead

Cathie Wood's Ark Investment Management anticipates a significant change to Tesla's business model by 2027.

currently.att.yahoo.com

 

[Knightshade]

By 2027, Ark Invest believes Tesla will be generating over $1 trillion in annual revenue, with 44% coming from the robotaxi business.

I believe that if you'd put your $ into an S&P500 Index fund 5 years ago instead of ARKK you'd have been almost twice as well off.

And my belief is actually based in facts, so I've got that going for me.

 

[powertoold]

11.4.9 did this for the first time yesterday

It patiently waited 5-7s for the left turn lanes to clear before proceeding into the intersection on a green light, and it actually proceeded right at the moment it was clear!

This was a routine disengagement for me on prior versions