FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Ramphex]

Agreed, I am devastated not having a video game and a fart speaker. Those bastards promised!

Hey man, everyone likes what they like. I don’t complain about you going home and mounting a massive wall banger, and I would appreciate if you didn’t complain about my love for Steam (which I can’t get, because I got a crippled infotainment system instead of what I was promised). Just because YOU don’t care about Steam, doesn’t mean Tesla doesn’t owe their customers what they promised.

 

[sleepydoc]

Technically not quite true, given the same input it will always generate the same output .. but predicting the output for a novel input is hard. What makes them "squishy" is that the input is rarely identical.

That’s what I’ve chalked some of the ‘randomness’ to subtle variations in shadows, angle of approach, etc. no idea if it’s true or not, and ultimately the system needs to come up with the correct route for all the different levels of ‘squishiness.’

 

[LowlyOilBurner]

11.4.2
Anyone else have the car randomly swerving into the oncoming lane at 60mph? Or off to the shoulder for no reason?

Went mtb biking this afternoon, but at a place that’s like 40miles away, with my bike on my hitch mount upright carrier. The visualization shows the bike/carrier as a blob or dust cloud behind the car. The only thing I can think, is it’s seeing this (bike rack blob) as an obstacle in the road? But if you look at the blue planner, it’s showing the swerve prior to it happening.

I’ll have to see what happens tomorrow, when the rack is off. Strange, because I ride almost everyday, but at a location that’s only like 3miles from home, I’ve never had this behavior on that short drive, but maybe the long drive has something to do with it. It’s pretty scary when it happens, I’ll tell that to anyone who will listen.

Anyone else having this issue?

 

[heapmaster]

The newest software is *sugar*. I’m constantly having issues. I’ve tried two scroll wheel restarts, formatting USB, let the car turn off and turn it back on.

Randomly throughout a drive the side cameras stop working:
View attachment 943452

Visualizer also stops working randomly:
View attachment 943453

Sentry mode randomly crashes while away from the car:
View attachment 943454

Pretty much restarting the car several times a day now.

EDIT: Auto wipers also stop working when this happens, guess the entire camera system crashes.

When you rebooted, did you make sure to unplug all USB devices prior to performing the reboot? Having them in can prevent some sub systems from shutting down.

 

[Ramphex]

When you rebooted, did you make sure to unplug all USB devices prior to performing the reboot? Having them in can prevent some sub systems from shutting down.

Nope, I’ll try that today though. Thanks.

 

[LowlyOilBurner]

Still does random lane changes. I’m not impressed.

Yep. Every time I try to have it drive me to my office, it wants to send me to Erie PA, west bound on 90, instead of East bound like it has since I’ve had a Tesla with NoA. Very frustrating.

 

[heapmaster]

Nope, I’ll try that today though. Thanks.

Let us know, if that doesn't work, I would do the full shut down with USBs unplugged:
1. Release the brake pedal if pressed
2. Go to the main screen, bring up the Safety option (could be under Service instead) and select "Power Off".
3. After a while the screens will have gone out, the interior light will have gone off and the system are shutting down.
4. If you can hear the car, you may still hear some background noises from the car. If this is the case wait until the car is quiet.
5. Wait a minimum of 5 minutes while in the car with the door closed and don't touch anything or open the door.
6. Press the brake pedal to reawaken the car.

 

[Ramphex]

Let us know, if that doesn't work, I would do the full shut down with USBs unplugged:
1. Release the brake pedal if pressed
2. Go to the main screen, bring up the Safety option (could be under Service instead) and select "Power Off".
3. After a while the screens will have gone out, the interior light will have gone off and the system are shutting down.
4. If you can hear the car, you may still hear some background noises from the car. If this is the case wait until the car is quiet.
5. Wait a minimum of 5 minutes
6. Press the brake pedal to reawaken the car.

I’ve tried that, just didn’t think of unplugging the drive. Thought it would power down with the car, but I guess there’s a possibility that something might stay on if a USB interface is detected.

 

[heapmaster]

I’ve tried that, just didn’t think of unplugging the drive. Thought it would power down with the car, but I guess there’s a possibility that something might stay on if a USB interface is detected.

Okay, also turn off the Bluetooth connection on your phone while you reboot just in case.

 

[Ramphex]

Okay, also turn off the Bluetooth connection on your phone while you reboot just in case.

Jeez, should I get the air out of my tires too?

 

[JulienW]

Just a quickie. Before today and since 11.4.2 I have had 3 trips (6 drives) and 3 of the drives were p and 3 were j. Today I decided to go for a run out of the city near Arabia Mountain. Lots of nice PATH trails (foundation that builds trails in ATL). I live in midtown/Atlantic Station (inner city) and Arabia Mountain is just outside of the Perimeter (outer loop) in east-southeast. The trail head I started at was the Stonecrest Library and about a 25 miles drive. Pulled out of deck and took about 3 blocks to lock GPS and engaged. It drove me the full distance (city streets/Interstate/Interchange/Interstate/construction detour/into the parking lot) without a single disengagement.

Was amazed and thought that well never happen again (until uncle Elon turns on my robotaxi ).

After nice 8 mile run got back in car and engaged FSD Beta in Stonecrest Library parking lot. And I’ll be damned if it didn’t fully (other than some interventions for speed) drive all the way back to my parking deck. And this was later (midday) when the traffic was more ATL style.

I’m still in shock since that was 50 miles from the middle of a city to the burbs and back.

I’ll probably get in the morning to go to the gym and have disengagements and cursing but today was and almost unreal experience for me.

 

[rlsd]

Just a quickie. Before today and since 11.4.2 I have had 3 trips (6 drives) and 3 of the drives were p and 3 were j. Today I decided to go for a run out of the city near Arabia Mountain. Lots of nice PATH trails (foundation that builds trails in ATL). I live in midtown/Atlantic Station (inner city) and Arabia Mountain is just outside of the Perimeter (outer loop) in east-southeast. The trail head I started at was the Stonecrest Library and about a 25 miles drive. Pulled out of deck and took about 3 blocks to lock GPS and engaged. It drove me the full distance (city streets/Interstate/Interchange/Interstate/construction detour/into the parking lot) without a single disengagement.

Was amazed and thought that well never happen again (until uncle Elon turns on my robotaxi ).

After nice 8 mile run got back in car and engaged FSD Beta in Stonecrest Library parking lot. And I’ll be damned if it didn’t fully (other than some interventions for speed) drive all the way back to my parking deck. And this was later (midday) when the traffic was more ATL style.

I’m still in shock since that was 50 miles from the middle of a city to the burbs and back.

I’ll probably get in the morning to go to the gym and have disengagements and cursing but today was and almost unreal experience for me.

View attachment 943649View attachment 943650View attachment 943651View attachment 943652View attachment 943653

Did the car go thru the narrow road with the post in the middle? And the wooden bridge?

 

[JulienW]

Did the car go thru the narrow road with the post in the middle? And the wooden bridge?

Those aren't roads, they are the running/bike/hiking PATH trails. The posts are to keep any motor vehicles from driving on the PATH trails. The trails are only about 8" wide so a car would "fill them up" . Just posted the pics for the hell of it and are unrelated to my drive. I always take lots of pics when I run (Film is cheep nowadays ). Just most of my runs are in the city amongst the buildings.

 

[Ramphex]

In today’s episode of the newest Junk shenanigans…..

Dude, where’s my dock?!

 

[rlsd]

Those aren't roads, they are the running/bike/hiking PATH trails. The posts are to keep any motor vehicles from driving on the PATH trails. The trails are only about 8" wide so a car would "fill them up" . Just posted the pics for the hell of it and are unrelated to my drive. I always take lots of pics when I run (Film is cheep nowadays ). Just most of my runs are in the city amongst the buildings.

I would try it with FSD.

 

[FSDtester#1]

In today’s episode of the newest Junk shenanigans…..

Dude, where’s my dock?!

That's what she said! ... oh sorry, I thought you said something else.

 

[JulienW]

I would try it with FSD.

You will have to wait on Elon to implement 2 wheel driving to get between the posts. Also don't think a 2 toner Tesla would last long on a boardwalk bridge.

 

[JulienW]

....and if Tesla ever becomes completely literate.

 

[rlsd]

You will have to wait on Elon to implement 2 wheel driving to get between the posts. Also don't think a 2 toner Tesla would last long on a boardwalk

Tesla will add a rocket to lift the car up where there is an imminent accident. It will not work in tunnels though.

 

[EVNow]

Tesla will add a rocket to lift the car up where there is an imminent accident. It will not work in tunnels though.

Or when the accident is imminent.