FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Porkalletta]

Last night, I was driving back home from work on a rural highway, around 90km/h. It's a 4-lane road and I was driving on the right lane with FSD on, and my hand is on the wheel as well...

It was at dusk (auto lights on), so things weren't very visible. I was obviously paying extra attention. Then all of a sudden, the car started to swerve towards the left lane. I couldn't figure out why, and then, WOW a person with a buggy (not sure what was in it) was walking on the line towards me! I only saw her when we were very close. The car avoided hitting her because we were already partially off from the center. Crazy that a person would even think about walking at that spot at that time of the day!

I'm not going to claim FSD saved an accident, but clearly, it saw more than I did, at least under the above said condition and acted before what I would have done.

Something similar happened to me. Car swerved left. I didn't know why until I saw a guy on a bike with no lights whatsoever riding on the right side of a dark 45 mph 2 lane road. I didn't see him until I was right on him. I believe FSDb saved an accident.

 

[tmoz]

Something similar happened to me. Car swerved left. I didn't know why until I saw a guy on a bike with no lights whatsoever riding on the right side of a dark 45 mph 2 lane road. I didn't see him until I was right on him. I believe FSDb saved an accident.

It's good to hear tales of FSD potentially saving someone vs potentially killing someone.

 

[powertoold]

Another video. Version 11.4.6 looks like major regression.
FSD 11.4.6

Everything I've seen for 11.4.6 so far looks like an improvement over 4.4.

This video in Chicago seems like a regression, but the two major issues in that video involved making unprotected lefts out of a parking lot. Parking lots in general have been hit or miss with any version.

 

[D Good]

Another video. Version 11.4.6 looks like major regression.
FSD 11.4.6

There is genuinely no way it could be any worse than 11.4.4.

 

[zazim]

I got my first strike driving home from work very late. I was paying attention with my hands on the wheel, but a strike for not paying attention. There were so many deer on the road the rest of the trip (they think they o‘em the roads that late). I really missed FSD for the first time. It does a good job with animals.

 

[Kablooie]

I've discovered that Tesla can quietly update software between major FSD system updates. This could account for why some days the car behaves well and other days it seems to be drunk. This morning I took a route I take every day but was surprised to see that the Tesla routed the car down streets it has never used before. I checked the software version, which was the same, 11.4.4, but a message said that software was up to date as of this morning.

This indicates that there are software updates happening behind the scenes that can affect the car's behavior. According to GreenTheOnly, different cars could be getting different updates. Not everyone gets the same thing. This could account for some people having good experiences and others not so with the same system update.

 

[MP3Mike]

This morning I took a route I take every day but was surprised to see that the Tesla routed the car down streets it has never used before. I checked the software version, which was the same, 11.4.4, but a message said that software was up to date as of this morning.

This indicates that there are software updates happening behind the scenes that can affect the car's behavior.

No, that likely means there was traffic on the normal streets for that destination, so it chose a different route that was quicker. (Routing is normally done online on Tesla servers, not on your car.)

 

[swedge]

No, that likely means there was traffic on the normal streets for that destination, so it chose a different route that was quicker. (Routing is normally done online on Tesla servers, not on your car.)

Or maybe the car just wanted a change of scenery. ;-)

 

[MP3Mike]

Or maybe the car just wanted a change of scenery. ;-)

I have wondered sometimes if the online routing will sometimes use streets where it doesn't have much data so that it can collect samples from that area.

 

[KLam]

I've discovered that Tesla can quietly update software between major FSD system updates. This could account for why some days the car behaves well and other days it seems to be drunk. This morning I took a route I take every day but was surprised to see that the Tesla routed the car down streets it has never used before. I checked the software version, which was the same, 11.4.4, but a message said that software was up to date as of this morning.

This indicates that there are software updates happening behind the scenes that can affect the car's behavior. According to GreenTheOnly, different cars could be getting different updates. Not everyone gets the same thing. This could account for some people having good experiences and others not so with the same system update.

Actually, if there is enough memory, FSD beta can contain many different NNs which can be switched in and out by command to test out different implementations. This would increase the usefulness of each update cycle.

 

[uscbucsfan]

I've discovered that Tesla can quietly update software between major FSD system updates. This could account for why some days the car behaves well and other days it seems to be drunk. This morning I took a route I take every day but was surprised to see that the Tesla routed the car down streets it has never used before. I checked the software version, which was the same, 11.4.4, but a message said that software was up to date as of this morning.

This indicates that there are software updates happening behind the scenes that can affect the car's behavior. According to GreenTheOnly, different cars could be getting different updates. Not everyone gets the same thing. This could account for some people having good experiences and others not so with the same system update.

I believe Green confirmed there aren't shadow updates, but map data is pushed with every drive. Your car saying it was up to date is the same message that it will say every morning. That means nothing.

 

[JulienW]

....but map data is pushed with every drive.....

This (Map Data Deltas) PLUS the Navigation engine is dynamic and makes different "decisions" based on current conditions on each drive.

 

[jebinc]

Still trying to get this junk removed from my car. Tesla is now resisting its removal - Exchange with SC attached. You can’t make this stuff up.

 

[PACEMD]

Be careful what you say. If Tesla doesn't like you, or if you complain too much, they have been known to put a virus in your software. "You think THAT'S junk? Here, try this one."

 

[clydeiii]

Still trying to get this junk removed from my car. Tesla is now resisting its removal - Exchange with SC attached. You can’t make this stuff up.

View attachment 959838

Most likely SC has no idea how to do this.

 

[jebinc]

Most likely SC has no idea how to do this.

Really, common now…. Is it also unlikely that those reading our many requests sent to both [email protected] and [email protected] that the reason for inaction and no response is because they are incapable of such basic actions? In both cases, I think not.

 

[jebinc]

Be careful what you say. If Tesla doesn't like you, or if you complain too much, they have been known to put a virus in your software. "You think THAT'S junk? Here, try this one."

Can’t be any worse (for me). I already have the dreaded refreshed S vibration virus.

 

[zoomer0056]

Can’t be any worse (for me). I already have the dreaded refreshed S vibration virus.

Maybe it's time for a new car. This one has problems that can only be fixed by selling. Oh, wait .... if the car has vibrations then the FSDb will follow owner to the new car ... forever. Bwa ha ha. just kidding. If it was a PC, we would just wipe it and reinstall. Tesla could do this.

 

[jebinc]

Maybe it's time for a new car. This one has problems that can only be fixed by selling. Oh, wait .... if the car has vibrations then the FSDb will follow owner to the new car ... forever. Bwa ha ha. just kidding. If it was a PC, we would just wipe it and reinstall. Tesla could do this.

Would have sold the car if it had not been for the 60%+ depression.

 

[dgatwood]

I got a lot of the drunken chimp behavior today. Much of it on the same roads I drive on every day. Different behavior today for some reason. Something is causing it to change its behavior. The software hasn't changed, the lighting hasn't changed, the temperature is about the same. Are there background map updates we don't know about? I can't figure out what is causing it to act differently all of a sudden….

In the software industry, we frequently use something called "feature flags", wherein we can turn on or off specific features. I know for a fact that Tesla uses these, or at least did use them at one time, because there was a post by GreenTheOnly that described some of those flags, including, for example, the rolling stop flag.

Typically, those flags can be pushed as part of a configuration push that is entirely separate and apart from software updates. In fact, it is quite likely that each car fetches a set of enabled feature flags on every drive.

The only question is why they keep switching it from a usable configuration to a downright homicidal one about a week after every release, over and over, without exception (at least for me so far).