FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Phlier]

I said the same with the recall and the response was, "No way there are that many cars on Beta" and then people ran with their own narratives.

What??? On TMC?? Ppfftt. Nevah!

 

[JulienW]

Any US car with FSD could potentially request the beta version, hence all FSD equipped cars were covered by the recall.

At the time FSD Beta was ONLY included and added in a "special" segregated and older branch of software (like 11.4.x is now) and was NOT included in regular software updates the way 11.3.6 is now. If you requested FSD Beta you then had to wait (often weeks) until the "special" FSD Beta branch of software passed your current software version and then you would get the older "special" software version that had FSD Beta added in to downloaded and install on your car.

So cars that didn't contain the "special" FSD Beta software branch were NOT included in the recall since they didn't have any FSD Beta software included at all. The recall was only for people who actively requested and waited on the "special" software version with FSD Beta to be downloaded to their car.

 

[willow_hiller]

v11.4.6 is likely paused at this point:

https://twitter.com/x/status/1686210210870984705

 

[uscbucsfan]

v11.4.6 is likely paused at this point:

https://twitter.com/x/status/1686210210870984705

Likely for the best. Turns after stop signs into traffic have been a serious regression safety wise. 11.4.X overall has been far more dangerous than 11.3.X. I assume that's a large reason for it to be gated behind 2023.7.X

 

[legendsk]

No, I agree. It sometimes stops well short of intersections and poses well beyond crosswalks and into oncoming traffic. This form of creeping sends off the distracted driver behavior message to other vehicles. Unknown why the team added it other than to try a crutch.

I think the last two updates have been a swing and miss which isn't a good sign for a FSD product that is so lacking in so many areas.

The really troubling component to me is:
After stopping way short of the stop sign,
It will very, very, fearfully, nervously creep and creep and creep to see if it safe to make the left turn,
Then when it decides it has crept far enough, it will stop and think about it for a long while before making a decision, { unfortunately while it was thinking, the approaching traffic (doing 75mph), has approached from 3/4 mile away to 50 feet }
Then without any warning at all, it will snap the steering wheel a full turn and start spinning the tires throwing gravel, apparently to get out of the intersection as fast as possible. FSDb has become so capable it would be great to let it drive, except it has become so incompetent and unpredictable it is too dangerous to let it drive at all if there are any other vehicles or obstacles in sight - which is mostly.

 

[zoomer0056]

The really troubling component to me is:
After stopping way short of the stop sign,
It will very, very, fearfully, nervously creep and creep and creep to see if it safe to make the left turn,
Then when it decides it has crept far enough, it will stop and think about it for a long while before making a decision, { unfortunately while it was thinking, the approaching traffic (doing 75mph), has approached from 3/4 mile away to 50 feet }
Then without any warning at all, it will snap the steering wheel a full turn and start spinning the tires throwing gravel, apparently to get out of the intersection as fast as possible. FSDb has become so capable it would be great to let it drive, except it has become so incompetent and unpredictable it is too dangerous to let it drive at all if there are any other vehicles or obstacles in sight - which is mostly.

A stop sign where cross traffic is going 75mph is a disaster waiting to happen!!! I would be fearful at that intersection, whether trying to cross or waiting for someone trying to cross in my 75mph path.

 

[kabin]

A stop sign where cross traffic is going 75mph is a disaster waiting to happen!!! I would be fearful at that intersection, whether trying to cross or waiting for someone trying to cross in my 75mph path.

Yes, at times even 45mph is way beyond FSD's current capabilities but apparently these are scenarios humans manage every day. The faster roadways should be straighter and provide more distance visibility but it's all irrelevant if FSD can't process and respond in time.

 

[JulienW]

Likely for the best. Turns after stop signs into traffic have been a serious regression safety wise. 11.4.X overall has been far more dangerous than 11.3.X. I assume that's a large reason for it to be gated behind 2023.7.X

Maybe Tesla should just go back to 11.4.3 since it seemed the most stable of the branch and that ain't saying much.

 

[Genie]

A stop sign where cross traffic is going 75mph is a disaster waiting to happen!!! I would be fearful at that intersection, whether trying to cross or waiting for someone trying to cross in my 75mph path.

That’s very true, but where I live the county roads are posted 50-55 and most traffic is about 60. Very few traffic lights or roundabouts- most roads just have a stop sign at the highway. I Always disengage before the stop, it is too nerve racking and dangerous to watch it do its stop sign hokey pokey.

 

[uscbucsfan]

Maybe Tesla should just go back to 11.4.3 since it seemed the most stable of the branch.

11.3.5 and 11.3.6 have been there most stable branches in the last year. All 11.4 have been a regression as far as miles per disengagement.

 

[JulienW]

11.3.5 and 11.3.6 have been there most stable branches in the last year. All 11.4 have been a regression as far as miles per disengagement.

Yes, I ambivalent if given the choose whether I would switch to 26.x/11.3.6 or continue to suffer with 7.x/11.4.x hoping 11.5.x will be an improvement and hopefully on a newer software branch.

Of course as a buyer I'm stuck and can't switch so suck it up and suffer I guess.

 

[rlsd]

Am I the only one who still thinks that creeping can sometimes be extremely unnatural and unhumanlike? It’s downright scary sometimes still. Humans don’t tend to creep so far forward when cars are obviously coming, however FSD doesn’t care weather or not it can see cars, it just creeps. Sometimes it’s jumpy on the brake pedal which only just makes things worse.

So creeping is definitely nowhere close to being solved yet really, but recent updates haven’t really addressed it at all. All of this because of the idiotic b pillar camera location

Yes. Sometimes the creeping seems too aggressive that forces me to disengage. The creeping may not cause accident but the way it moves the car quickly to the busy main street makes me feel nervous. We need to be able to know what FSD is thinking: does it creep only or is it ready to make turn. We need a better way to know touchscreen

Likely for the best. Turns after stop signs into traffic have been a serious regression safety wise. 11.4.X overall has been far more dangerous than 11.3.X. I assume that's a large reason for it to be gated behind 2023.7.X

Looking forward to 11.4.8. 11.4.7 will be skipped too.

 

[legendsk]

A stop sign where cross traffic is going 75mph is a disaster waiting to happen!!! I would be fearful at that intersection, whether trying to cross or waiting for someone trying to cross in my 75mph path.

Truely. But so common as to be the normal here. A very good friend was killed at one of those intersections not long ago. I talked to the Highway Patrolman who was in charge of the accident scene. He said, "I've covered a million of these and can just look at the scene and it is clear what happened. Distracted driving. I've talked to her and have no doubt he pulled up to the stop sign and stopped. She started ragging on him about something and he looked over at her while he stepped on the gas and just pulled out in front of the semi rock truck. The poor guy driving the semi could see the car pull up and stop and relaxed for a second thinking they were really stopped and then they weren't, but it was too late. Not his fault at all and he feels terrible, but that's what distracted driving causes."

And you're absolutely correct, every one of those intersections is a fearful place, whether you're doing 75 down the highway or stopped at the stop sign watching for an opportunity with which you feel comfortable, or waiting behind someone who isn't having much luck finding a gap they are comfortable with.

 

[Dewg]

Remember that many of these intersections were designed and built a long time ago. Typical passenger cars weren't as powerful nor was the societal norm to drive faster 20+ years ago.

 

[swedge]

Is the FSD development team in trouble? TeslaFi shows 11.4.6 to have stalled. 11.4.5 didn't even show up at all there.

So in reality this all looks more like de-ja vu than Tesla team in trouble.

v11.4.6 is likely paused at this point:

https://twitter.com/x/status/1686210210870984705

I still hope JulienW was right. I'm way past ready for a new version. And for a new merge with the production branch.

 

[JulienW]

I still hope JulienW was right. I'm way past ready for a new version. And for a new merge with the production branch.

I don't think he knows what the hell he is talking about since it does appear 11.4.6 is dead. He may say differently but don't listen to his dumb a$$ because it looks like we are going to be waiting on 11.4.7.

[crazy a$$ conjecture]My suspicion is problems getting HW4 and HW3 balanced and stable together in the same software is what is causing the 11.4.x problems.

 

[Rocinante777]

From the "other companies can screw this up, too" department:

I had a Cruise taxi almost hit me last week in downtown San Francisco. It was shortly before 10 pm. Intersection of Mason and O'Farrel. I was most of the way across Mason with an active walk signal when the Cruise came eastbound on O'Farrel and made a right turn across the crosswalk without slowing down. I basically jumped to the curb.

I'm not sure if it would have actually hit me, but if it missed it would have still come way closer than any car should come to a pedestrian without substantially slowing. Inches, not feet. It either completely failed to see me or it has a very warped idea of personal space for a pedestrian.

 

[Mardak]

Who gets to decide how much 11.4.x training compute gets allocated away from v12?

https://twitter.com/x/status/1686513363495346178

There's probably business goals such as getting some 11.4.x stable enough to deploy to new HW4 vehicles, but that would come at the cost of delaying v12 alpha testing then early access then eventual wide deployment to reach other goals. Seems likely there'll be regressions in early alpha versions that will take some time to work through either by restoring some control code or collecting enough training data to generally solve regressions.

 

[jeewee3000]

Who gets to decide how much 11.4.x training compute gets allocated away from v12?

Been wondering that myself since Elon tweeted there already is a v12 alpha build. Long term it seems more efficient to drop 11.x.x entirely and focus fully on v12.

 

[JulienW]

Who gets to decide how much 11.4.x training compute gets allocated away from v12?

There's probably business goals such as getting some 11.4.x stable enough to deploy to new HW4 vehicles, but that would come at the cost of delaying v12 alpha testing then early access then eventual wide deployment to reach other goals. Seems likely there'll be regressions in early alpha versions that will take some time to work through either by restoring some control code or collecting enough training data to generally solve regressions.

V10 lasted for 1½ years and Elon started V11 talk about work on it a year before its release (look at the date of the start of this thread). V11 is still less than 5 months old. So if history is a guide we may not see V12 for about another year. Hell at the rate V11.4 is (not) progressing we may not see a stable 11.4.x released for another month or so.