FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[JulienW]

Thank you for the clarification! So what you’re saying is I’ll likely get the standard updates for park assist/other features and one day FSD will have a version that connects to those versions and then I’ll get it, and in the meantime enjoy my $15k stop light and stop sign add on.

The fastest way to get FSD Beta is to NOT install any updates. Installing updates almost guarantees that you will always be ahead of the Beta and never get it. Tesla may one day get FSD Beta on the most current branch but history says that is not the case.

 

[JHCCAZ]

Not clear to me that this would be the natural result. There is not really anything human-like about it (in spite of the weird “neural nets” anthropomorphic term).

My comment doesn't fundamentally equate the neural-net approach with human-like behavior in the resulting product. The machine learning or "Software 2.0" concept is a way of attacking a very large, complex and hence difficult problem by teaching through example and reinforcement, without having to describe and quantify millions of details in the underlying operations - details that even competent and experienced human experts don't know or find it impossible to describe using tractable language.

In my own field of analog circuit design, I lived through the progression from paper sketches and breadboards, to simulation aids and design rule checkers, to an entire software environment that became the only way to handle the design of large and/or very high performance products.

Along the way, there was a constant debate about whether the transistor and other device simulation models should have internal equations and parameters that a) represent physical reality of the semiconductor, or b) would converge faster and better if they were built as a more general-purpose "Black Box" with the appropriate ports and time-dependent operational behaviors.

In this example, I'm not suggesting that a neural net architecture would be in the black box, but the "b" approach still involves a transformation of external descriptors into a computer-friendly and mathematically stable/efficient model that produces better results - even though observation of the internal calculations would yield little insight, and look very different from the semiconductor equations of the device's electrical and thermal behavior.

As an outside but interested observer, I see the machine learning paradigm and its application to the "Software 2.0" design approach in a similar way. You have to teach it the result you want, by developing a giant optimizer that can manipulate the huge number of internal parameters and math operations - in ways that you could never hope to describe or even understand. The guts can be made out of so-called neurons, or graphs or fuzzy logic matrices or whatever is dreamed up in the future.

It can be taught to mimic humans not because the insides are so-called neurons, but because its architecture can adapt and learn incredibly complex behaviors, and can converge on and remember which ones are successful and preferred.

 

[fasteddie7]

The fastest way to get FSD Beta is to NOT install any updates. Installing updates almost guarantees that you will always be ahead of the Beta and never get it. Tesla may one day get FSD Beta on the most current branch but history says that is not the case.

Thank you for this, so I should take the update setting off of advanced for sure, and if an update comes through, ignore it. And if that’s the case, how would I know I got the FSD update if the notification for the standard update track is showing pending, or does it look different? Sorry about the questions, this is all so helpful though!

 

[Dewg]

has anyone else had any luck figuring out the problem where the AP steering wheel flashes on/off making it difficult to engage FSDb all together?
Just wondering if maybe it is related to cameras being clean, or if anyone contacted service?

I have not had that problem so far. My only recommendation would be:

1) Enter the Service Mode and force a reinstall of the existing firmware
2) Recalibrate your cameras (after cleaning them completely)
3) Set back up your settings

 

[JB47394]

Just wondering if maybe it is related to cameras being clean, or if anyone contacted service?

I have one case where it's definitely related to the width of the lane (two lane road, well-marked and -paved), and another for which I haven't figured out a cause (divided four lane road, well-marked and -paved). As mentioned, Chuck Cook found that it will not enable if you're at the edge of a unmarked road. In my case, FSDb will happily drive on the problem roads if enabled beforehand, but doesn't want to enable on those roads.

Ultimately, it isn't accidental. Tesla is intentionally not allowing FSDb to be enabled if they don't like the current driving environment. Like most things they do, it's too conservative.

 

[Tronguy]

I have not had that problem so far. My only recommendation would be:

1) Enter the Service Mode and force a reinstall of the existing firmware
2) Recalibrate your cameras (after cleaning them completely)
3) Set back up your settings

Just got back from one of my several-times-a-year trips from NJ to Boston and back on 11.3.4; and, when got home, did the update to 11.3.6.

I’ve seen the flickering you-can-go-into-FSD icon a bunch of times. Pretty sure this isn’t a matter of clean cameras. My working hypothesis is that the system is having some trouble getting, “locked on” in some sense to the environment. Poor striping on the road from wear or construction, iffy road edges from construction and/or no curbs, narrow road with good centerline stripes but with cars parked on the right so that one has to be over the yellow lines when moving straight ahead, and so on.

There seems to be something that fixes the issue: pulling down on the shifter, once, seems to get the car into TACC, at which point the icon stops flickering. But am not quite sure that’s the cure. Too many variables and the environment changing.

Once one gets into FSDb it stays there, of course, and does its job (or not) as one goes forward.

 

[JulienW]

Thank you for this, so I should take the update setting off of advanced for sure, and if an update comes through, ignore it. And if that’s the case, how would I know I got the FSD update if the notification for the standard update track is showing pending, or does it look different? Sorry about the questions, this is all so helpful though!

Bookmark this site. The little red number indicates an FSD Beta update. You need the Blue software number to be lower than your software number to get into the FSD Beta queue.

TeslaFi.com Firmware Tracker

 

[thewishmaster]

Not quite sure if this is new in 11.3.6 because I didn't get to use 11.3.4 for long enough, but it takes way too long to go from 1mph to 0mph at stop signs... adds a lot of time. 11.3.4 was already pretty bad but 11.3.6 feels nearly unusable at stop signs without overriding with the accelerator pedal

 

[fasteddie7]

Bookmark this site. The little red number indicates an FSD Beta update. You need the Blue software number to be lower than your software number to get into the FSD Beta queue.

TeslaFi.com Firmware Tracker

View attachment 928138

I’m the one lonely car with 2023.2.301. Looks like I have a ways to go before it catches up, plus I have ap4 so who knows if there will be a delay with the new cameras.

 

[JulienW]

I’m the one lonely car with 2023.2.301. Looks like I have a ways to go before it catches up, plus I have ap4 so who knows if there will be a delay with the new cameras.

That is NOT you. You must be a TeslaFi subscriber to be counted. It is likely that only a couple of % of Tesla owners are TeslaFi subscribers (I'm not either). So the numbers must be multiplied to = actual owners. Also new owners (x.x.x00) are less likely to be subscribed.

 

[kabin]

Not quite sure if this is new in 11.3.6 because I didn't get to use 11.3.4 for long enough, but it takes way too long to go from 1mph to 0mph at stop signs... adds a lot of time. 11.3.4 was already pretty bad but 11.3.6 feels nearly unusable at stop signs without overriding with the accelerator pedal

Not sure how Tesla sources the velocity data (tire/motor sensors) but would bet the lag has always been there related to normal delays from data communications and filtering noisy data. I would hope the team didn't add a bogus delay giving more processing time when stopped or at intersections.

 

[Ramphex]

Have yet to drive on Junk 7.0

 

[scottf200]

has anyone else had any luck figuring out the problem where the AP steering wheel flashes on/off making it difficult to engage FSDb all together?
Just wondering if maybe it is related to cameras being clean, or if anyone contacted service?

I've had it happen a fair amount more on v11 than v10. It seems to correlate with how well the lanes are "defined".

My recent TIP to people: least annoying way to engage (failure chime) when it is intermittently flashing is to
* pull once (TACC),
* wait briefly to make sure the grey steer wheel is still there,
* then pull a second time if it is.

 

[JHCCAZ]

but it takes way too long to go from 1mph to 0mph at stop signs...

Not sure how Tesla sources the velocity data (tire/motor sensors) but would bet the lag has always been there related to normal delays from data communications and filtering noisy data. I would hope the team didn't add a bogus delay giving more processing time when stopped or at intersections.

Certainly there's a digital filter to smooth out the numeric speedometer display. This filtering lag, plus the legally mandated ~1 mph deliberate high-side bias in the readout, created the situation that some reviewers (Dirty Tesla for one) noted: that the car on 10.69.x was making short full stops, but the speedometer readout never went all the way to zero. Although it's not clear, some people think that NHTSA folded this issue into the complaint about the "perception" of whether a full stop was being honored by FSDb.

It seems like Tesla's solution, at the moment, is to keep the car stopped for long enough that the digital filtering plus bias finally rolls down to 0 mph display for some minimum time, before the car is allowed to begin the next (painfully slow) creep phase past the stop sign.

My suggestion would be for Tesla to modify the readout filtering algorithm in the range of 0-1 -2 mph. Reduce the amount of noise filtering and reduce the lag. Also modify the bias calculation so that the display reads 0 mph essentially simultaneous with the actual full stop event.

Combine this readout adjustment with a slightly more aggressive (but still low jerk) approach to the stop line, and finally work on optimizing the creep-and-go phase. Then it can all work out legal and decent, even if it's not quite as fast as the old rolling stop.

Of course, if some of the new regressions discussed up-thread are causing the car to get confused about oncoming cross traffic and appropriate gaps, those need to be solved first.

 

[JHCCAZ]

I've had it happen a fair amount more on v11 than v10. It seems to correlate with how well the lanes are "defined".

My recent TIP to people: least annoying way to engage (failure chime) when it is intermittently flashing is to
* pull once (TACC),
* wait briefly to make sure the grey steer wheel is still there,
* then pull a second time if it is.

I've only had it be reluctant to engage FSD a couple of times, but I didn't try pausing between the two clicks. Approximately how long can you pause and still have it register as a double-click?

 

[scottf200]

I've only had it be reluctant to engage FSD a couple of times, but I didn't try pausing between the two clicks. Approximately how long can you pause and still have it register as a double-click?

It is surprisingly a long time ... and buy that I mean maybe 1-2 seconds ... I don't think I've tested how long but after TACC engages with one click then I simply do the 2nd click just as soon as I see the steering wheel is *still* there!

I found this surprisingly useful because I was annoyed with the failure chime and it is *worse* when you have a passenger(s)!

 

[PACEMD]

Ok, stop me if you've heard this one before..........hey, this is FUBARed beyond all recognition...........cut and paste, cut and paste, cut and paste...........version slash version slash version..........why I oughta..........

 

[jabloomf1230]

Chuck Cook has a video of it happening when driving too close to the right side road edge. I never experienced it before but did yesterday on 3.6 and I was in the right most lane. I wouldn't be surprised if there were other conditions that trigger it.

There's no pattern to it, except it seems common after a disengagement. It makes FSDb almost unusable.

 

[jabloomf1230]

I've had it happen a fair amount more on v11 than v10. It seems to correlate with how well the lanes are "defined".

My recent TIP to people: least annoying way to engage (failure chime) when it is intermittently flashing is to
* pull once (TACC),
* wait briefly to make sure the grey steer wheel is still there,
* then pull a second time if it is.

That's been my workaround, but it works for me about half the time.

 

[aronth5]

Just a nuance complaint on the operation of the Scroll Wheel Button for Chill/Average/Assertive. When you first press it changes to the next Up or Down. To me it should FIRST show what you are on and then change on the next press.

Hmm. This would work but it would piss people off who want to make a quick change with one click… Debatable tbh.

I agree with @Julien W. Today I was on a long highway drive and a few times only wanted to change the value of the Minimal Lane Change setting. Every time I did that I had to toggle back to the Average setting since as soon as I clicked on the scroll wheel the setting gets changed. Since you cannot see your setting I've found on more than one occasion I have forgotten which setting I was on so a quick change could be problematic.

At least they fixed the problem I had on 11.3.3 where the scroll wheel would not change the chill/avg/assv setting. I could only do that from the main display. 11.3.6 fixed that for me.