FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Rikster]

I keep reading about all the problems everyone is having with the auto wipers setting, and it made me wonder why I never have "dry-wipes" on my 2022 Model S Refresh LR. It started raining today while I was driving (I very seldom drive my Tesla in the rain), and the wipers didn't come on automatically. When I turned them on manually, I noted that they had been set to the "off" position all this time.

So somehow my version of FSDb isn't requiring that I leave the wipers set to Auto mode. I'm very happy for that, of course, as I never get dry-wipes and I like to control my wipers myself anyway. But I do wonder why I can get away with not having my wipers set to auto mode and still use FSDb daily?

Not complaining, though ;-)

 

[sleepydoc]

I tested turnout lane change errors today on a four-lane divided highway (TX-24 north of Commerce, TX) (not limited access) that has many left turnout lanes. I used FSD 11.4.7.2. Not surprisingly for a minor point release, the car showed no improvement over 11.4.7 and attempted to enter about half of the turnout lanes when it was to continue straight.

I noticed two interesting things that might explain what is going on. First is that the car did not activate the turn signal as it moved into the turn lane. The second was at one point I used the turn signal to initiate a lane change to the right just before reaching a left turnout lane. Even though the right turn signal was active, the car attempted to enter the left turnout lane! Third, I found that even with Minimal Lane Changes active, if the car was in the left lane, it still attempted to enter left turnouts.

From these, I infer that the car is actually not attempting to change lanes into the turnout lane, but mistakenly thinks that the turnout lane is the continuation of the left through lane. This particular stretch of highway has turnout lanes that have a more gradual opening than most, so there may be something about that that doesn't trigger an NN designed to detect turnouts.

Yes - this also goes along with a lot of the phantom turn signals. It seems that FSD has a difficult time knowing what lane is ‘straight.’ It’s quite puzzling since it’s so obvious to us but i have to assume that the coding solution is more difficult since they’ve struggled with it so long.

 

[AlanSubie4Life]

It’s quite puzzling since it’s so obvious to us but i have to assume that the coding solution is more difficult since they’ve struggled with it so long.

They have had problems with longitudinal control for a long time too. And you’d think that would be the first thing they would master!

I think it is just overall a really hard problem and requires an extremely complex solution - after all, they are trying to emulate something mind-bogglingly complex!

There aren’t any shortcuts. You can’t just throw a bunch of training data at it and train up low-complexity neural net hardware and expect it to work, either. Obviously.

 

[jebinc]

11.4.7.2 is worse than 11.4.7, at least after two days of using this junk. More stopping mid turn in spots that I did not have issues before. Not giving pedestrians enough space, yet swerving for pedestrians walking on sidewalks. Highway performance is worse yet. Phantom braking, taking west bound exits when it’s supposed to take east bound exits.

Call, 1-800-GOT-JUNK

 

[arnolddeleon]

11.4.7 got better as time went on (assuming map updates) but it seems whatever cached state was present gets reset. Because 11.4.7.2 has regressed back to the similar behavior right when I got 11.4.7.

On right turns where a quick switch to the left lane is needed it used to execute it like that, make the right turn into the rightmost lane and then immediately move/signal. The one data point I had today for a similar situation had it execute a "wide right turn" where it ended up pretty much directly in the left lane.

I also didn't have voice control immediately after the upgrade yesterday. Today it worked just fine, didn't do anything to fix.

I small feature change on the instrument panel, make it easier to figure out if it's a left or right turn signal that is on (the arrow direction is not quickly discernable for these aging eyes). This is important for supervising the car. I've found myself looking at one side while the car moves to the opposite. A few ideas:

• Put a landmark between where the arrows appear, the FSD steering wheel indicator would be good since this when it is useful.
• Another option is to back to an earlier design where the speed was displayed centered in the top of the instrument panel area.
• Display a grayed arrow for both directions always.
• Put the "PRND" indicator in center.
• Pull them further apart (this might have happened in recently)

 

[Ramphex]

Had an ironic error today. Came up with exclamation point in a yellow triangle and said “auto lane change is unavailable”, I thought to myself… “perfect, I don’t want to switch lanes right now because there’s no reason to”

 

[Pdubs]

Had an ironic error today. Came up with exclamation point in a yellow triangle and said “auto lane change is unavailable”, I thought to myself… “perfect, I don’t want to switch lanes right now because there’s no reason to”

That should be a default. Not the auto wipers turning off

 

[WilliamG]

Finally got the new update, yay!

 

[Pdubs]

Yeah. I sht you not, after I installed it and went to the course, this is the song Tidal kicked.

Then half way to work

And when I was pulling in.

Tidal has a better take on this than any of you sorry fanbois

 

[Scott Chapman]

Looks like another big block of users are getting the update

 

[tivoboy]

Looks like another big block of users are getting the update

Yeah, just got 23.27.6 last night..hopefully, it doesn’t REALLY break voice commands since I use them a LOT

 

[LowlyOilBurner]

Had 11.4.7.2 make a left turn when the route called for this turn to be a right. Absolutely ridiculous.

 

[heapmaster]

I keep reading about all the problems everyone is having with the auto wipers setting, and it made me wonder why I never have "dry-wipes" on my 2022 Model S Refresh LR. It started raining today while I was driving (I very seldom drive my Tesla in the rain), and the wipers didn't come on automatically. When I turned them on manually, I noted that they had been set to the "off" position all this time.

So somehow my version of FSDb isn't requiring that I leave the wipers set to Auto mode. I'm very happy for that, of course, as I never get dry-wipes and I like to control my wipers myself anyway. But I do wonder why I can get away with not having my wipers set to auto mode and still use FSDb daily?

Not complaining, though ;-)

Im on the public build (11.3.6) and luckily dont get any dry wipes, but have a few times randomly got washer fluid sprays (I guess the window is a bit dirty, but not enough to where I would clean it) and then a few wipes after, so at least its wetting it first.

 

[FSDtester#1]

Im on the public build (11.3.6) and luckily dont get any dry wipes, but have a few times randomly got washer fluid sprays (I guess the window is a bit dirty, but not enough to where I would clean it) and then a few wipes after, so at least its wetting it first.

I have had 2 loaners in the last 60 days, both were on 11.3.6
Obviously what they think is there best available software

 

[JulienW]

Looks like another big block of users are getting the update

Well I'm still rocking the same now 33 week old software and per the usual bringing up the extreme ass end. I swear I wish I could get off this insufferable and misnomer of a name Early Access.

Maybe I'll get it before November 31st.

And of course I may get the Navigation Data update as my Holiday update. Thanks Elon.

 

[rlsd]

Had 11.4.7.2 make a left turn when the route called for this turn to be a right. Absolutely ridiculous.

I had that situation with the previous releases. I thought I had hearing problems.

 

[heapmaster]

I have had 2 loaners in the last 60 days, both were on 11.3.6
Obviously what they think is there best available software

Ya, I would say it doesnt have as many issues as everyone here reports, but it does have its issues...

 

[rlsd]

Had 11.4.7.2 make a left turn when the route called for this turn to be a right. Absolutely ridiculous.

You should ignore what you hear. FSD is based on vision, not audio.
WYSIWYG

 

[jebinc]

Well, I received the latest version of Junk about four days ago, but couldn't be bothered to drive it until this morning. Highlights include:

The Good (relatively speaking)
Programmable left scroll wheel
Movable blind spot camera
J didn't spazz out when approaching the gate on the way out (see bad)
No dry wipes or dry holes
10 mile trip to the store, while letting the junk drive, was largely acceptable (see bad)

The Bad (aka "The Junk")
The Junk spazzed out with Red hands of death when approaching the gate on the return (see Good)
10 mile trip home from the store had three disengagements
- Da junk wanted to drive into my lamp pole
- The Junk did not slow down for speed bumps (but I didn't expect it to)
- Da junk just would not want to make a sharp right turn, even when there were no cars coming. The Junk just creeped endlessly. Probably because the cameras and camera placements on the MS just suck, and suck hard at that.


Well, maybe I will try in again later this week, we'll see....

 

[mtndrew1]

Question here from someone new to FSDb: I thought that the problem of the lane change because of an interruption of a left-side barrier was solved.

Yesterday I was driving along the CA-37 - a long two-lane road, one in each direction mostly separated by a cement barrier - and at one of the barriers' interruption the FSDb thought that was a good idea to move to the imaginary lane on the left. Luckily the turn signal caught my attention because FSD was already steering in fractions of a second...

Note: the picture of the exact point attached is taken from Google Maps, I'm 90% positive that now there is a yellow solid line on the right over that new patch of tarmac.

FSDb is more effective than coffee for both waking up and vacating your bowels.