FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Knightshade]

I agree with your conclusion based on what I see on my HW4 car now, but I would change “isn’t happening” to “isn’t happening anytime soon” for both HW3 and HW4 cars. Most, if not all, of the people on this forum have more knowledge about Tesla’s FSD capabilities than I do… but right now, I do not see how we can get there (comfortable L3) from here (nervous L2) anytime soon.

I think L3 highways remains quite doable for even HW3 cars- off highway not so much (and HW4 is too unknown to judge if it's more enough to matter)- don't see L4 likely with either, ever.

 

[aronth5]

I think L3 highways remains quite doable for even HW3 cars- off highway not so much (and HW4 is too unknown to judge if it's more enough to matter)- don't see L4 likely with either, ever.

This.... anything more is unlikely

The possible exception is L3 for city/streets which will depend on how well Tesla can program the hand off to the driver so there is adequate time.

 

[kabin]

Teslafi's v11.4.4 numbers appear to have peaked since the big dump from a day or two ago. Maybe it's deemed unworthy of a full rollout?

 

[jebinc]

.... FSD was supposed to be a "mind-blowing" step function disruptor.

... for $15,000...

 

[kabin]

I think L3 highways remains quite doable for even HW3 cars- off highway not so much (and HW4 is too unknown to judge if it's more enough to matter)- don't see L4 likely with either, ever.

There's no FSD foundation for L3 on HW3. Heck L2 for HW3 is marginal at best with little significant signs of progress. The team's first goal is safety and yet they are unable to stick a fork in the Feb 2022 NHTSA stop sign recall. It's a punch in the gut.

 

[Knightshade]

There's no FSD foundation for L3 on HW3.

Uh--- what?

Remove the driver paying attention checks when on the highway, slightly increase the "take over now" code for highway use, and it's L3 on the highway today.

 

[FSDtester#1]

1st time poster... only a couple of miles of fsd. /s
4.3 was soo good I did not want to install any further updates.
I am a complete idiot for installing. For hw3, 4.4 is 3 steps back, 1 step forward.
Instead of the usual 2 forward 1 back.
@fasteddie7 , keep doing your videos, your production quality is excellent. I love them. Also if you could mount your body cam just a few inches higher that would be great. Also Eddie, you are in a MS With hw4. Is it a Plaid? Thanks in advance. In general since I actually have more beta miles than anyone on planet earth /s...well I might beside the whole mars. I have around 40K beta miles.
Watch out for sudden lane changes and drifting into adjacent lanes. I just avoided a side collision since the uss are turned off and ego didn't see the car directly next to me on the right.
On a positive, it does take off quickly when it decides to go.

Edit: I will try and post a video of going through a residential area.
About 20 turkeys run into front of the car. They car did slow down and successfully navigated around all of them. It's pretty funny.
I was laughing so hard I almost forgot to save the clip.

 

[fasteddie7]

1st time poster... only a couple of miles of fsd. /s
4.3 was soo good I did not want to install any further updates.
I am a complete idiot for installing. For hw3, 4.4 is 3 steps back, 1 step forward.
Instead of the usual 2 forward 1 back.
@fasteddie7 , keep doing your videos, your production quality is excellent. I love them. Also if you could mount your body cam just a few inches higher that would be great. Also Eddie, you are in a MS With hw4. Is it a Plaid? Thanks in advance. In general since I actually have more beta miles than anyone on planet earth /s...well I might beside the whole mars. I have around 40K beta miles.
Watch out for sudden lane changes and drifting into adjacent lanes. I just avoided a side collision since the uss are turned off and ego didn't see the car directly next to me on the right.
On a positive, it does take off quickly when it decides to go.

Thank you! It's an LR. I will it's an easy change since it's on a telescopic dji stabilizer attached to the center console. I opted to not get a mount since I can control left and right pan to look out the side windows.

 

[Tronguy]

Since variation analysis is what I teach (quantitative analysis and regression modeling), I will pipe up here. The variation reported here is a factor of all of the variables associated with both FSD software AND (most importantly), the driving environment. The problem is that there are so many variables that keeping track of them, and reporting them here on the forum when we complete driving reports, that it quickly becomes overwhelming. Some of these variables for example:

Visibility associated with time of day and direction relative to the sun and moon.
Phase of the moon for night driving.
Weather (Rain, clouds, barometric pressure, etc.)
Barometric pressure
Environment (rural, urban)
Actual traffic environment (other vehicles and obstacles)
Direction relative to the sun
Direction relative to the moon during night driving
Road markings or lack thereof
Road obstructions
Road sign obstructions
Wind
Behavioral predictability (both animate and inanimate)
Tire pressures and changes in tire pressures while driving
FSD variability
Map variability
Vehicle cleanliness
Color of the tesla (different colors absorb and reflect heat differently, causing thermal distortion of the air coming off the hood, heat waves if you will)
and
on
and
on

Not only do all of these variables exist, but (in quantitative analysis terms) they interact with each other, and the vehicle. Additionally, even with the simplistic off-the-cuff variables, each of the above variables have near infinite possibilities. For example, let’s pick a variable from the above list that looks simple:

Temperature.

With respect to temperature, we have the variables associated with:

The vehicle interior temperature
The air temperature
The temperature of every single object in the cars field of vision
Temperature deviations (both high and low frequency deviations)
Tire temperatures
Etc.

Each of these sub variables has their own sub variables, for example, with respect to tire temperatures, we have:

Left front tire temperature
Left rear tire temperature
Right rear tire temperature
Right front tire temperature

And each of these tire temperature have sub variables:

Left front tire temperature at time t = 0
Left rear tire temperature at time t = 0
Right rear tire temperature at time t = 0
Right front tire temperature at time t = 0
Left front tire temperature at time t = 1
Left rear tire temperature at time t = 1
Right rear tire temperature at time t = 1
Right front tire temperature at time t = 1
etc.

And every variable, sub variable, sub-sub variable, and sub-sub-sub variable (and so on, you get the idea) interacts with other variables, sub variables, sub-sub variables, sub-sub-sub variables (and so on, you get the idea).

The infinite combination of variables and variable interaction terms should quickly become obvious.

And then, to top everything else off, we have human variability, both with respect to the driver of the Tesla and other drivers and vehicles.

Given the above, not even close to complete list of variables that influence FSD behavior, the variability of driver reports of FSD performance should not be at all surprising.

Joe

And, speaking as a design EE who's deeply into clocks:

• Sampling rate of clocks in CPUs subject to temperature variations.
• Phase variation of so-called synchronized clocks in the system.
• Propagation delays of I/O subject to temperature variations.
• Changes in the resistivity of FETs and bipolars in the power train: Changes the gain of the power system in response to varying input of demand power. Which changes the poles and zeroes of the control loop.

and on and on and on. As you correctly point out, variational analysis is "fun", especially when it gets to things that are analog, or look analog.

 

[rlsd]

Bad behaviors with FSD 11.4.4 today:

1. Went thru intersection greenlight then slowed down like going to stop for no reason.
2. Erratic lane changing. It changed to the second lane even the left turn was about 1 mile ahead. It changed to the second lane, I changed back to the first lane. It did that 3 times and I disengaged. It did that even when the left turn to destination was only .1 mile ahead.

 

[Dennisis]

I have the impression FSD thinks the first lane is for passing only.

I just received 4.4 a few days ago so haven't had a lot of time to test but yes, I've experienced these same lane changes just prior to upcoming left turns. It seems they're trying to fix the staying-in-the-left-lane-all the-time behavior but overshot the mark.

My car is still trying to pull out in front of traffic on left turns too. It stops just before a stop sign, creeps up a bit but not really far enough to see crossing traffic, then says to hell with it and accelerates into traffic. I tried waiting as long as I could before intervening but it showed no signs of stopping - this happened several times. Again, they seem to have overshot the mark when trying to alleviate slow downs and hesitation in turns.

 

[rlsd]

I just received 4.4 a few days ago so haven't had a lot of time to test but yes, I've experienced these same lane changes just prior to upcoming left turns. It seems they're trying to fix the staying-in-the-left-lane-all the-time behavior but overshot the mark.

My car is still trying to pull out in front of traffic on left turns too. It stops just before a stop sign, creeps up a bit but not really far enough to see crossing traffic, then says to hell with it and accelerates into traffic. I tried waiting as long as I could before intervening but it showed no signs of stopping - this happened several times. Again, they seem to have overshot the mark when trying to alleviate slow downs and hesitation in turns.

I removed the line you quoted because my guess was probably wrong. On my daily way to work on the same road, FSD stays on the first lane and it only changes to the second lane when it's about .8 mile to enter the freeway on the right.

 

[kabin]

Uh--- what?

Remove the driver paying attention checks when on the highway, slightly increase the "take over now" code for highway use, and it's L3 on the highway today.

It would be nice but my money says it won't happen on HW3. There's just too many issues.

 

[jabloomf1230]

I just received 4.4 a few days ago so haven't had a lot of time to test but yes, I've experienced these same lane changes just prior to upcoming left turns. It seems they're trying to fix the staying-in-the-left-lane-all the-time behavior but overshot the mark.

My car is still trying to pull out in front of traffic on left turns too. It stops just before a stop sign, creeps up a bit but not really far enough to see crossing traffic, then says to hell with it and accelerates into traffic. I tried waiting as long as I could before intervening but it showed no signs of stopping - this happened several times. Again, they seem to have overshot the mark when trying to alleviate slow downs and hesitation in turns.

This has been happening for several versions now. Previously, the car would display, "Creeping forward for visibility", creep to the blue line, then wait to assess if any blue cars were in cross traffic. Now it creeps and then turns without worrying at all about cross traffic.. Sometimes it creeps partially out into traffic and then hesitates. Combining everything into one stack has created new holes in the leaky dam that appear with every firmware upgrade.

 

[Dennisis]

This has been happening for several versions now. Previously, the car would display, "Creeping forward for visibility", creep to the blue line, then wait to assess if any blue cars were in cross traffic. Now it creeps and then turns without worrying at all about cross traffic.. Sometimes it creeps partially out into traffic and then hesitates. Combining everything into one stack has created new holes in the leaky dam that appear with every firmware upgrade.

Yes I had it on 4.2, never received 4.3, and it's still there on 4.4. I don't recall it being an issue before 4.2. Definitely a bit disconcerting if you don't know of the issue but once it happens you remember!

As for the moving to the right lane before left turns issue I'm hoping it at least moves to the right lane after passing on the highway now, won't be able to test for a week or so. Camping out in the left lane unless someone lurks 20 feet from your bumper is crazy.

 

[jebinc]

Uh--- what?

Remove the driver paying attention checks when on the highway, slightly increase the "take over now" code for highway use, and it's L3 on the highway today.

It IS NOT L3 on the highway today. Tesla takes zero liability. See autonomy definitions for more detail.
@diplomat33 did a deep dive primer thread once on this. Perhaps he can add the link?

 

[jebinc]

It IS NOT L3 on the highway today. Tesla takes zero liability. See autonomy definitions for more detail.
@diplomat33 did a deep dive primer thread once on this. Perhaps he can add the link?

Found it!

Primer on SAE Levels of Autonomy

Since I know we have a lot of discussions about the SAE levels, I thought I would write up a primer from the SAE document: General The levels are not in order of quality, or merit or technology. L5 is not necessarily “better” than L4. L4 is not necessarily “better” than L3. A “high...

teslamotorsclub.com

 

[diplomat33]

It IS NOT L3 on the highway today. Tesla takes zero liability. See autonomy definitions for more detail.
@diplomat33 did a deep dive primer thread once on this. Perhaps he can add the link?

Thanks. FSD Beta is definitely not L3 on the highway.

Here is the link the Primer: Primer on SAE Levels of Autonomy

(I feel like the Primer could use an update though)

Also here is a nice chart by Dr. Koopman that explains what the Driver and System are responsible for in each SAE level.

cc: @Knightshade

 

[jebinc]

Thanks. FSD Beta is definitely not L3 on the highway.

Here is the link the Primer: Primer on SAE Levels of Autonomy

(I feel like the Primer could use an update though)

Also here is a nice chart by Dr. Koopman that explains what the Driver and System are responsible for in each SAE level.

cc: @Knightshade

That was great work; do update it at your convenience!

 

[aronth5]

It IS NOT L3 on the highway today. Tesla takes zero liability. See autonomy definitions for more detail.
@diplomat33 did a deep dive primer thread once on this. Perhaps he can add the link?

Everyone knows FSD is not L3 on the highway today and I haven't seen a single post to that affect.

What we're beginning to see from some posters is that L3 on controlled access highways is doable contingent on Tesla having an updated hand off to the driver. Second, fix the remaining issues which of course there certainly are some. In my experience when I get on the highway by selecting the "minimal lane change" option I can easily let FSD drive without any interventions. I have done this many times in the past few weeks and gone well over a 1k miles just to see how FSD does and it's fine. Third Tesla would have to take responsibility. Fourth have Tesla enable display for videos.

Do I think Tesla will do this probably not but since I don't care about robotaxi this would be a step by Tesla in the right direction.