FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[rlsd]

Looks like 44.30.8 is slowing down. Wonder if we are about to get "even more minor bug fixes" 44.30.9? Well we ever get the final 44.30.420 Holiday update? At this rate EVERYONE may be stuck on last year's software the way we Early Access users have been for years.

We will get 2023.44.30.9/V12 then 2023.44.30.10/V12.1. You are not excited?

 

[jebinc]

Early this morning before sunrise, I experienced dry wipes on 11.4.9. I know most humans can handle wiper controls.

Yesterday (and the day before), during drizzle and in Auto mode, the damn wipers refused to activate - no matter. Did 44.30.7 undo the fix found in 44.30.5.1 and .6? What crap, I tell you...

 

[AlanSubie4Life]

Start to decelerate WAY earlier for stopping/slowing situations instead of waiting till the last second and using hard friction brakes. (See desired fix #1)

That is how 11.4.9 is working for me. It hardly uses the friction brakes at all.

Replying here instead of v12.

I agree that 11.4.9 still charges into situations where it is obviously going to need to slow down, much more aggressively than a human.

In many cases though, it slows down nearly entirely with regen braking. If it is on a downward slope that is not the case though.

Even more important than what type of braking is used, this behavior is disconcerting to passengers, and they seem to have destabilized the deceleration task again - pulsing jerky slowdowns are now common, when they had been mostly suppressed in prior version.

Could be related to instability due to the new faster video module - making it faster definitely could make the system less stable. Hopefully they’ll fix that in a point update.

Anyway, a big step back so far on comfort and driving policy (see prior lists).

Agreed with other complaints about lane positioning on certain streets. It’s odd.

It also takes corners differently now, and is cutting them. It is getting closer to curbs with rear wheel on right turns and cutting corners on left turns in a way that interferes with other vehicles. This may be dependent on the road type as well; not sure.

 

[MP3Mike]

In many cases though, it slows down nearly entirely with regen braking. If it is on a downward slope that is not the case though.

I don't see that, it uses regen all the way to a stop even going downhill. (I even saw it apply power to stop instead of using the friction brakes.)

Even more important than what type of braking is used, this behavior is disconcerting to passengers, and they seem to have destabilized the deceleration task again - pulsing jerky slowdowns are now common, when they had been mostly suppressed in prior version.

I'm just not seeing that behavior. (I can see the power bar jitter a little as it rides the regen vs. friction brake line, but it isn't something that can be felt.)

 

[AlanSubie4Life]

I don't see that, it uses regen all the way to a stop even going downhill. (I even saw it apply power to stop instead of using the friction brakes.)


I'm just not seeing that behavior. (I can see the power bar jitter a little as it rides the regen vs. friction brake line, but it isn't something that can be felt.)

Time for some video it seems! Got my crappy GoPro plugged in. What you want to see on the regen bar is a smooth increase to a steady green bar which smoothly reduces in length as you come to a stop. In general the amount of pulsing (amount of regen/friction braking) should be small.

If a human puts effort into smooth stopping, that's what it will look like - and Tesla's awesome drivetrain makes it easy of course.

 

[Dewg]

Time for some video it seems! Got my crappy GoPro plugged in. What you want to see on the regen bar is a smooth increase to a steady green bar which smoothly reduces in length as you come to a stop. In general the amount of pulsing (amount of regen/friction braking) should be small.

If a human puts effort into smooth stopping, that's what it will look like - and Tesla's awesome drivetrain makes it easy of course.

For reference, make sure you take videos of stopping manually, so everyone can see the smooth regen bar you're referring to.

 

[AlanSubie4Life]

For reference, make sure you take videos of stopping manually, so everyone can see the smooth regen bar you're referring to.

Of course; would have to have comparison videos. (As you know, I've posted videos of similar behavior, showing this, before!) For stop signs I can definitely easily do that, but for the typical traffic lights it's a bit more work and getting a comparable stop is dependent on factors (like the light timing and other traffic) outside of my control. Under manual control it can usually be smooth regardless, but the exact situation may not be directly comparable. Plus it takes a lot longer to go back and repeat them.

amount of pulsing (amount of regen/friction braking)

A typo here - I mean the amount of CHANGE in the regen/friction braking.

 

[Twiglett]

I got to do my semi regular weekend camping trip this weekend with its 120 mile round trip.
Still on 2023.44.30.5.1
One very interesting observation - one of the annoying FSDb behaviors has been to treat flashing yellow warning lights as traffic lights, so instead of slowing down slightly it tries to stop.
I noticed with this release that the nav now shows traffic lights in the places where the warning lights are, so now I'm wondering if this is a mapping data issue rather than camera.
My drive back was really pretty darned good, with no disengagements needed. There were a few times where it needed a bit of a prod to get going and twice where it figured the left lane was "following the route".
Everything was going swimmingly until on a 65mph divided highway it returned to form and lurched into a right hand exit lane. No signal and not on the route. I let it go because we were due to turn right in .5 mile and I thought we were there already. The only problem was that it wasn't slowing down to make a turn so I had to brake hard to stop it driving into a field.
Manually driving back onto the route I re-engaged a it was able to get home without further incident.
So close to being pretty good.

 

[enemji]

The one change I've noticed so far is more confidence in scenarios where it needs to make a lane change immediately after a turn. It's quickly turning into the appropriate lane at an angle, and then quickly initiating the required lane changes.

It should make a wider turn into the next lane rather than a tight turn from a right lane to a right lane especially on 3+ lane roads

 

[enemji]

Bad weather feedback:

The car continues to straddle the right most lane, rather than staying the middle lane. This is especially bad during bad weather as the water piles up on the right most lanes due to the crowning of the roads, and is putting itself in a bad position of hydroplaning and loss of steering when hitting a puddle of water.

 

[willow_hiller]

It should make a wider turn into the next lane rather than a tight turn from a right lane to a right lane especially on 3+ lane roads

Although human drivers do that all the time, I think what you describe is illegal in many states. Here's the letter of the law in Maryland:

§21–601.

(a) If the driver of a vehicle intends to turn right at any intersection, he shall approach the intersection and make the right turn as close as practicable to the right-hand curb or edge of the roadway.

(b) If the driver of a vehicle intends to turn left at any intersection or crossover, he shall approach the intersection or crossover in the extreme left-hand lane lawfully available to traffic moving in the same direction.

It actually becomes a bit dangerous when there are multiple turn lanes and FSD Beta is in the exterior lane. The humans on the inside-lane just assume you're going to turn wide, and turn wide themselves.

 

[mtndrew1]

I’ve been tracking my FSDb performance since late 2021 and I’ve updated my spreadsheet with 11.4.9. See below.

FSD doc

 

[MP3Mike]

Looks like 44.30.8 is slowing down. Wonder if we are about to get "even more minor bug fixes" 44.30.9?

Haven't you learned yet? Updates go out in waves, and one wave is mostly finished before the next starts. (It seems like a new wave of 2023.44.30.8 was just recently started.)

It seems the majority, 80+%, of 30.8 vehicles are HW3 vs. ~12% HW4. (HW4 is only ~5% of the TeslaFi fleet.)

 

[MP3Mike]

Some interesting observations about FSDb 11.4.9 and rain:

• In general, the auto wipers work great, especially during the day.
• At dusk they seem to want to go full speed with just light rain.
• On the highway, with a 60MPH speed limit, it would only let me set it to 65MPH because of the heavy rain.
• About a mile down further when the rain was lighter and a 50MPH "construction" speed limit started it automatically dropped the setting to 62MPH, but let me set it as high as 70MPH.

That was the first time I had seen it limit my speed setting based on weather...

 

[Supcom]

Some interesting observations about FSDb 11.4.9 and rain:

• In general, the auto wipers work great, especially during the day.
• At dusk they seem to want to go full speed with just light rain.
• On the highway, with a 60MPH speed limit, it would only let me set it to 65MPH because of the heavy rain.
• About a mile down further when the rain was lighter and a 50MPH "construction" speed limit started it automatically dropped the setting to 62MPH, but let me set it as high as 70MPH.

That was the first time I had seen it limit my speed setting based on weather...

I had some heavy rain today and the auto wipers on 2023.44.30.5.1 worked well. FSDb also worked well. It did slow from 70 mph speed limit to 65 in most of the rain and further down to 60 mph in the heaviest. Neither seemed inappropriate given the visibility.

I did notice that when FSDb was reporting poor weather it seemed to act like it was in chill mode with minimal lane changes. there was a couple cases where the lead car on the highway was driving considerably slower than my car's set speed, yet FSDb showed no interest in passing the car until I initiated a manual lane change. There were no errors for visibility issues with the side cameras, and the car made required lane changes for exits just fine.

 

[sleepydoc]

It should make a wider turn into the next lane rather than a tight turn from a right lane to a right lane especially on 3+ lane roads

MN law states you need to turn in to the closest lane. If there are two lanes then the first lane turns into the closest lane and the next lane turns into the next closets lane, etc. Making a wider turn is technically illegal.

 

[hybridbear]

Some interesting observations about FSDb 11.4.9 and rain:

• In general, the auto wipers work great, especially during the day.
• At dusk they seem to want to go full speed with just light rain.
• On the highway, with a 60MPH speed limit, it would only let me set it to 65MPH because of the heavy rain.
• About a mile down further when the rain was lighter and a 50MPH "construction" speed limit started it automatically dropped the setting to 62MPH, but let me set it as high as 70MPH.

That was the first time I had seen it limit my speed setting based on weather...

Speed has been limited due to rain since before the recall version. We experienced that on 2023.27.7 (11.4.7.3).

 

[Dewg]

It actually becomes a bit dangerous when there are multiple turn lanes and FSD Beta is in the exterior lane. The humans on the inside-lane just assume you're going to turn wide, and turn wide themselves.

I've been honked at twice over this. Oddly, both times were BMW drivers. My Tesla was in the right-hand left turn lane, and turned into the #2 lane, as it should. The other car in the left-hand left turn lane should have turned into the #1 lane, but instead wanted to turn into the #2, and nearly hit me, and honked at me. Idiots...

 

[Supcom]

I've been honked at twice over this. Oddly, both times were BMW drivers. My Tesla was in the right-hand left turn lane, and turned into the #2 lane, as it should. The other car in the left-hand left turn lane should have turned into the #1 lane, but instead wanted to turn into the #2, and nearly hit me, and honked at me. Idiots...

Today, my car was in the same situation, but turned into the rightmost of the three destination lanes, then made an immediate lane change to the center lane. In this intersection, the right-hand left turn lane is dash-lined to the rightmost destination lane, though the lines are long worn away. So, the car did the nominal thing.

Usually, the car makes this turn from the left-hand left turn lane into the leftmost destination lane then makes an immediate lane change to the center lane, if safe to do so. Quite freaky (though legal in Texas) as the lane change is made in the middle of the next intersection.

The car used to turn left into a left turn lane then move right into a thru lane. Then, for a while, it would turn directly into the middle thru lane, which was also legal, but scary since cars in the other turn lane often turned into the middle lane as well.

I can't wait to see how V12 handles this intersection!

 

[Dewg]

Today, my car was in the same situation, but turned into the rightmost of the three destination lanes, then made an immediate lane change to the center lane. In this intersection, the right-hand left turn lane is dash-lined to the rightmost destination lane, though the lines are long worn away. So, the car did the nominal thing.

Usually, the car makes this turn from the left-hand left turn lane into the leftmost destination lane then makes an immediate lane change to the center lane, if safe to do so. Quite freaky (though legal in Texas) as the lane change is made in the middle of the next intersection.

The car used to turn left into a left turn lane then move right into a thru lane. Then, for a while, it would turn directly into the middle thru lane, which was also legal, but scary since cars in the other turn lane often turned into the middle lane as well.

I can't wait to see how V12 handles this intersection!

A new behavior I've seen with the holiday build is single-lane protected left turns. There is one intersection by my house that navigation always takes me through. It's a single left turn lane, protected, turning onto a 3-lane (technically 4 lanes, but the #4 lane almost immediately ends and merges into #3). My car used to always turn into the #1 lane, and then immediately start changing lanes to the right to make the right turn at the next intersection. Now, with the holiday build, it will turn wide, all the way to the #4 lane, which then has to merge into #3. Most people turn wide and into #2 or #3 lanes on this turn, so it's more human-like. I worry about people turning right in the opposite direction, but technically, since it's a protected left, and the opposite direction has a red light, they shouldn't be turning until the left-turners are done.