FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[AlanSubie4Life]

The two places it identified braking events were unremarkable.

This is quite a bad braking score, nevertheless. What the g threshold for the measurement?

I would guess it is similar to what I am seeing. Do you have a lot of stop signs?

I guess I’ll have to drag out the GoPro. Such a hassle. But probably should document from time to time.

It would be helpful for people to see the tiny amount of braking being applied coming up to stop followed by reduction in acceleration. New mounting position might be needed though; hard to get the resolution required without an HDMI hack.

 

[MP3Mike]

This is quite a bad braking score, nevertheless. What the g threshold for the measurement?

What makes you think it is a bad score? (Notice that they even say it is "average.") That is just two individual braking events. It is just a counter. It shows like that if it is two events over 1 mile or two events over 200 miles. They won't tell me what the limit is. But it is very low. (I trigger it just as much if not more than FSDb does now.)

I would guess it is similar to what I am seeing. Do you have a lot of stop signs?

I doubt it is the same, as the grey portion of the regen bar never shows up if the battery is warm. So almost no physical brake usage. I wouldn't say a lot of stop signs, the majority of my normal trip is stop lights. (But the stops for the stop signs and red lights are almost identical.)

 

[AlanSubie4Life]

I doubt it is the same, as the grey portion of the regen bar never shows up if the battery is warm. So almost no physical brake usage. I wouldn't say a lot of stop signs, the majority of my normal trip is stop lights. (But the stops for the stop signs and red lights are almost identical.)

Do you have traffic? Any traffic and the behavior would of course disappear.

My SOC is 50% with temps in the 50s so no issues there with that being a contributor (no dots).

What makes you think it is a bad score? (Notice that they even say it is "average.") That is just two individual braking events.

I don’t know. All I have is Safety Score as a reference. There it was extremely difficult to get a hard braking event (if you were trying and not on a steep hill). Maybe this is more sensitive?

FSD beta would be triggering the Safety Score threshold pretty frequently meaning its deceleration (and likely jerk as well) exceeds human driving for a required profile.

With the exception of the massive unnecessary emergency stop I mentioned in one of these threads, I don’t mean to imply that it is slamming on the brakes. It is just using them slightly, and then easing off, which makes no sense at all and leads to higher peak deceleration and higher jerk than an optimal regen-only profile (Tesla’s drivetrain control in this regard without FSD mucking about is silky smooth of course).

 

[aronth5]

In my experience, it slows 100% of the time for speed bumps with large chevrons painted on them. If the bump is unpainted, it's a toss up.

Also, unless they're able to update the map metadata very quickly, it must be at least partially vision-based. My neighborhood had new speed bumps paved a couple of weeks ago, and FSD Beta started slowing for them as soon as they were installed.

The bumps in my area are large and well marked and FSD almost never slows down. I'm of the opinion the map quality where I drive (greater Boston) is just bad.

I drove a road again today where FSD comes to a complete stop in the road. Navigation shows FSD should do U-turn in the middle of the road so FSD stops the car and then wants to do a U-turn. Total failure of course. I asked around and this road had bridge construction work several years ago where the road was closed. If you approach this from the other end of the road same behavior. Just stops when the car approaches where the bridge was closed.

Once a month I resend the Tesla all the information they should need and the problem remains. Going on 2 years.

 

[MP3Mike]

My SOC is 50% with temps in the 50s so no issues there with that being a contributor (no dots).

I run ~80% and temps are currently in the 40s. But even in the 50s regen is limited and friction brakes get blended in if I haven't preconditioned the car long enough.

I don’t know. All I have is Safety Score as a reference. There it was extremely difficult to get a hard braking event (if you were trying and not on a steep hill). Maybe this is more sensitive?

The State Farm Drive Safe & Save is very sensitive. Just regen braking can ding you sometimes, so regen is right at the g-force limit. (Like I said, I don't think it should have gotten dinged at either spot.)

 

[AlanSubie4Life]

But even in the 50s regen is limited and friction brakes get blended in if I haven't preconditioned the car long enough.

I don’t have to worry about it. My car is kept at low SOC in a warm garage (unfortunately my furnace likes to heat the garage). It’s not a problem.

It’s not the issue here. Regen dots are possible of course if it is cold enough, but not what is happening and we’re talking about stopping from 10mph to 0 over fairly long distances. There’s just very little power required.

 

[willow_hiller]

I don’t have to worry about it. My car is kept at low SOC in a warm garage (unfortunately my furnace likes to heat the garage). It’s not a problem.

It’s not the issue here. Regen dots are possible of course if it is cold enough, but not what is happening and we’re talking about stopping from 10mph to 0 over fairly long distances. There’s just very little power required.

Odd question, but are the stop signs near you lit in some way? Or deviate from a normal sign in any way?

I have experienced the double-stop behavior you're describing, but only on stop signs that have red blinking LEDs around the perimeter of the sign.

 

[AlanSubie4Life]

Odd question, but are the stop signs near you lit in some way? Or deviate from a normal sign in any way?

I have experienced the double-stop behavior you're describing, but only on stop signs that have red blinking LEDs around the perimeter of the sign.

No they are not. They are just normal stop signs. You can cruise around this slightly dilapidated neighborhood. Google Maps

I did some more looking at this as I went through a few on my way home.

Obviously to some extent it depends on the specifics of the situation, but generally speaking in a 25mph zone, set to 12% over or so (and I did not see this exact behavior at every stop; just at many of them; this is for this specific one: Google Maps ):

1) Starting at 28mph, slows down reasonably normally to around 8-10mph.

2) From 8mph down to about 1-2mph, it slows down a bit faster (faster than the deceleration in step 1), and if you watch, it will be using a small amount of friction braking (the bar is extremely small at this point of course since there is virtually zero stopping power required for even extremely harsh deceleration at 8mph)

3) It stops doing this friction braking at about 1-2mph, and then takes a relatively long time to coast to 0mph using very small amounts of regen.

4) Sometimes after it comes to a halt, there's a very brief secondary roll and halt. This may be dependent on the stop since it did not always happen.

Super backwards. It should just regen all the way down to 3mph, nice and silky smoothly and consistently without any regen pulsing, then start using the brakes to firmly come to a stop in the right position and get the speedometer to show zero (this should be extremely quick since brakes will rapidly halt the car unlike the default regen tuning).

----

Also 11.4.9 drives in totally the wrong place in the road. Always too far to the right, kicking up the gravel in places, or driving excessively close to parked cars for no reason at all. Just drive next to the dotted white line (or yellow line, just at the right distance as appropriate for oncoming traffic)!

In my traffic school I attended many years ago, they told me to hug the white line on the right, and this is often the right thing to do for high-speed roads to allow maximum reaction time. But that's not the situation here. Driving in the gravel is a terrible idea.

At least this may be one place that a neural net training approach might help with - should be able to get road positioning reliably correct, I hope. No more of this nonsense where it is always changing as they tweak and tune. But honestly I don't understand how they don't notice this right away before rolling out even with the current approach. It should show up in simulation. There's no safety reason for it and it is likely less safe.

 

[bhakan]

The bumps in my area are large and well marked and FSD almost never slows down. I'm of the opinion the map quality where I drive (greater Boston) is just bad.

I drove a road again today where FSD comes to a complete stop in the road. Navigation shows FSD should do U-turn in the middle of the road so FSD stops the car and then wants to do a U-turn. Total failure of course. I asked around and this road had bridge construction work several years ago where the road was closed. If you approach this from the other end of the road same behavior. Just stops when the car approaches where the bridge was closed.

Once a month I resend the Tesla all the information they should need and the problem remains. Going on 2 years.

View attachment 1006039

I agree with you on the FSD going at the speed and never slowing down even once! I am in a very congested area with residential with kids and bike travelers and commercial including restaurants. There is always chaos in the traffic. There are a lot of speed bumps and the car will go around the parked cars at full speed. I dial down my speed to 15mph on 25mph ahead of time and that's the only way I can drive without hitting the bikers who are also on the road. The car never slows down on pot holes. Never. For the ones I know I change lane on my FSD ahead of time to save the tires. One of the is already getting worse and need patch or whatever. Oh the pleasures of driving my car!

 

[sleepydoc]

This is quite a bad braking score, nevertheless. What the g threshold for the measurement?

I would guess it is similar to what I am seeing. Do you have a lot of stop signs?

I guess I’ll have to drag out the GoPro. Such a hassle. But probably should document from time to time.

It would be helpful for people to see the tiny amount of braking being applied coming up to stop followed by reduction in acceleration. New mounting position might be needed though; hard to get the resolution required without an HDMI hack.

If that’s state farm then it doesn’t surprise Me. Their braking threshold is quite sensitive - appropriately stopping for a yellow light that changes right at the threshold of when you should stop vs continue through will always cause it to mark you down. FSD has gotten better but still tends to brake a bit hard, too. It isn’t horrible but definitely enough to trip the sensitive State Farm sensor.

 

[cbdream99]

I have seen it couple of times now, while stuck on back to back traffic on the interstate, it's driving at driving below 20 mph stop and go, at one or two instances, it says stopping for stop sign but there is absolutely no stop sign anywhere near in sight and it doesn't make sense to have a stop sign in the middle of the interstate anyway. I pushed the pedal to keep it from actually stopping.

 

[PACEMD]

I think removing beta tag at this stage will be a major shift and people will assume they don’t have to pay attention anymore. So, I don’t think it will ever be removed.

Wait, we're supposed to pay attention when driving? Now you tell me...........And how much is that gonna cost me?

 

[PACEMD]

I would say, Tesla has the best software, and that's a crying shame.

It's my Tesla and I'll cry if I want to, cry if I want to, cry if I want to...........you would cry too if it happened to you..........

 

[PACEMD]

Since the internal camera cannot detect whether I'm paying attention when I wear my sunglasses I need to keep my hands on the wheel like always. No change. You often hear people say Tesla should stop the nags since the camera is monitoring you but since Tesla knows certain sunglasses beat the driver monitoring system how can they safely stop nags?

It's 106 miles to Chicago, we've got a 100% battery charge, half a pack of cigarettes, it's dark and we're wearing sunglasses....

 

[PACEMD]

My first red light runner in months.

It's a red light runner, red light runner.......... loved that song...........Jefferson Airplane wasn't it?

 

[PACEMD]

Nah, we are talking about "Uncle Elon," here.... it needs to be on April 1st!

I've heard rumors of a recall, you know, to change the snake oil.........

 

[PACEMD]

Got 44.30.8 yesterday for the FSDb M3P. Haven't tried it yet but I can already say my minor bugs look soooooooo much better fixed..........

 

[JulienW]

Got 44.30.8 yesterday for the FSDb M3P. Haven't tried it yet but I can already say my minor bugs look soooooooo much better fixed..........

.....but doesn't 44.30.8 have FSD 12.69? Someone in the V12 Coming thread last week was Elon "promising" we would be getting V12 in "2 weeks".

 

[D Good]

The bumps in my area are large and well marked and FSD almost never slows down. I'm of the opinion the map quality where I drive (greater Boston) is just bad.

I drove a road again today where FSD comes to a complete stop in the road. Navigation shows FSD should do U-turn in the middle of the road so FSD stops the car and then wants to do a U-turn. Total failure of course. I asked around and this road had bridge construction work several years ago where the road was closed. If you approach this from the other end of the road same behavior. Just stops when the car approaches where the bridge was closed.

Once a month I resend the Tesla all the information they should need and the problem remains. Going on 2 years.

View attachment 1006039

The map data they use is utter trash. They need to get a better partner for map data if they are going to use it at all for FSD. Surprisingly, Apple Maps consistently gives better routes than whatever Tesla uses. Tesla follows the routes that google maps does, so either they are pulling from google maps or pulling from the same (awful) provider that google pulls from.

 

[jebinc]

I've heard rumors of a recall, you know, to change the snake oil.........

Nah, Elon's brand of snake oil has no shelf life, or use, restrictions.....