FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[FOOLSLFDRVNG]

It was on interstate 405 in Bellevue. The temporary repair to get me back home wasn't that expensive, as Les Schwab found me a used wheel. I still need to buy a permanent wheel to replace the temporary one though.

Unless there’s a tire leak, you can do a ghetto repair with a 2x4 and a sledge hammer.

 

[JHCCAZ]

Unless there’s a tire leak, you can do a ghetto repair with a 2x4 and a sledge hammer.

I'm more familiar with the ghetto replacement plan

 

[sleepydoc]

It was on interstate 405 in Bellevue. The temporary repair to get me back home wasn't that expensive, as Les Schwab found me a used wheel. I still need to buy a permanent wheel to replace the temporary one though.

Frustrating, but FSD doesn’t avoid potholes or small objects - one of the reasons one needs to stay vigilant.

 

[Ed Hart]

Here is a specific situation in which FSD in both of our cars performs identically every time we come to this intersection. I will try to find someone at Tesla to share it with....and I am posting it here in hopes that someone at Tesla is patrolling this forum. I will also try to contact the local traffic department to see if the sign can be moved further from the intersection, but this is a great example of how FSD needs to learn what humans would do in this situation. Even without seeing the final 40 mph sign, I can report that all drivers slow dramatically, since they can see the intersection very close ahead.

 

[JHCCAZ]

Here is a specific situation in which FSD in both of our cars performs identically every time we come to this intersection. I will try to find someone at Tesla to share it with....and I am posting it here in hopes that someone at Tesla is patrolling this forum. I will also try to contact the local traffic department to see if the sign can be moved further from the intersection, but this is a great example of how FSD needs to learn what humans would do in this situation. Even without seeing the final 40 mph sign, I can report that all drivers slow dramatically, since they can see the intersection very close ahead. View attachment 1020349

Nicely presented, thanks! The perception stack labeling is ignoring the unusual 40 mph sign that's enclosed within the warning diamond surround.

Please post back on this after you receive v12 and test it. According to numerous reports, v12 with Auto speed offset is very good at selecting appropriate speeds regardless of the signage.

 

[Yelobird]

Here is a specific situation in which FSD in both of our cars performs identically every time we come to this intersection. I will try to find someone at Tesla to share it with....and I am posting it here in hopes that someone at Tesla is patrolling this forum. I will also try to contact the local traffic department to see if the sign can be moved further from the intersection, but this is a great example of how FSD needs to learn what humans would do in this situation. Even without seeing the final 40 mph sign, I can report that all drivers slow dramatically, since they can see the intersection very close ahead. View attachment 1020349

This is interesting and well presented. Well done. I do however question how I would handle this situation without FSD. The sign is clearly stating that the speed “will” be 40 mph but it is also stating “ahead”. I personally would do the same as FSD and maintain speed until the ahead posted speed is reduced to 40 and start decelerating. Rarely see signs like this personally but I believe that’s how I would address it.

 

[AlanSubie4Life]

It's really unfortunate that FSD doesn't avoid potholes. The car hit a big one this weekend on the interstate and dented and cracked my wheel. If FSD wasn't on, I would have at least taken my foot off the accelerator or maybe even had time to swerve, but FSD proceeded as if nothing was different. I was on a weekend trip away, so it was a pain having to deal with a wheel replacement in order to get home

View attachment 1020116 View attachment 1020117

Yeah pretty high up there on the list of reasons I disengage.

But danger still exists. When actively driving it’s just natural and easy to dodge. Always delayed with FSD.

Here is a specific situation in which FSD in both of our cars performs identically every time we come to this intersection. I will try to find someone at Tesla to share it with....and I am posting it here in hopes that someone at Tesla is patrolling this forum. I will also try to contact the local traffic department to see if the sign can be moved further from the intersection, but this is a great example of how FSD needs to learn what humans would do in this situation. Even without seeing the final 40 mph sign, I can report that all drivers slow dramatically, since they can see the intersection very close ahead. View attachment 1020349

The intersection is 500 ft past the bridge.

I am sure traffic sometimes stacks up and FSD should respond to that, but no need to respond to the 40mph sign early, except to ensure the car is going about 40mph when it is reached. Which should be possible.

 

[PACEMD]

How does a fresh FSD user knows how to avoid nags?

Now don't go getting all fresh with FSD, you will get nagged alright.........

 

[JHCCAZ]

This is interesting and well presented. Well done. I do however question how I would handle this situation without FSD. The sign is clearly stating that the speed “will” be 40 mph but it is also stating “ahead”. I personally would do the same as FSD and maintain speed until the ahead posted speed is reduced to 40 and start decelerating. Rarely see signs like this personally but I believe that’s how I would address it.

I don't remember seeing a sign quite like that, so I didn't know it meant "40 mph Zone somewhere ahead". ( I have seen the more generic .."Reduced Speed Ahead").

So I guess v11 wasn't violating anything but also wasn't preparing for the upcoming speed zone. Again though, V12 should have a better chance of handling this in the proper way.

 

[Ed Hart]

Yeah pretty high up there on the list of reasons I disengage.

But danger still exists. When actively driving it’s just natural and easy to dodge. Always delayed with FSD.

The intersection is 500 ft past the bridge.

I am sure traffic sometimes stacks up and FSD should respond to that, but no need to respond to the 40mph sign early, except to ensure the car is going about 40mph when it is reached. Which should be possible.

That is not how it works. FSD starts slowing when it reaches the sign. Since you know the area, give it a go. Safely! Approach the bridge at 60, then don't let up on the throttle until you are alongside the 40 mpg sign. Few thrills will top that in your day!

 

[Ed Hart]

This is interesting and well presented. Well done. I do however question how I would handle this situation without FSD. The sign is clearly stating that the speed “will” be 40 mph but it is also stating “ahead”. I personally would do the same as FSD and maintain speed until the ahead posted speed is reduced to 40 and start decelerating. Rarely see signs like this personally but I believe that’s how I would address it.

You are going to have to trust me on this one: If you are still doing 60 when you reach that 40 mph sign, you will be standing on the brakes immediately.

 

[Ed Hart]

Nicely presented, thanks! The perception stack labeling is ignoring the unusual 40 mph sign that's enclosed within the warning diamond surround.

Please post back on this after you receive v12 and test it. According to numerous reports, v12 with Auto speed offset is very good at selecting appropriate speeds regardless of the signage.

Will report back when I get V12. Hope to also get the city to move the 40 mpg sign just to do a comparative test.

 

[rlsd]

Here is a specific situation in which FSD in both of our cars performs identically every time we come to this intersection. I will try to find someone at Tesla to share it with....and I am posting it here in hopes that someone at Tesla is patrolling this forum. I will also try to contact the local traffic department to see if the sign can be moved further from the intersection, but this is a great example of how FSD needs to learn what humans would do in this situation. Even without seeing the final 40 mph sign, I can report that all drivers slow dramatically, since they can see the intersection very close ahead. View attachment 1020349

I think this is related to wrong map information. A long stretch at the beginning of Carmel Valley Rd in San Diego has wrong speed info. The speed on this road is 45 and 50 mph. When FSD exits to this road from either west or east of freeway 56, both FSD v11 and V12 goes at max speed 20 mph. It goes almost 1/4 mile and only changes speed to 45 mph when it sees the posted limit 45 on the road. I usually overcome by manually changing max speed. This works with v11 but does not work with V12. The current V12 version doesn't follow max speed set by the user. It's frustrating because this is a heavy commuting artery.

 

[Yelobird]

You are going to have to trust me on this one: If you are still doing 60 when you reach that 40 mph sign, you will be standing on the brakes immediately.

Just stating based on your center picture the speed reduction is very far from the reduce speed to 40 ahead sign. I would (and likely others do) reduce speed to 40 at the sign that states in a mile ahead.

 

[KArnold]

The current V12 version doesn't follow max speed set by the user

Wait. Maybe I missed this. V12 will not let you manually scroll up to a faster speed?

 

[rlsd]

Wait. Maybe I missed this. V12 will not let you manually scroll up to a faster speed?

I scroll up but it V12 mostly ignored. I saw it went up from 20 to 30 when I scrolled up to 40 once. It's not always consistent.

 

[Ed Hart]

Just stating based on your center picture the speed reduction is very far from the reduce speed to 40 ahead sign. I would (and likely others do) reduce speed to 40 at the sign that states in a mile ahead.

You are going to have to trust me on this one: If you are still doing 60 when you reach that 40 mph sign, you will be standing on the brakes - hard! - immediately. If you happen to live near Yorba Linda, why not SAFELY give it a try?

 

[AlanSubie4Life]

then don't let up on the throttle until you are alongside the 40 mpg sign

That is not what I said. I said be going 40mph-ish at the sign as required by law.

I live nowhere near here.

Anyway slowing inappropriately a half mile in advance would be unnecessary and infuriating. Seems like FSD is mostly working fine.

My guess is it fails to properly slow for obviously stopped traffic, but that is a standard issue and kind of unrelated to anything specific here.

Anyway v12 will fix everything.

 

[Supcom]

Wait. Maybe I missed this. V12 will not let you manually scroll up to a faster speed?

V12 has an optional Auto Speed setting that allows the car to set the speed on city streets based on whatever the NNs decide is best. If enabled, there is no user adjustment via the scroll wheel.

It does not apply to interstates - at least not yet.

 

[sleepydoc]

Just stating based on your center picture the speed reduction is very far from the reduce speed to 40 ahead sign. I would (and likely others do) reduce speed to 40 at the sign that states in a mile ahead.

Legally you're required to have your speed reduced to 40 at the 40 MPH sign, not start reducing it at the sign. That's the purpose of the yellow-bordered advisory sign, to warn you so you can start reducing your speed ahead of time. Unfortunately, FSD still reacts to the speed limit signs rather than anticipating the way it should. At least it takes less than ¼ mile to drop its speed now as opposed to previous versions.