FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[kriskros91]

So who is getting excited and antsy for 11.3.x yet? I know watching these influencer videos that I am ready to experience this for myself!!

Watching these videos I feel that the NHTSA recall has only half been addressed. I see the stopping to 0 MPH "issue" addressed (I like some people feel that this was never really an issue (apart from the rolling stop thing which probably should never have been allowed) and felt that it was great already) but I do not see real changes to the stale yellow light behavior at all. I do think there have been better attempts at preventing going straight on turn only lanes as well.

 

[AlanSubie4Life]

Why would a safety-based FSD system be aggressively moving into the intersection when the other car is already creeping forward without stopping to 0mph?

Because there is no risk of a collision, and not obeying traffic norms and laws is dangerous and can lead to collisions.

So who is getting excited and antsy for 11.3.x yet?

It looks like nearly exactly the same code as what we have, with the exception of the recall fixes, except tuned to perform worse, so I am happy to wait.

 

[powertoold]

There's definitely some regression in 4-way stop behavior. I'm thinking it has to do with forcing the 0mph for the recall. Edit: actually, in this case, it's the car incorrectly perceiving the stop as a 2-way instead of a 4-way. We can see that the car is creeping forward for visibility when it doesn't need to and should just proceed:

 

[willow_hiller]

There's definitely some regression in 4-way stop behavior. I'm thinking it has to do with forcing the 0mph for the recall:

It's almost like they've coded "wait until the intersection is clear, and no other vehicle is moving toward the intersection before proceeding." And then human-drivers in the intersection see FSDb's hesitancy and proceed when it's not their turn, causing FSDb to wait again, causing more drivers to enter the intersection, etc. It's a negative feedback loop of overly-cautious software and impatient human drivers.

 

[AlanSubie4Life]

It's almost like they've coded "wait until the intersection is clear, and no other vehicle is moving toward the intersection before proceeding." And then human-drivers in the intersection see FSDb's hesitancy and proceed when it's not their turn, causing FSDb to wait again, causing more drivers to enter the intersection, etc. It's a negative feedback loop of overly-cautious software and impatient human drivers.

Yep, something like that. At least we can definitively say it is not due to the 0mph thing (which was not part of the recall, as we all know - what on earth makes people think this??? That was part of the recall a year ago! Maybe people are messing up their Google search results? Absolutely nothing about 0mph on the latest recall. Just adjusting the perception and profile, that is all!).

Need to tweak the sliders though I think the underlying behavior is broken, as it has always been (I don’t see any fundamental change). The behavior is particularly broken in occluded scenarios, which are common. But broken everywhere, to be clear.

 

[qdeathstar]

Because there is no risk of a collision, and not obeying traffic norms and laws is dangerous and can lead to collisions.


It looks like nearly exactly the same code as what we have, with the exception of the recall fixes, except tuned to perform worse, so I am happy to wait.

Huh??? You have single stack v11?

 

[EVNow]

There's definitely some regression in 4-way stop behavior. I'm thinking it has to do with forcing the 0mph for the recall. Edit: actually, in this case, it's the car incorrectly perceiving the stop as a 2-way instead of a 4-way. We can see that the car is creeping forward for visibility when it doesn't need to and should just proceed:

This is current behavior for me in one of the stops. Probably a map problem.

 

[AlanSubie4Life]

There's definitely some regression in 4-way stop behavior. I'm thinking it has to do with forcing the 0mph for the recall. Edit: actually, in this case, it's the car incorrectly perceiving the stop as a 2-way instead of a 4-way. We can see that the car is creeping forward for visibility when it doesn't need to and should just proceed:

7:30 in that video is a better example of the car:

1) Slowing too early
2) Slowing too slowly particular near 0mph
3) Stopping for too long (looked like about 2 seconds but may have been less given point 2)

Huh??? You have single stack v11?

Pretty sure we all do, with the exception of the freeway behavior and other minor tweaks.

 

[powertoold]

7:30 in that video is a better example of the car:

1) Slowing too early
2) Slowing too slowly particular near 0mph
3) Stopping for too long (looked like about 2 seconds but may have been less given point 2)

1 and 2 are preferential. I don't think it's slowly too early.

3 is due to the NHTSA recall. They want it at 0mph and clearly at 0mph every time.

 

[archae86]

the 0mph thing (which was not part of the recall, as we all know - what on earth makes people think this???

Maybe because they read the 23V-085 report text, and noticed these words:
"2) the perceived duration of the vehicle’s static position at certain intersections with
a stop sign, particularly when the intersection is clear of any other road users"

"static position" might seem rather like 0 mph to them.

 

[Reddy Kilowatt]

Funny enough ... FSD does better with a lead car sometimes. Like in roundabouts or right turns.

Essentially wherever FSD waits for unnaturally long time - if there is traffic that makes it natural to wait, it feels more natural

Both of your comments have been what I notice. A lot of times Im by myself and it dark also. FSD does great. Add more traffic and obsticals its starts having issues.

Following other cars does make it smoother. I have issue with X cross over exit ramps. By myself it just sits there and wont move until I hit the gas. Follow another car and it speeds through the large left turn.

I do still have some issues with sun rise and blinded cameras.

 

[AlanSubie4Life]

Maybe because they read the 23V-085 report text, and noticed these words:
"2) the perceived duration of the vehicle’s static position at certain intersections with
a stop sign, particularly when the intersection is clear of any other road users"

"static position" might seem rather like 0 mph to them.

Exactly.

So I am not sure why people are tying themselves into knots on this. It is about perceived duration - not the duration.

Specifically:

Improved the longitudinal slowdown control profile when leading up to stop sign intersections to make the overall maneuver feel more perceptible and natural.

This likely includes the speedometer display (which is erroneous as we all know). That is my speculation though. And clearly they made adjustments to the profile as well.

Of course, I am sure they have made other slider adjustments to make performance worse, including at stop signs, but that is neither here nor there and rather irrelevant.

I’m not sure why they don’t use the default Tesla drivetrain profile (which is excellent at stopping), possibly adding a tiny bit of braking at the end to speed things up. It’s very mysterious.

 

[qdeathstar]

7:30 in that video is a better example of the car:

1) Slowing too early
2) Slowing too slowly particular near 0mph
3) Stopping for too long (looked like about 2 seconds but may have been less given point 2)


Pretty sure we all do, with the exception of the freeway behavior and other minor tweaks.

Damn, I didn’t know that. I don’t use fsd beta, but I’m still strangely excited to get the next update.

 

[qdeathstar]

Damn, I didn’t know that. I don’t use fsd beta, but I’m still strangely excited to get the next update.

Nah, I still got 10.69….

 

[AlanSubie4Life]

Nah, I still got 10.69….

Me too. But that is what I mean. It’s all basically the same. Look at the release notes for v11! There’s nothing there about major changes, nor are any evident (other than the freeway change, which was of course anticipated and is major, but that is a separate issue).

 

[Supcom]

Exactly.

So I am not sure why people are tying themselves into knots on this. It is about perceived duration - not the duration.

Specifically:


This likely includes the speedometer display (which is erroneous as we all know). That is my speculation though. And clearly they made adjustments to the profile as well.

Of course, I am sure they have made other slider adjustments to make performance worse, including at stop signs, but that is neither here nor there and rather irrelevant.

I’m not sure why they don’t use the default Tesla drivetrain profile (which is excellent at stopping), possibly adding a tiny bit of braking at the end to speed things up. It’s very mysterious.

Perhaps the issue is not the driver's perception? Maybe it's the policeman's perception?

 

[Twiglett]

Not sure its a good sign that 11.3.1 is still built on 2022.4x base, especially when they seem to be getting into their release stride for 2023 already and almost 60% of the fleet on 2023 releases.
Last year there was a bunch of speculation that Tesla were about to release FSD on top of whatever base release was current. That happened for a day or two and now seems to be very far from current practice again.

 

[powertoold]

Here's another case of road debris on the left lane showing up as a blob on the visualization:

 

[AlanSubie4Life]

Perhaps the issue is not the driver's perception? Maybe it's the policeman's perception?

make the overall maneuver feel more perceptible and natural.

Definitely seems like that is not what being described. It is clearly indicates that this is about the driver’s perceptions.

For the policeman it would be “look” - and it would also be silly because the car either goes to 0mph or it doesn’t and the telemetry will be available to the accused and admissible as evidence. Maybe it isn’t that granular, but my point is that the car stops, or it doesn’t.

Right now it stops, and can show 1mph as the minimum speed before proceeding (a matter of trivia).

 

[Supcom]

Definitely seems like that is not what being described. It is clearly indicates that this is about the driver’s perceptions.

For the policeman it would be “look” - and it would also be silly because the car either goes to 0mph or it doesn’t and the telemetry will be available to the accused and admissible as evidence. Maybe it isn’t that granular, but my point is that the car stops, or it doesn’t.

Right now it stops, and can show 1mph as the minimum speed before proceeding (a matter of trivia).

So, NHTSA is now concerned about the driver's feelings? Curious. For safety, one would think that the concern would be how the other driver's perceive your stop.

No matter, stop sign performance is still horrible with no real logic improvements. Just increased the minimum stoppage time.

Other recall fixes are equally half-assed, unfortunately. The car now frequently runs red lights instead of stopping for stale yellow lights. That's a complete and total failure/

Slowing down for reduced speed limits, for the few tests I've seen online, is no better than before. At least, there is still zero attempt to begin slowing before passing a speed limit sign. Making matters worse, the car cannot figure out how to speed up on an onramp outside of California. 30 mph on an onramp? For real?

In general, speed limits seems to change randomly while driving down a street. Very strange.

Oh, and the car does appear to get out of turn lanes when it gets in them by accident by stopping before the lane line goes solid and waiting for an opening. Very much not like a human and not likely to win over new friends on the road.

I do like the quicker lane changes on highways. That's the only good thing I've seen with this release. I am amused to see the car repeatably act like an idiot squeezing in at the head of a line of cars.

Glad that Tesla didn't roll this stinker out wide.