FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[lzolman]

Agreed, I think we've identified the main issues with 11.3.6. Now that there are fixes on the way which Tesla appears to have acknowledged and addressed in 11.4 I'm focused on when will it be released outside of the employee realm. I'm guessing it will be within a week, two at the most, based on how quickly updates have been flowing recently. As someone else mentioned, exciting times!

What I did not see in the 11.4 notes was any reference to an improvement in navigating unprotected left turns, especially when there's no traffic. It's got to get less sluggish.

 

[AlanSubie4Life]

Purely due to a tailgating and/or distracted motorist, actually.

As a driver you have a duty to avoid accidents, whether they are your fault or not.

 

[powertoold]

11.3.6, despite all the minor unnecessary slowdowns, is getting me to and from a lot of places without disengagement. It's also quite good at windy mountainous roads.

Can't wait to see if 11.4 reduces the slowdowns. Despite what all y'all's think, I think Tesla is very close to solving fsd to 2x+ human safety.

 

[dckiwi]

I feel like if FSD had a Trello board it would be like 1000 pages long of TODOs. Watch I bet you next year they'll start working hard on making roundabouts work properly for the first time. They just (for the first time), in 11.4, are shipping a model that attempts to drive at the proper speed for road conditions and weather. You know, just basic stuff like driving at an acceptable speed. Maybe one day it'll handle school zones, or interpret per-lane navigational signs instead of using map data.

I'm not trying to hate on the pace of progress or say the AP team is bad, it's just that Elon has to be aware of these things. It's great stuff taken on its own, one of the coolest software products out there, and I like being able to follow along and test out changes personally, it's just how Elon represents the state of FSD. If anyone works on AP reads this, you guys are awesome, please do an AMA.

When there is a new build I'm super excited to try it out. When my iPhone gets a new update it's the most boring time on earth, maybe some new emoji were added this time, I couldn't tell? These guys at Tesla are solving actual problems and we get to experience the progress in near real-time. But Elon does such a disservice to everyone working on it by massively misrepresenting the current state of the thing, like the fact they were talking to investors about robotaxis years ago and how your Tesla is going to appreciate to $500k because it will soon become a taxi driving 24/7 for you. He has to have seen the backlog for this thing, he sits in the weekly meetings, give me a break. I expect him to be a cheerleader, that is kind of his job, but man it's just too much, too long.

This x 1000.

If he had just kept quiet and let the team do their thing, everyone would be blown away with the FSD capabilities we have today. But because he said it was all done back in 2016 (and in hindsight it's clear they hadn't even started working on FSD), the narrative is that Tesla failed because we don't have robotaxis yet.

 

[Ramphex]

By misusing the term so much you have devalued it. Sad.

Yea, you should see it in person. I went from greeting my homies with “sup dawg?!” to “sup shill?!”

 

[kabin]

I take those impressive % improvements with a grain of salt as there's too many unknowns. A few releases back several release notes reported huge improvements with cross traffic kinematics and associated needless braking and I still experience it with V11.3.6 but now get the added bonus of near misses.

 

[JB47394]

If he had just kept quiet and let the team do their thing, everyone would be blown away with the FSD capabilities we have today.

But how many people would have ponied up $8,000, $10,000, $12,000, or $15,000 for it? My guess is that everyone interested would go with the subscription and wait for the completed implementation before considering buying it. Tesla took in over $1 billion in deferred revenue as a result of Elon claiming that they'd have it working in a year or two.

It's not ethical, but it worked.

With people literally invested in the beta program, I'm sure there's a greater desire by owners to test the software to see what they got for their money. Subscribers would likely be more inclined to test it for a month, see what it can do, then end the subscription. That's going to limit the volume of test data that Tesla has to work with.

 

[kabin]

I feel like if FSD had a Trello board it would be like 1000 pages long of TODOs. Watch I bet you next year they'll start working hard on making roundabouts work properly for the first time. They just (for the first time), in 11.4, are shipping a model that attempts to drive at the proper speed for road conditions and weather. You know, just basic stuff like driving at an acceptable speed. Maybe one day it'll handle school zones, or interpret per-lane navigational signs instead of using map data.

Much refinement has been neglected so they can spend more time hiding shortcomings with kludgy short cuts.

I fear roundabouts, losing the crawl, human-like stop sign response, and general decisiveness has zero chance until HW4 leaves the nest.

 

[Ramphex]

But how many people would have ponied up $8,000, $10,000, $12,000, or $15,000 for it? My guess is that everyone interested would go with the subscription and wait for the completed implementation before considering buying it. Tesla took in over $1 billion in deferred revenue as a result of Elon claiming that they'd have it working in a year or two.

It's not ethical, but it worked.

With people literally invested in the beta program, I'm sure there's a greater desire by owners to test the software to see what they got for their money. Subscribers would likely be more inclined to test it for a month, see what it can do, then end the subscription. That's going to limit the volume of test data that Tesla has to work with.

If the subscription was available when I got it, even if I was dumb enough to keep it for the entire ownership of my current vehicle, it would be cheaper than $10k I spent on buying it. FSD is something I’ll probably never see the end product before upgrading my car. I’m 100% positive I’m not adding FSD to my next Tesla, if that’s my next car choice, especially not for the current price.

 

[Supcom]

and it would not be the fault of the person behind me.

Actually, it would be the fault of the tailgater. It's always the fault of the tailgater, unless your car in front acted to intentionally cause the accident. It is the trailing driver's responsibility to maintain safe following distance from the car ahead. That distance must account for unexpected braking, whether in response to a child running into the street from behind a parked car or the lead car having a failure that results in a rapid deceleration.

Sadly, most drivers get up close to the lead car so that other cars cannot cut in between them. They do so at their risk.

 

[Supcom]

As a driver you have a duty to avoid accidents, whether they are your fault or not.

While you have no direct control over how closely other vehicles follow you, the simple solution to tailgaters is to just gradually slow down until they get the message and pass you. Nonetheless, if the tailgater persists and runs into you, it will be his fault.

 

[Ramphex]

While you have no direct control over how closely other vehicles follow you, the simple solution to tailgaters is to just gradually slow down until they get the message and pass you. Nonetheless, if the tailgater persists and runs into you, it will be his fault.

There’s always a less annoying option. Put on the hazards and move towards the side of the road.

 

[DelPhonic1]

No, I agree totally with Alan above that, right now (11.3.6), using FSD with traffic at all nearby to my rear is inviting my getting rear-ended, and it would not be the fault of the person behind me. I actually won't use it off the highway with traffic behind me, and since a recent incident ON the highway where it seemed I was the one heading toward rear-ending some slowing traffic in front of me (I disengaged before finding out if that was going to be my fate), I wouldn't even use it on the highway unless I'm willing to remain ultra-vigilant. As it's hard to remain that vigilant for a long time, I just don't use it much. I test each new version until it does something I'm not willing to have it do on a frequent basis. That usually doesn't take long.

It's always the persons fault that rear ends you. Always.

 

[Mark II]

It's always the persons fault that rear ends you. Always.

From the original message" It tried to change lanes quite abruptly and unnecessary and then slammed the brakes and the car behind me was not suspecting it and had to hard brake and honked at me.

 

[jebinc]

No worries. I hear V12 will transition to the latest version of Fortran for a 25% improved FSD arcade-feel over V11. It's gonna be the bomb!

And it will likely be “junk”’as well!

 

[dgatwood]

Someone should do the math on this. To me it seems like there is a hidden category here (presumably false positives on cut-ins which are not due to lane changes into adjacent lanes…and perhaps there are other categories). Because overall reduction lowered by a bit less than the individual percentages…so no weighted average could work.

I could be wrong, but I suspect that the improvement numbers can't be directly compared with one another — that the 39% and 66% are one kind of number on one scale, and the 33% is a different kind of number on a different scale.

The way I read that sentence is:

• "Improved recall for partial cut-ins by 39%: The predicted path of a hidden vehicle that reappears is 39% more accurate than before (or perhaps the vehicle is 39% more likely to still be remembered in the first place).
• "Improved... precision for false positive cut-ins due to lane changes into adjacent lanes by 66%": The predicted path of a non-hidden vehicle that tries to creep into your lane is 66% more accurate.
• "resulting in a 33% reduction in overall lane-changing prediction error": The number of mistakes in deciding whether to brake or not are down by 33%.

So the 33% number isn't comparable to the 39% or 66%, because one is a path fitting measurement (how far off the predicted path is compared with the actual path), and one is a measure of how often the path fit was bad enough to cause an incorrect decision, which depends not only on the precision, but also on the path itself (more specifically, how on close they got to the line).

Now how one interprets a positive number for an improvement of precision... i have absolutely no idea. Perhaps it means that the predictions are 66% or 39% more likely to be within the minimum margin of error that they consider acceptable?

But this is mostly guessing on my part.

There’s always a less annoying option. Put on the hazards and move towards the side of the road.

What's the fun in being less annoying?

 

[JHCCAZ]

And it will likely be “junk”’as well!

I think you misplaced that invitation to your local chapter of Optimist International

 

[jebinc]

I think you misplaced that invitation to your local chapter of Optimist International

Good one! It’s tough to be optimistic on this one, given the long storied history of Charlatan misdirects (I.e., lies).
@Ramphex

 

[Bladerskb]

11.3.6, despite all the minor unnecessary slowdowns, is getting me to and from a lot of places without disengagement. It's also quite good at windy mountainous roads.

Can't wait to see if 11.4 reduces the slowdowns. Despite what all y'all's think, I think Tesla is very close to solving fsd to 2x+ human safety.

Lmao, so you drive what 100 miles without safety disengagement and that means they are close to a million miles without safety disengagement?
Where are you driving?

 

[Ramphex]

I could be wrong, but I suspect that the improvement numbers can't be directly compared with one another — that the 39% and 66% are one kind of number on one scale, and the 33% is a different kind of number on a different scale.

The way I read that sentence is:

• "Improved recall for partial cut-ins by 39%: The predicted path of a hidden vehicle that reappears is 39% more accurate than before (or perhaps the vehicle is 39% more likely to still be remembered in the first place).
• "Improved... precision for false positive cut-ins due to lane changes into adjacent lanes by 66%": The predicted path of a non-hidden vehicle that tries to creep into your lane is 66% more accurate.
• "resulting in a 33% reduction in overall lane-changing prediction error": The number of mistakes in deciding whether to brake or not are down by 33%.

So the 33% number isn't comparable to the 39% or 66%, because one is a path fitting measurement (how far off the predicted path is compared with the actual path), and one is a measure of how often the path fit was bad enough to cause an incorrect decision, which depends not only on the precision, but also on the path itself (more specifically, how on close they got to the line).

Now how one interprets a positive number for an improvement of precision... i have absolutely no idea. Perhaps it means that the predictions are 66% or 39% more likely to be within the minimum margin of error that they consider acceptable?

But this is mostly guessing on my part.



What's the fun in being less annoying?

Funny enough, the US is so overly regulated and the people still suck at driving. I’ve been in Philippines for 7 days now and the driving here is a complete *sugar* show…. But the crazy part is…. It works. Legit. I have yet to see a single crash and it’s the most bizarre type of driving that pretty much daily violates most of the road rules in the US.