FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[EVNow]

Which FSD version is this?

Nice thing is the current version can actually do this - in almost all cities. Tesla has come a long way in 3 years.

 

[EVNow]

Yeah this is good but I have seen videos with highway drives with people disengaging at times. Truthfully highway NoA has gotten so so so good that I barely ever have a disengagement except for a particular reason (mostly personal preference). To me any amount of unexpected disengagement would be a fairly noticable regression.

Any particular reason you could gather for the disengagement ?

 

[Ramphex]

Nice thing is the current version can actually do this - in almost all cities. Tesla has come a long way in 3 years.

When it can actually do that, with no nags, is when I’ll be impressed.

 

[Ramphex]

I don’t care when or whether you are impressed or not

It’s clear to me you do not have what it takes to learn to make use of FSDb properly in its current form. Nothing wrong with that …

Who said anything about learning to use FSDb? I’m talking about FSD…. And it’s current status…. And Tesla’s 3 year old video of it that they still proudly sport on their YouTube channel. According to the video, you just double tap the scroll wheel and sit there while the car takes you places.

I’m not sure what you’re saying here… are you saying that’s not how it works, there’s more steps to it, and learning tricks to make it work? Or are you saying we were lied to?

 

[aronth5]

The reason people were expecting regressions was because the current NoA stack is quite mature and stable. As such, rewriting and combining it with the rest of FSD meant that even small bugs would cause regressions.

Highways, especially restricted highways are so much simpler than city/streets that testing the single stack had to be much easier for Tesla especially with some employees using V11 for almost a year. Clearly that time was well spent and for once Tesla did adequate QA.. There was probably a lot less code rewriting then you thought since they were able to leverage the work on city/streets.

 

[jabloomf1230]

When it can actually do that, with no nags, is when I’ll be impressed.

The "nags" are in place because anyone with a Tesla and the money to purchase FSD can drive all over the US and Canada. That's a whole lot of jurisdictions that need to be comfortable with the concept. The geofenced approaches only have to cope with a few jurisdictions. And we've already seen one of them (SF) with officials expressing concerns. The whole AD issue hinges on attaining a safety standard well above the median value.

My personal view, based on financial considerations alone, is that robotaxis make sense in the short term (5 years) for buses and smaller people movers that operate at low speeds.

 

[Ramphex]

The "nags" are in place because anyone with a Tesla and the money to purchase FSD can drive all over the US and Canada. That's a whole lot of jurisdictions that need to be comfortable with the concept. The geofenced approaches only have to cope with a few jurisdictions. And we've already seen one of them (SF) with officials expressing concerns. The whole AD issue hinges on attaining a safety standard well above the median value.

My personal view, based on financial considerations alone, is that robotaxis make sense in the short term (5 years) for buses and smaller people movers that operate at low speeds.

The point is that FSD still can’t do what was advertised in Tesla’s video on YouTube 3 years ago.

 

[Knightshade]

The "nags" are in place because anyone with a Tesla and the money to purchase FSD can drive all over the US and Canada. That's a whole lot of jurisdictions that need to be comfortable with the concept.

Currently 0 state/local jurisdictions in the areas you cite have any regulations of L2 ADAS systems like Teslas at all.

The whole AD issue hinges on attaining a safety standard well above the median value.

Among the states that DO allow >L2 systems, 0 of their laws have any requirement whatsoever to demonstrate a specific safety standard above any value at all.

They simply require the maker of the car/system to certify to the state that:
The system is capable of self driving as self-certified by the car/system maker (typically leveraging or often directly quoting the SAE J3016 level wording for what that means)

The system will obey all traffic laws

The maker of the car/system has insurance since they will be liable if the system/car causes an accident.

That's it.

 

[jabloomf1230]

The point is that FSD still can’t do what was advertised in Tesla’s video on YouTube 3 years ago.

True, but even if Tesla removed the nags, the video is at odds with the present state of FSD.

 

[jabloomf1230]

Currently 0 state/local jurisdictions in the areas you cite have any regulations of L2 ADAS systems like Teslas at all.




Among the states that DO allow >L2 systems, 0 of their laws have any requirement whatsoever to demonstrate a specific safety standard above any value at all.

They simply require the maker of the car/system to certify to the state that:
The system is capable of self driving as self-certified by the car/system maker (typically leveraging or often directly quoting the SAE J3016 level wording for what that means)

The system will obey all traffic laws

The maker of the car/system has insurance since they will be liable if the system/car causes an accident.

That's it.

Regulations may be lacking, but public perception is more important. The media is focused on any new technology and its potential to create risks to the public. If AD vehicles cause little or no issues, then their use will spread. But if crashes occur, the negative impact on public opinion will likely be out of proportion to the actual threat to vehicular safety. Government rulemaking may follow as a consequence.

If a Tesla with FSD gets in a crash, Tesla can examine vehicle data to see what the actual cause was. With nags applied after 15 seconds ± of inattentiveness, Tesla has another option for teasing out the cause of the crash. When the GMen come a callin', it's one more tool to use in understanding the crash.

 

[dacpdx]

Regulations may be lacking, but public perception is more important.

No, Performance is important.

In college chemistry lab, 40 years ago, on the first day the T.A. gives an overview of the course, expectations, grading, etc. Then we all were given a chance to ask questions and just about every question was like: how much does homework count (for the grade)?; how are exams weighted? etc. The T.A. tried to answer these questions, but was clearly not pleased. Finally he said, in his Indian accent, "go after the knowlege, and the grade will follow" Great advice.

So, FSD needs to perform. If it does, the public and regulators will accept it.

 

[Ramphex]

No, Performance is important

The public thinks that FSD is out and that it does exactly what’s shown in that 3 year old video. They don’t know that only the beta is out and even that can’t perform as well as the video.

 

[EVNow]

Who said anything about learning to use FSDb? I’m talking about FSD…. And it’s current status…. And Tesla’s 3 year old video of it that they still proudly sport on their YouTube channel. According to the video, you just double tap the scroll wheel and sit there while the car takes you places.

I’m not sure what you’re saying here… are you saying that’s not how it works, there’s more steps to it, and learning tricks to make it work? Or are you saying we were lied to?

Thats how it works now. If you had spent some time to learn how to use FSDb you would have found out.

BTW, you replied to my post and actually reported that innocuous post for "snippiness" ? LOL.

 

[sleepydoc]

Highways, especially restricted highways are so much simpler than city/streets that testing the single stack had to be much easier for Tesla especially with some employees using V11 for almost a year. Clearly that time was well spent and for once Tesla did adequate QA.. There was probably a lot less code rewriting then you thought since they were able to leverage the work on city/streets.

True, but there can always be bugs and if Tesla’s trying to introduce any new features to they highway stack at all then they have a chance to creep in. There are also a lot of seemingly simple areas in city driving that FSDb regularly messes up.

 

[sleepydoc]

The "nags" are in place because anyone with a Tesla and the money to purchase FSD can drive all over the US and Canada. That's a whole lot of jurisdictions that need to be comfortable with the concept. The geofenced approaches only have to cope with a few jurisdictions. And we've already seen one of them (SF) with officials expressing concerns. The whole AD issue hinges on attaining a safety standard well above the median value.

My personal view, based on financial considerations alone, is that robotaxis make sense in the short term (5 years) for buses and smaller people movers that operate at low speeds.

No, the nags are because people kept doing stupid things (like climbing out of the passenger seat on the highway, watching movies, falling asleep, etc.) As @jabloomf1230 said, public perception matters and Tesla has had numerous incidents where it was blamed (often prematurely) for accidents. To be sure, Elon plays into that as well, but regardless, public perception can lead to backlash, regulation and lower sales, so it does matter.

So, FSD needs to perform. If it does, the public and regulators will accept it.

but how much of their acceptance is due to the actual performance vs their perception of its performance?

Public perception is not rational. If there are 99 accidents due to people being stupid and abusing the system human nature means people will focus on the 1, not the 99
(we've even seen Tesla being blamed when autopilot & FSD weren't even engaged.)

Elon himself stated something similar with respect to FSD accident rates, noting that even if a system improves accident rates over humans it needs to be significantly better for people to accept it because people focus on the accidents that happen, not the ones that don’t.

 

[kabin]

I wonder what fixes will be priority for V11.3.2's release and how many 2 weeks it will take. Hopefully recall items will finally be put to bed. I wonder how NHTSA decides if recall items are resolved? Would five sevens be sufficient?

 

[JulienW]

I wonder what fixes will be priority for V11.3.2's release and how many 2 weeks it will take. Hopefully recall items will finally be put to bed. I wonder how NHTSA decides if recall items are resolved? Would five sevens be sufficient?

Hope we see a release in the next week and wouldn't be surprised if it was 11.4 in a 23.x.x container. Probably will just to the same influencers with all the others influencers added. Then likely in a few days if all is well Tesla will start the SLOOOOOOW rollout to a few of us regular Beta users (sure would love to "lotto" in that group".

On NHTSA I believe the fixes are completely up to Tesla. So the recall will be resolved when Tesla notifies NHTSA that the fixes are "installed". Then NHTSA will be on the lookout for new complaints.

Also what is 5 7s?

 

[AlanSubie4Life]

Then likely in a few days if all is well Tesla will start the SLOOOOOOW rollout to a few of us regular Beta users (sure would love to "lotto" in that group".

Based on the large number of improvements still needed I expect a couple months.

Also they seem to be using all the same code as 10.69.x - it really looks basically identical, and they have said as much - so there’s a question about whether they will make any big changes as they have made noise about (and deal with all the regressions) before wider release. In that case could be 3-4 months or more. Though I suspect that won’t happen and they will punt the full rewrite until later.

 

[Ramphex]

Based on the large number of improvements still needed I expect a couple months.

Also they seem to be using all the same code as 10.69.x - it really looks basically identical, and they have said as much - so there’s a question about whether they will make any big changes as they have made noise about (and deal with all the regressions) before wider release. In that case could be 3-4 months or more. Though I suspect that won’t happen and they will punt the full rewrite until later.

Which is interesting, considering Shillmasterflex9000 (Musk) said monthsssss ago that 11 is fire.

 

[kabin]

It's funny how sensitive FSD can be with light rain. Or is it a more random-like response? In this case the roads appear dry with mist like moisture on the windshield and FSD warns about possible performance degradation. Chuck Cook was able to use V11.3.1 in far worse weather. Whatever the cause they need to dial it in so it isn't the little boy crying wolf.

.