FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[archae86]

Does anyone actually use FSD beta after they've been given it and tested it out a few times? I stopped using it. In the maybe 10 instances I've used it it failed/freaked-out trying to do a U-turn and generally just moves too slow or brakes akwardly causing me anxiety and clearly confusing drivers behind me.

I have had FSDb on one car since June 2022, and added another at the turn of this year. I estimate I've had FSDb engaged for about 80% of the miles driven in that time. Yes, it does strange things often, and alarming things now and again that keep me on my toes. But I believe in the current state that it driving with me very closely monitoring and sometimes intervening is a safer combination than me driving myself.

It is also way more interesting than just driving along my usual paths yet again.

 

[powertoold]

It is also way more interesting than just driving along my usual paths yet again.

Yes! It brings some drama and excitement to my drives lol. This is an underappreciated benefit I think.

 

[Mardak]

Does anyone actually use FSD beta after they've been given it and tested it out a few times?

Some people like me have been using Autosteer on city streets well before FSD Beta was available even though it did some pretty dangerous things like attempting a turn then aborting. I would guess over 95% of miles were on Autopilot before FSD Beta with the main limitation of it only allowed activation with lane lines (thus excluding many residential streets here).

Probably 80% of my miles are on FSD Beta now easily totaling over 10k miles since October 2021. With FSD Beta 11 handling highways too, it'll basically be 100% of miles for me. Especially with the alternative navigation routes functionality, I have FSD Beta take city streets and avoid highways even if it takes a few minutes longer just to test it out a little bit more.

 

[Supcom]

Does anyone actually use FSD beta after they've been given it and tested it out a few times? I stopped using it. In the maybe 10 instances I've used it it failed/freaked-out trying to do a U-turn and generally just moves too slow or brakes akwardly causing me anxiety and clearly confusing drivers behind me.

I have been using FSDb daily for over a year.

 

[BritneySpears]

I have had FSDb on one car since June 2022, and added another at the turn of this year. I estimate I've had FSDb engaged for about 80% of the miles driven in that time. Yes, it does strange things often, and alarming things now and again that keep me on my toes. But I believe in the current state that it driving with me very closely monitoring and sometimes intervening is a safer combination than me driving myself.

It is also way more interesting than just driving along my usual paths yet again.

I should clarify this is in regard to version 10.69.25.1 (FSD beta). I frequently use autopilot (lane keep and ACC) on the highway and even on streets but to navigate somewhere on non-highway streets and use FSD beta - that's the annoyance. I really don't see how anyone thinks it's anything but a fun-to-watch gimick for now.

 

[Supcom]

I should clarify this is in regard to version 10.69.25.1 (FSD beta). I frequently use autopilot (lane keep and ACC) on the highway and even on streets but to navigate somewhere on non-highway streets and use FSD beta - that's the annoyance. I really don't see how anyone thinks it's anything but a fun-to-watch gimick for now.

Everyone's FSDb experience is different. But, for me, driving in the Dallas, TX area, FSDb works quite well these days. It's fairly predictable where it will fail and I regularly have drives with no disconnects. The most common intervention I have is to apply a little acceleration when FSDb is too hesitant at a turn.

It does take time to become accustomed to it. The biggest thing is to get used to the fact that FSDb does not drive like you (I hope!). I think that is a hard thing for a person sitting in the driver's seat to accept.

 

[Gbills]

Everyone's FSDb experience is different. But, for me, driving in the Dallas, TX area, FSDb works quite well these days. It's fairly predictable where it will fail and I regularly have drives with no disconnects. The most common intervention I have is to apply a little acceleration when FSDb is too hesitant at a turn.

It does take time to become accustomed to it. The biggest thing is to get used to the fact that FSDb does not drive like you (I hope!). I think that is a hard thing for a person sitting in the driver's seat to accept.

If everyone says it doesn’t drive like them, ( like I read all the time) the driver model for FSDb must be in the minority.

 

[tmoz]

If everyone says it doesn’t drive like them, ( like I read all the time) the driver model for FSDb must be in the minority.

Like Robot speech? You can understand what it is saying, but no human talks like that.

 

[BritneySpears]

Everyone's FSDb experience is different. But, for me, driving in the Dallas, TX area, FSDb works quite well these days. It's fairly predictable where it will fail and I regularly have drives with no disconnects. The most common intervention I have is to apply a little acceleration when FSDb is too hesitant at a turn.

It does take time to become accustomed to it. The biggest thing is to get used to the fact that FSDb does not drive like you (I hope!). I think that is a hard thing for a person sitting in the driver's seat to accept.

It's no like me but especially not like other drivers on the road near my car and that's the reason I stopped using it. I always hit the accelerator to move it along and at stop signs it hogs the entire lane preventing other cars pulling up next to me to turn right - so I just quit. Lane keep assist and ACC is great when not navigating. I love that in any car but I just don't get how people enjoy navigating on FSD beta when it requires consistent intervention. I guess for some people it's fun and interesting.

 

[Mardak]

I always hit the accelerator to move it along and at stop signs it hogs the entire lane preventing other cars pulling up next to me to turn right - so I just quit

Yes, I also need to help it along to be courteous to others on the road, but how I see it is if I wasn't using FSD Beta, I would need to do those things anyway like accelerating from a stop or sharing the road at a turn. Specifically for letting others pull up, most of the time after doing so manually, FSD Beta will continue just fine even if it's not in its originally desired location, so FSD Beta still provides value.

I'm sure this type of behavior will still exist with FSD Beta 11 on highways of it doing things people wouldn't do such as existing Navigate on Autopilot's desire to yield extra early for merging onramp vehicles, but FSD Beta will probably improve on that by being smarter in calculating/predicting the need to yield based on the actual movements of the other vehicles instead of legacy stack's behavior mostly based on another vehicle existing resulting in many complaints of phantom braking.

 

[powertoold]

Basically, if there's no one behind me, I just let FSD do its thing.

95% of the time, I'm only worried about being rear-ended.

 

[EVNow]

Potentially yes, but we can't see them. Each accident reports "ADAS/ADS Version" as well as "Within ODD?" and "Narrative" but all of Tesla's entries are "[REDACTED, MAY CONTAIN CONFIDENTIAL BUSINESS INFORMATION]." For example, there's a likely ADAS entry for the San Francisco Bay Bridge crash with the driver saying he had FSD Beta, but there doesn't seem to be any noticeable difference in the NHTSA public data for that vs regular Autopilot.

Do you have the link to the reports handy ? Does it show whether the road is highway or city ?

 

[Dewg]

It's no like me but especially not like other drivers on the road near my car and that's the reason I stopped using it. I always hit the accelerator to move it along and at stop signs it hogs the entire lane preventing other cars pulling up next to me to turn right - so I just quit. Lane keep assist and ACC is great when not navigating. I love that in any car but I just don't get how people enjoy navigating on FSD beta when it requires consistent intervention. I guess for some people it's fun and interesting.

I've been saying this for some time - Beta is for a certain type of person and not for everyone. It takes a commitment to the process, and a desire to test, report, and help improve it over time. You "just don't get how people enjoy navigating on FSD beta when it requires consistent intervention". It's because we want to be Beta testers and we want to help improve the system for people like you who benefit from our time and effort.

 

[EVNow]

Does anyone actually use FSD beta after they've been given it and tested it out a few times? I stopped using it. In the maybe 10 instances I've used it it failed/freaked-out trying to do a U-turn and generally just moves too slow or brakes akwardly causing me anxiety and clearly confusing drivers behind me.

I use it for 95% of my city driving.

Only times I don't use FSD b in the city are
- Its raining heavily and FSDb has fallen back to AP or doesn't turn on
- Very short trips (either to get on freeway or from one shop to another)
- Within parking lots
- Heavy construction zone

Anyway its like a catch 22 situation
- The more you use FSDb, the more you get used to it and will start using it more often ...
- The less you use FSDb, more scary it seems and you will use it even lesser ...

 

[Mardak]

Does it show whether the road is highway or city ?

Yes, you can see "Roadway Type" and probably infer from "Posted Speed Limit (MPH)" and some other data. You can download the csv data with the "Level 2 ADAS Incident Report Data" button from Standing General Order on Crash Reporting | NHTSA

I believe Report ID 13781-4338 (currently row 50) is the San Francisco Bay Bridge crash that FSD Beta 11 single stack might have avoided.

 

[Rikster]

I'm definitely in the group that thinks FSD Beta is useful and fascinating to use most of the time. But as others have pointed out, it definitely has it's quirks, and still has to be monitored pretty closely.

It doesn't drive as well as I do (or at least it doesn't drive the same as I do), but I've had a lot of intervention-free drives. I've had many drives where my car would have made it safely from point A to point B without me even being in the car (though of course that's not an option just yet). It would have annoyed other drivers at time due to it's indecision, but it would have arrived safely at the destination, which is saying a lot, all things considered.

If I'm in a hurry to get somewhere, I will drive myself as that's a bit faster. If there's a lot of traffic, I'll disengage a lot for convenience and to avoid annoying others. I don't get any dangerous phantom braking, but do get quite a few minor slow downs (2-4 mph) where my car thinks it has seen something dangerous.

With light traffic and if I'm not in a hurry, I enjoy letting my car drive itself. There are some spots where it predictably and consistently makes the wrong lane decision, but I just hit the turn signal to help coax it into the correct lane and it does fine.

Overall, I'm happy to be a part of this "experiment," and wouldn't take a refund of the $10k I paid for FSD if offered today.

 

[tivoboy]

I use it for 95% of my city driving.

Only times I don't use FSD b in the city are
- Its raining heavily and FSDb has fallen back to AP or doesn't turn on
- Very short trips (either to get on freeway or from one shop to another)
- Within parking lots
- Heavy construction zone

Anyway its like a catch 22 situation
- The more you use FSDb, the more you get used to it and will start using it more often ...
- The less you use FSDb, more scary it seems and you will use it even lesser ...

I still don’t use it around a school zone at all, and certainly not during the morning or afternoon coming and goings - if I’ve been so stupid as to come and go at the same time!

And I still can’t get it to correctly navigate the speed and approach with many of the speed bumps in our area - many of which have been updated with even better markings these days but it has made no better performance

 

[wknickless]

If I'm in a hurry to get somewhere, I will drive myself as that's a bit faster. If there's a lot of traffic, I'll disengage a lot for convenience and to avoid annoying others. I don't get any dangerous phantom braking, but do get quite a few minor slow downs (2-4 mph) where my car thinks it has seen something dangerous.

With light traffic and if I'm not in a hurry, I enjoy letting my car drive itself. There are some spots where it predictably and consistently makes the wrong lane decision, but I just hit the turn signal to help coax it into the correct lane and it does fine.

Ditto - but I also drive myself if I'm on the phone. I've spent too many decades splitting my attention between a spoken conversation and what other cars are doing on the road. Having to monitor the car's driving while splitting my attention on a phone call, even hands-free? Nope.

 

[mtbwalt]

I use FSD beta for a good bit of my driving. Frankly I think it helps me avoid tickets by keeping my speed under the ticket threshold. It has matured a LOT over the last year IMO.

My main irritation is nonsensical or inappropriate lane choice/changes. I’d love to have more granular override of when it is allowed to change lanes without asking me. I’d also like it to learn my preferred lanes for to 20 or so trips I take most regularly. Just that would probably cover 90% of my driving.

 

[kriskros91]

When do you guys think we'll have school zone awareness and/or no turn on red? Feels like those are easy enough to implement and absolutely necessary to fix ASAP because you cannot be a full self driving car without understanding these (non) edge cases.