FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Pdubs]

When I'm in FSDb, which is nearly all the time, I routinely look ahead for reasons the car will need to slow anyway, and use the right scroll wheel to dial down the current max speed progressively before FSDb would have started slowing. I'm pretty sure I am avoiding some brake wear this way, and even saving a little energy consumption. I also just like the drive better that way.

I do similar, except I exit out of FSDwtf and send in a voice message.…“YOU REALLY HAVE NO SPATIAL AWARENESS” or, “Dude! READ THE ROOM” or my all time favorite, “WTF”???

Then when it slows down using regen, I reengage the FSDbj

 

[AlanSubie4Life]

or treat

It is certainly not a treat.

 

[AlanSubie4Life]

I do similar, except I exit out of FSDwtf and send in a voice message.…“YOU REALLY HAVE NO SPATIAL AWARENESS” or, “Dude! READ THE ROOM” or my all time favorite, “WTF”???

Then when it slows down using regen, I reengage the FSDbj

This is the way.

 

[Pdubs]

FSDx (where “x” = whatever the *beep* you want) — Charlatan trick, or treat?!

Off topic for this thread but I found a work around for when my car refuses to change lanes when I use a turn signal. If I hit the accelerato, she makes the lane change. Seriously. Turn indicator, no response, hit accelerator, lane change. It is repeatable.

 

[jebinc]

Off topic for this thread but I found a work around for when my car refuses to change lanes when I use a turn signal. If I hit the accelerato, she makes the lane change. Seriously. Turn indicator, no response, hit accelerator, lane change. It is repeatable.

1-800….

 

[Dennisis]

there's no reason for the car to choose to go into the right turn lane when the route clearly indicates pulling into the left turn lane, which has not changed for decades.

I had similar experiences on a recent trip. The route was clearly indicated as straight ahead but the car signaled and moved quickly into random right turn lanes. Prior to 11.4.7.X this wasn't an issue for me.

 

[drtimhill]

According to reports to the CA DMV Waymo has one disengagement per 17k miles driven (2022). In city driving Tesla has a disengagement ever 5-15 miles according to FSD Community tracker 2023. That's 1000-2000x.

If we only count critical* DE:s for Tesla they are at about 35 miles per DE, which would only be 500x worse than Waymo, but 1000x worse than the top reporters 2022.

So I'd say "three orders of magnitude" (1000x) is a pretty fair ballpark estimate all things considerered.

*) Avoid accident, taking red light/stop sign, wrong side of the road, unsafe action

Waymo cars can drive in small geofenced areas of 4 major cities, about 400 square miles. Tesla's FSD can drive in any part of the USA and Canada, about 10,000,000 square miles. Sounds like 25,000x better to me. Or, if you want factor in your 1000x argument (valid or not), FSD is 25x better at driving overall, since Waymo cannot drive AT ALL outside its geofenced areas.

Anyone can cherry pick statistics to suit their personal bias. The point is Waymo has taken one approach, Tesla another. It's pointless to argue that Waymo is inherently "better" since both are trying to converge on the same goal but from different directions. Waymo have to improve their geographical coverage, Tesla have to improve driver disengagements.

 

[KArnold]

On 11.4.4 (HW4)
The one consistent improvement I've had over the past few months is highway merges. They used to be terrible. Now except when highway traffic is stop and go and FSD wants to fly by everyone merges have been decent. Blinker gets used and most of the time FSD moves over before the end of the zip lane.

Is that consistent? I've not seen that but I'm also now normally manually signaling when getting on the interstate. I might be beating it to the signal.

 

[AlanSubie4Life]

I had similar experiences on a recent trip. The route was clearly indicated as straight ahead but the car signaled and moved quickly into random right turn lanes. Prior to 11.4.7.X this wasn't an issue for me.

Yeah, a little different than the described behavior, but it is routine for the car to dive into right-turn-only lanes on divided highways when going straight at 60-65mph. (E.g. 89A west from Sedona to Cottonwood, it likes to dive).

No idea if this is new behavior.

 

[Therealm80]

Just out of sheer curiosity, since it’s supposed to snow for the first time tomorrow, does anybody know if the antilock brakes actually work or those beta too?

 

[aronth5]

Is that consistent? I've not seen that but I'm also now normally manually signaling when getting on the interstate. I might be beating it to the signal.

Yes consistent except as I stated when the highway traffic is extremely slow. FSD then often times speeds past the line of stop and go traffic and runs out of the zip merge lane. But definitely a lot better then then FSD used to behave trying to merge when it rarely used the blinker. I've learned to wait to see what happens.

 

[jabloomf1230]

I do similar, except I exit out of FSDwtf and send in a voice message.…“YOU REALLY HAVE NO SPATIAL AWARENESS” or, “Dude! READ THE ROOM” or my all time favorite, “WTF”???

Then when it slows down using regen, I reengage the FSDbj

Yeah, that will work just as well as when I swear at Alexa.

 

[Tronguy]

Just out of sheer curiosity, since it’s supposed to snow for the first time tomorrow, does anybody know if the antilock brakes actually work or those beta too?

Having been driving a 2018 M3 and a 2021 MY in NJ, where snow, ice, and slush are a thing, they work. I’ve stuck with all-season tires successfully, but we don’t get the seriously deep stuff around here. Those further north swear by snow tires, but that’s kind of true for any vehicle, not just Teslas.

 

[EVNow]

Do you legitimately not understand the strategic differences between the approach Tesla is taking, and the approach Waymo is taking?

Its like comparing SAP with QuickBooks.

since there is no such thing as self driving, where can you find a self driving expert?

Anyone working on such an effort ?

I am an electrical engineer with nearly 40 years of experience in designing engineering and commissioning automation and controls.

I'm a software guy and have been for decades - though not as old as you are . So are a lot of people in this forum. We all understand software and software projects etc. We've all done it for a living. For decades.

I am saying my opinion is that Elon’s approach has failed. He has not delivered the product, sold me in over five years. do any of you actually believe that even by 2025, a 2018, Tesla will be operating in full self driving without operator 100% responsibility and supervision,

Waymo has worked for way longer and to go by what you are saying, it has failed, since it can't do squat outside its geofence. Everything is a "failure" until it works. Tesla was a failure before 2018 since it couldn't bring a mass market EV to the market. Tesla was a failure until 2013 because it couldn't bring designed from scratch car to the market. Whether you paid for it beforehand or after makes little difference ... to whether the project will ultimately succeed or not.

I'm 100% sure by 2025, my '18 model 3 will NOT be level 4/5.

 

[jabloomf1230]

Back to the condition of the present FSD firmware. I took some time since 11.4.7.3 was released to accumulate a few drives because I noticed one significant improvement and at first I didn't believe it. At least for the single lane roundabouts near my house, FSDb now does them effectively. In the past it would either inappropriately stop before entering rather than yielding. It would also use its turn signal. And lastly, if left to its own devices (heheh), it would often cut people off and speed through the roundabout when the correct choice would have been to stop and wait for a gap.

Now, it slows down before the roundabout and either yields or stops depending on the roundabout traffic. It doesn't signal, yields and it proceeds through the roundabout at no more than the speed of the traffic flow.

It surprisingly also mostly does the awful multilane roundabouts near my house correctly and it doesn't change lanes from the outer to inner loop, except to exit. These multilane roundabouts are a design debacle that confounds most human drivers, so I doubt that FSDb will ever do them perfectly everywhere all of the time, at least for the near future. But at least there's progress in improving the NN for roundabouts. Elon must be getting serious about releasing FSDb in Europe.

 

[DrGriz]

I had similar experiences on a recent trip. The route was clearly indicated as straight ahead but the car signaled and moved quickly into random right turn lanes. Prior to 11.4.7.X this wasn't an issue for me.

I've had the random car picks a side road, signals and turns off the mapped route as well. Not only here in Idaho, but in Washington as well. Sometimes left, sometimes right.

 

[spacecoin]

Waymo cars can drive in small geofenced areas of 4 major cities, about 400 square miles. Tesla's FSD can drive in any part of the USA and Canada, about 10,000,000 square miles. Sounds like 25,000x better to me. Or, if you want factor in your 1000x argument (valid or not), FSD is 25x better at driving overall, since Waymo cannot drive AT ALL outside its geofenced areas.

This is completely false. Waymo can drive better than Tesla’s joke of a software anywhere. It’s just that Waymo can’t drive everywhere unsupervised with the safety margins needed for L4/5. And there is no real money to be made by solving “shitty computer driver” tbh. Let’s name it “Full Time Driving School Teacher For At Stupid Student With No Pay”. (FTDSTFASSWNP).

Anyone can cherry pick statistics to suit their personal bias. The point is Waymo has taken one approach, Tesla another. It's pointless to argue that Waymo is inherently "better" since both are trying to converge on the same goal but from different directions. Waymo have to improve their geographical coverage, Tesla have to improve driver disengagements.

Tesla is claiming to be solving the exact same problem as Waymo , and claiming to be using more or less the same technical approach in software. The only thing that differs to a person that thinks Tesla is going robotaxi, is the roll out strategy. Of course we should compare them as long Tesla claims to have that same (or even higher) goal.

Getting a car to drive a route 90-95% of the time is no small feat but is laughingly trivial compared to getting it right 99.999999% of the time.

Please let me know when FSD reliably handles the most basic case of stopping for red lights and stop signs at 99.999999%.

 

[JulienW]

This is completely false. Waymo can drive better than Tesla’s joke of a software anywhere. It’s just that Waymo can’t drive everywhere unsupervised with the safety margins needed for L4/5......

Apple and oranges. If you were to put a Waymo on flatbed as is from San Francisco and move it to I80 in the middle of Nebraska it could not drive an inch (or 2.54 centimeter in science ). If you put a Tesla on a flatbed from San Francisco as is and move it to the same location it would drive right away near flawlessly since this is rural Interstate.

 

[spacecoin]

Apple and oranges. If you were to put a Waymo on flatbed as is from San Francisco and move it to I80 in the middle of Nebraska it could not drive an inch (or 2.54 centimeter in science ). If you put a Tesla on a flatbed from San Francisco as is and move it to the same location it would drive right away near flawlessly since this is rural Interstate.

Apple and oranges indeed, and also incorrect.

The Waymo Driver likely wouldn’t drive, but it could if they turned off the safety requirements that Tesla doesn’t have nor care about. I am certain the Waymo Driver will beat FSD in any environment over a few hundred trips, and that should be obvious to anyone.

Tesla FSD on the other hand can’t drive anywhere autonomously. Do you think hw3/hw4 ever will? If so in what type of roads/environment and at what speed? I will bet anyone any amount of money that FSD won’t be autonomous L3+ with Tesla taking liability in any meaningful ODD by the end of 2025.

Again, let me know when FSD handles stop signs and red lights consistently if ever.

 

[drtimhill]

The Waymo Driver wouldn’t drive, but it could if they turned off the safety requirements that Tesla doesn’t have nor care about. I am certain the Waymo Driver will beat FSD in any environment, and that should be obvious to anyone.

Tesla FSD on the other hand can’t drive anywhere autonomously. Do you think hw3/hw4 ever will? If so in what type of roads/environment and at what speed? I will bet anyone any amount of money that FSD won’t be autonomous L3+ with Tesla taking liability in any meaningful ODD by the end of 2025.

Again, let me know when FSD handles stop signs and red lights consistently if ever.

So you are comparing actual Tesla FSD to some imagined version of Waymo that they dont actually deploy? Yeah, apples to oranges indeed!

As I've noted, Waymo can drive very well on a small subset of roads, Tesla can drive moderately well on pretty much any road. You cant argue that Waymo is inherently "better" because both systems currently have severe limitations, and you cannot supply any factual argument that one set of limitations is less significant than the other. You can prefer one approach over the other but anything else is pure speculation/bias.

Let me know when Waymo can drive from one city to another.