FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[swedge]

If my dog is riding with me I have to fasten the seat belt so the alarm doesn't keep going off. I don't think the police would consider my dog a qualifying passenger for the HOV lane.

We used a "child seat" setting to keep it from beeping about our dog. At least I think that was what it did, it was a while ago.

 

[Knightshade]

Some day, an FSD beta update will show up without a bunch of notes explaining the minutiae of implied improvements.

It will show up with one sentence, “Tesla assumes all liabilities when this version of FSD is engaged”.

That will be the announcement that the AI development challenges have been solved.

 

[clydeiii]

Perhaps. With better hardware. Pretty sure HW3 is tapped out.

These sorts of posts are ridiculous.

 

[Supcom]

We used a "child seat" setting to keep it from beeping about our dog. At least I think that was what it did, it was a while ago.

Yes. Set each of the three rear seats to a child seat and the car will stop the alerts.

 

[Knightshade]

These sorts of posts are ridiculous.

I mean not as ridiculous as the fact Green told us HW3 was out running out of compute in mid-2020 and there's somehow people out there in 2023 using the current beta and still believing it's ever gonna do L4 or L5....

 

[clydeiii]

I mean not as ridiculous as the fact Green told us HW3 was out running out of compute in mid-2020 and there's somehow people out there in 2023 using the current beta and still believing it's ever gonna do L4 or L5....

"running out of compute" also a ridiculous claim... as if optimization isn't a possibility at all...

 

[clydeiii]

 

[mtndrew1]

I'd strongly suggest you re-read my post.

Because you appear to...not have? Because it points out why "only knowing how many people are in the car" doesn't actually let the car know if it's ok to use a given HOV lane or not.

And since that's the only info the cabin camera can provide, the car can not know if it's ok to use a lane from that.

On the 14 freeway north of Los Angeles the time of day and day of the week are also factors.

The HOV on/off selector needs to be on the top level menu.

 

[clydeiii]

 

[sleepydoc]

Yes. Set each of the three rear seats to a child seat and the car will stop the alerts.

My dog sits in the front seat - there’s no car seat setting for the front seat. (At least there wasn’t, I haven’t checked in a while)

 

[clydeiii]

"running out of compute" also a ridiculous claim... as if optimization isn't a possibility at all...

If you're following along at home, Google just released PaLM 2 and it's "significantly smaller" in size than the original PaLM LLM, while also being significantly more capable. Hopefully, Tesla has years of improvements to their models that enable this sort of shrinkage too.

 

[jebinc]

Not great. Turning into the middle lane is appalling. How is this basic stuff not solved?!

Still passing “loading/unloading” school busses as well, I assume….

 

[FSDtester#1]

Still passing “loading/unloading” school busses as well, I assume….

Nope, it's now programmed to aim for small children when they exit the bus...

 

[jebinc]

Nope, it's now programmed to aim for small children when they exit the bus...

I know it still blows through school speed limit zones, so why not? Go for those little buggers!

 

[powertoold]

I mean not as ridiculous as the fact Green told us HW3 was out running out of compute in mid-2020 and there's somehow people out there in 2023 using the current beta and still believing it's ever gonna do L4 or L5....

There's still some juice left from HW3 judging by the road geometry visualizations in 11.4.1. Overall, the visualizations look much more representative of reality vs 3.6.

The limitation seems to be the accuracy and density of the autolabeler.

 

[FSDtester#1]

I know it still blows through school speed limit zones, so why not? Go for those little buggers!

The smaller the better, it gives beta more time to process if they can't run fast.

 

[jebinc]

Even “three buck Chuck” says it’s unsafe!

 

[WilliamG]

Even “three buck Chuck” says it’s unsafe!

And we don't even need Chuck to tell us this!

 

[Knightshade]

There's still some juice left from HW3 judging by the road geometry visualizations in 11.4.1. Overall, the visualizations look much more representative of reality vs 3.6.

The limitation seems to be the accuracy and density of the autolabeler.

Visualizations are done by the MCU last I knew- not the AP/FSD hardware...This previously came up when they released the "FSD preview" visualizations for only HW3 cars, even though HW2.5 cars could've done them just as easily since the MCU was the same...

https://twitter.com/x/status/1209131690762915841

Has that changed?

"running out of compute" also a ridiculous claim... as if optimization isn't a possibility at all...

What brand of hopium do you prefer, specifically?

It still can't reliably do Chucks ULTs, after sending test cars and engineers to Chucks ACTUAL TEST STREET for weeks. Months ago.

It doesn't even have a >L2 OEDR. At all.

Not an "unoptimized" one- it's an entire branch of capability the system lacks (which is the reason Tesla specifically gave to CA DMV about why city streets can't ever be more than L2).

But you think they're going to add massively more capability it doesn't even have yet and then shrink down the compute needed for all of it by well over 50% to get it back into a single node (because otherwise your Robotaxi fails every time either node reboots-- which we already see it do in just the recent videos posted a couple pages ago)?


FWIW I do think HW3 could probably get good enough to do L3 highway... which is all I ever wanted when I bought FSD anyway... but I've got 0% belief it's sufficient for a system that'll never require a human behind the wheel.

 

[WilliamG]

Visualizations are done by the MCU last I knew- not the AP/FSD hardware...This previously came up when they released the "FSD preview" visualizations for only HW3 cars, even though HW2.5 cars could've done them just as easily since the MCU was the same...

https://twitter.com/x/status/1209131690762915841

Has that changed?

I'm not sure I buy that from Green. If so, why is the frame-rate so crap, even on the Ryzen Model S/X with high-end GPU? It's been bad for 2 years now.