FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Therealm80]

I suppose it’s relative. I think it’s just a fun and cool thing to have in my opinion and well worth the money as I enjoy feeling like I’m in the future even though it’s far from perfect. I spent twice the cost of FSD on a Mac Pro a few years back and ended up not feeling like I got my money’s worth from it even though others with a similar configuration of Mac were much happier than I was using it.

They make Mac Pro’s for 30k?? Wow

 

[Dewg]

Some great discussions recently, and I enjoy seeing the different experiences with the recent versions. One thing I'll add to the discussion: AP/NoA/FSD Beta are L2 driver assist features. They work in concert with you, the driver.

Beta moving too fast down a street? Use the wheel and scroll down the speed. Beta hesitant at a 4-way stop? Use the accelerator to get moving. Traffic jamming up a lane ahead? Use your turn signal and direct the car to change lanes away from the jam.

0 disengagement drives are always a goal, but don't be afraid of intervention. Be safe out there, intervene when necessary, disengage when necessary, and make sure you voice report your disengagements.

 

[fasteddie7]

They make Mac Pro’s for 30k?? Wow

They make Mac pros for $50k but I really didn't need that one. Plus wife was fully prepared to strangle me over 30 so 50 could have ended in me not being able to use it due to her ending me lol

 

[MARKM3]

We keep getting these incremental updates. I wonder if all these updates will eventually result in a non-beta reliable FSD package, or if these updates are just small improvements on a fundamentally doomed approach.

I agree. The current build is fundamentally doomed.

The current build improves with smoothness and overall user satisfaction with the highway city stack merge.

Un protected lefts, right hand turns, stop sign intersection interactions, route planning, lane selection, map data, speed data all need quicker feedback loops and more work. When I disengage/bug report that a highway has the wrong speed limit, my car should update that for all Tesla's until I give Tesla a reason not to trust my feedback. It should be verified by recognizing speed limit signs. When there are no 25mph speed limit signs, you only pass 55mph speed limit signs, and the car goes back to 25mph, it just makes you go Why??. And that same highway has been wrong for 5+ years with no change. Going 25mph on a 55mph highway is dangerous.

Honestly based on my experience, we are looking at minimum 5 more years until these current issues are ironed out to a comfort level that of an aware safe driver. I can't use it for my commute comfortably. Standard cruise control is far superior on my daily drive to work.

 

[D Good]

They make Mac pros for $50k but I really didn't need that one. Plus wife was fully prepared to strangle me over 30 so 50 could have ended in me not being able to use it due to her ending me lol

They don’t anymore. They switched to Apple silicon. Not they max out around 15k.

 

[fasteddie7]

Honestly based on my experience, we are looking at minimum 5 more years until these current issues are ironed out to a comfort level that of an aware safe driver. I can't use it for my commute comfortably. Standard cruise control is far superior on my daily drive to work.

Does anyone think that regardless of approach, truly functional autonomous driving is far off regardless of hardware approach? Is everyone years off in their own way? Such as LiDAR and radar equipped vehicles that work in nearly every situation but are limited to a small square mile area would take years to build out the areas where it can work vs vision only that can theoretically work everywhere, when it works, but has to spend the time improving situational understanding which will also take years (and maybe additional hardware revisions)

 

[uscbucsfan]

Does anyone think that regardless of approach, truly functional autonomous driving is far off regardless of hardware approach? Is everyone years off in their own way? Such as LiDAR and radar equipped vehicles that work in nearly every situation but are limited to a small square mile area would take years to build out the areas where it can work vs vision only that can theoretically work everywhere, when it works, but has to spend the time improving situational understanding which will also take years (and maybe additional hardware revisions)

Absolutely. Waymo doesn't even believe true level 5 is ever possible.

 

[fasteddie7]

They don’t anymore. They switched to Apple silicon. Not they max out around 15k.

right?! My $30k computer is worth the smallest fraction of that now. This time I went Mac Studio as $8k is way easier to swallow and make upgrading every 3 years easier to stomach.

 

[scottf200]

I spent twice the cost of FSD on a Mac Pro a few years back

$30K on a Mac Pro?!?! /s -- his point was $15K ... I'm fairly happy when I got our for $6K in 2017. This was back before 'free' AP I think and I have loved loved loved having AP for road trips. I use FSDb now but while I like the novelty of it, it can be uncomfortable and erratic when your passengers are in the car and see it act goofy.

 

[boonedocks]

Does anyone think that regardless of approach, truly functional autonomous driving is far off regardless of hardware approach? Is everyone years off in their own way? Such as LiDAR and radar equipped vehicles that work in nearly every situation but are limited to a small square mile area would take years to build out the areas where it can work vs vision only that can theoretically work everywhere, when it works, but has to spend the time improving situational understanding which will also take years (and maybe additional hardware revisions)

so much so that I didn't bat an eye at NOT buying FSD on my 2021 Refresh S. Burned 3X already buying FSD and I don't see any chance of it happening in any way possible for the next 5+ years.

 

[LowlyOilBurner]

so much so that I didn't bat an eye at NOT buying FSD on my 2021 Refresh S. Burned 3X already buying FSD and I don't see any chance of it happening in any way possible for the next 5+ years.

Yeah, I’ll rent it when I replace my car next year. Bought it twice already.

 

[fasteddie7]

so much so that I didn't bat an eye at NOT buying FSD on my 2021 Refresh S. Burned 3X already buying FSD and I don't see any chance of it happening in any way possible for the next 5+ years.

This just falls into the “buy it for what it does today, not for what it may do in the future” category like so many other tech products that promise software updates that bring features that are shown at the keynote. There are some that see the value in what it can do now that it’s money we’ll spent, any future updates are looked forward to with excitement, and others that the promise of tomorrow doesn’t cut it and what it does today isn’t worth it for them.

 

[aronth5]

An aware driver is much safer than FSD in it's current state. Doesn't matter if it has 15 cameras, from my experience something is either wrong in the design of the B pillar cameras or the software is fatally flawed. Majority of the time my unprotected left or right hand turns onto a 55mph highway are done dangerously close to oncoming traffic or fast approaching traffic. There is a much higher risk for me being rear ended or t boned using FSD. I either always have to slam the accelerator or hit the brake.

I am extremely skeptical of them being able to fix this. I think the cameras just can't see far enough. UPL's and Righthand turns should be flawless.

We will have a much better idea when the capability is added to actually view the B-pillar cameras. (Tesla Software Update - All Access Cameras! (2023.20.4.1)
First thing I'm going to do is drive up the intersection I take every day to leave my neighborhood creep as far as FSD does and then look at the B-pillar camera. I fully expect the field of view will be less then when I lean as far forward as possible which is the only safe way to take this intersection. If the field of view is less then mine than clearly FSD will never be safe at these types of intersections on HW3. And in fact if unsupervised by a human this intersection will certainly have bad accidents.

 

[willow_hiller]

We will have a much better idea when the capability is added to actually view the B-pillar cameras. (Tesla Software Update - All Access Cameras! (2023.20.4.1)
First thing I'm going to do is drive up the intersection I take every day to leave my neighborhood creep as far as FSD does and then look at the B-pillar camera. I fully expect the field of view will be less then when I lean as far forward as possible which is the only safe way to take this intersection. If the field of view is less then mine than clearly FSD will never be safe at these types of intersections on HW3. And in fact if unsupervised by a human this intersection will certainly have bad accidents.

Don't forget to check the the wide angle forward facing camera. I don't think anyone other than Green has been able to pull footage from that view until this version.

 

[fasteddie7]

Don't forget to check the the wide angle forward facing camera. I don't think anyone other than Green has been able to pull footage from that view until this version.

Wonder if this is something that HW4 cameras will fix/get better. I tried looking up what the two active camera types are I can't remember what type they were and a quick search didn't yield any results, and if there isn't a wide one maybe that's what the 3rs empty spot will be for.

 

[scottf200]

We will have a much better idea when the capability is added to actually view the B-pillar cameras. (Tesla Software Update - All Access Cameras! (2023.20.4.1)
First thing I'm going to do is drive up the intersection I take every day to leave my neighborhood creep as far as FSD does and then look at the B-pillar camera. I fully expect the field of view will be less then when I lean as far forward as possible which is the only safe way to take this intersection. If the field of view is less then mine than clearly FSD will never be safe at these types of intersections on HW3. And in fact if unsupervised by a human this intersection will certainly have bad accidents.

Don't forget to check the the wide angle forward facing camera. I don't think anyone other than Green has been able to pull footage from that view until this version.

Re: Tesla Software Update - All Access Cameras! (2023.20.4.1)

 

[thewishmaster]

Re: Tesla Software Update - All Access Cameras! (2023.20.4.1)

Pretty cool to see the amount of overlap between the fender camera and the pillar camera. Didn't know the pillar cams can see so far back

 

[jebinc]

The turns my car struggle with are not blind, you can clearly see traffic on either side. The car still makes unsafe turn decisions. Why does it do this if the car is accurately modeling a 3d vector space around it? It baffles me every time.

The poor lane decisions, bad speed limits, and other various quirks are all understandable disengagements and failures. But safety related such as UPL's and right hand turns really lower the credibility of FSD ever becoming a reality. The car does now what it should have done 5 years ago, but still has fatal flaws where everyone initially expected.

And people pay $15,000 for this. It's one of the biggest scams you could purchase.

Agreed.

 

[boonedocks]

Yeah, I’ll rent it when I replace my car next year. Bought it twice already.

Currently renting as well l.

 

[Supcom]

This has always been said and while technically true, ours move, have much higher quality resolution, depth perception, and a brain that varies per user.

That's always been the qualm with me and the pillar camera position. It was a poor placement as my head can move forward and see things that the car obviously cannot without creeping dangerously.

Your single sensor can see in only one direction at a time. And, when you focus on one item in that direction, your brain that varies will ignore everything else in the FOV. Despite having lower resolution, the car can estimate distance to many objects simultaneously to greater accuracy than you can for a single object.