FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[jeremymc7]

So is this a general rollout to everyone with FSD beta, or a limited “influencer” release?

Influencer for now. Hence five people on TeslaFi with no pending.

 

[Dan D.]

Centre Pointe Dr, Milpitas, CA to Huntwood Ave, Union City

Doesn't stop for the stop sign at 0:03
Pulls off to the right and messes up the straight-ahead, is forced to turn right at 0:46
Drives over all sorts of bike lanes & the right turn only lane at 0:58
Comes to a full stop on the street at 1:14
Fails to take correct lane at 1:38 - Disengagement
Goes through red light at 4:22
Fails to clear windshield for miles
Tries to go into Toll Express lane, driver has to cancel 8:47, 9:41,

 

[ChaoticNeutral]

Is it known if V11 eliminates the need for ultrasonic sensors for NOA to work on HWYs on vehicles with a non-functioning USSs (as well as auto-park, and summon)?

 

[jeremymc7]

Is it known if V11 eliminates the need for ultrasonic sensors for NOA to work on HWYs on vehicles with a non-functioning USSs (as well as auto-park, and summon)?

See nothing in release notes. Something like that would likely be called out specifically.

Honestly I personally don’t expect the whole USS and related missing features thing to be resolved for months; perhaps many many months.

 

[aronth5]

The problem with roundabouts is calculating the behavior probabilities for each human driver within, entering and leaving the roundabout. No traffic? No problem. Hesitation and stumbling around by FSDb both have no impact in minimal traffic conditions. Add a lot of cars to a multilane roundabout and FSDb has no idea what to do. I'm convinced that roundabouts need a Chuck Cook ULT-like effort to be made by Tesla.

I have 1 large roundabout I take every day and FSDb slows to 1 mph before proceeding even when there are no cars. And it doesn't matter where I enter the roundabout. One of these days it will get fixed .

 

[JulienW]

Chuck gone WILD and a 3ed V11 video of the day (yesterday). Still in ORL (work) so no JAX UPLs yet. Also another near rear ending at 8:25. That is several I have seen so far and something we have not had in Beta. May be a V11 bug that needs to be fixed.

Also at 25:00 it comes to a 4 Way Stop and BLOWES it by sitting there. The new Stop Sign behavior of waiting forever is a bust at a 4 Way Stop sign. All it will do is piss everyone off and have them going in front of you.

 

[JulienW]

AI DRIVR Some parking lot fun. Actually not too bad for not having parking lots yet.

 

[dramsey]

Chuck gone WILD and a 3ed V11 video of the day (yesterday). Still in ORL (work) so no JAX UPLs yet. Also another near rear ending at 8:25. That is several I have seen so far and something we have not had in Beta. May be a V11 bug that needs to be fixed.

Also at 25:00 it comes to a 4 Way Stop and BLOWES it by sitting there. The new Stop Sign behavior of waiting forever is a bust at a 4 Way Stop sign. All it will do is piss everyone off and have them going in front of you.

It's not new (at least for me): I've seen similar behavior with 10.69.<whatever> for months. Still, it's a minor annoyance, easily handled by pressing the accelerator yourself at an appropriate time.

 

[PianoAl]

I haven't seen any videos showing the new slowing behavior when encountering lower speed limits.

I'm surprised the problem with minimal and late blinker signals has not been resolved.

 

[Gilliland]

I haven't seen any videos showing the new slowing behavior when encountering lower speed limits.

I've watched 6-8 videos in the past 24 hours, so I don't remember which one showed it, but I did see one such situation posted. The car went from a 55 zone to a 35 zone as it entered a town, and it failed to slow down. The driver disengaged and hit the brakes, then reported it. In short, it behaved exactly like past versions. No improvement, no change.

 

[powertoold]

I haven't seen any videos showing the new slowing behavior when encountering lower speed limits.

I'm surprised the problem with minimal and late blinker signals has not been resolved.

 

[AlanSubie4Life]

Tesla Bull video below)

Stopping to 0mph for stop signs not relevant to the road

What do you think it stopped for in the video?

Answer: a truck. Very wrong of course.

 

[powertoold]

Answer: a truck. Very wrong of course.

Perhaps, the exact same thing happened in one of whole mars videos, but without a truck and a stop sign on the far right side

 

[AlanSubie4Life]

Perhaps, the exact same thing happened in one of whole mars videos, but without a truck and a stop sign on the far right side

Stop sign looked too far back in this case.

 

[powertoold]

Stop sign looked too far back in this case.

 

[AlanSubie4Life]

Yep. That one is totally different. Stopped at the stop sign.

Certainly not the exact same in Bull video, where stop sign was way before the stop. Truck it is.

 

[kabin]

Chuck gone WILD and a 3ed V11 video of the day (yesterday). Still in ORL (work) so no JAX UPLs yet. Also another near rear ending at 8:25. That is several I have seen so far and something we have not had in Beta. May be a V11 bug that needs to be fixed.

Also at 25:00 it comes to a 4 Way Stop and BLOWES it by sitting there. The new Stop Sign behavior of waiting forever is a bust at a 4 Way Stop sign. All it will do is piss everyone off and have them going in front of you.

Agreed. The near rear ends are far too frequent with V11 which shouldn't be the case for observant safe driving. It's unacceptable for the lead vehicle differential velocity to not approach zero under braking conditions. Some of it seems to be related to poor planning like a last second risky cut-in. Inadequate deceleration might raise red flags about system latency.

It's good to see braking and steering appear more refined but there's a compromise between smoothness, system response, and safety. We'll see if HW3 has the where-with-all to get it done.

Stop signs and some intersection right turns are poor. And the new phantom braking might now be a full stop in the middle of traffic. Oy!

 

[FearL0rd]

Does it bring summon back?

 

[willow_hiller]

Really looking forward to v11. Remains to be seen if FSDb can appropriately respond to a small highway obstacle like a tire, but the occupancy network can recognize it. I feel like NOA would happily ignore a tire and plow straight into it:

https://twitter.com/x/status/1633847942233698304

 

[AlanSubie4Life]

Remains to be seen if FSDb can appropriately respond to a small highway obstacle like a tire

Seems likely - seems to respond to nearly everything (except perhaps red lights).