FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[sleepydoc]

11.3.6 consistently failed this intersection for me - I need to turn right and it starts the turn, then gets confused mid-turn and turns left (note the path tracing in the 3rd picture.) I think it gets confused by the lane markings. There’s no turn lane, even though there’s room. FSD starts to enter the area where the turn lane would be, then sees the lines and veers to the left again. My guess is it’s thinking it’s in the wrong land and so can’t turn right? Either way it’s a straightforward intersection for a human.

 

[Yelobird]

11.3.6 consistently failed this intersection for me - I need to turn right and it starts the turn, then gets confused mid-turn and turns left (note the path tracing in the 3rd picture.) I think it gets confused by the lane markings. There’s no turn lane, even though there’s room. FSD starts to enter the area where the turn lane would be, then sees the lines and veers to the left again. My guess is it’s thinking it’s in the wrong land and so can’t turn right? Either way it’s a straightforward intersection for a human.

View attachment 941928View attachment 941932View attachment 941933

Well, it does say Detour lol. Possible it’s a mapping issue?

 

[sleepydoc]

PSA: For those experiencing unusual amounts of hesitation, visualization glitches, or red-hands system errors: it could be your USB drive.

Two days ago, I had a system crash that threw up red-hands and disabled all TACC/FSDb for the remainder of my drive. Even a two-thumb reset didn't fix it.

When I got home, I pulled out my Sentry Mode SSD drive and ran a Check Disk utility on it: irreparable drive errors. I removed my music partition and formatted the drive, and let my Model 3 sleep over night.

The next day, not only was FSDb back, but it was driving smoother than ever. Much less hesitation and latency, much smoother driving, and several zero-disengagement drives.

I don't know if the FSD computer mounts the Sentry Mode drive directly (maybe it uses the space to cache clips), but it's pretty clear to me now that drive errors do seem to negatively impact FSDb performance.

Thanks for the tip. I wonder how many others are experiencing this without knowing it. Seems like something Tesla should be aware of and maybe have the system check and alert the driver. Iv’e seen plenty of posts here about bad USB drives so it’s certainly not a rare event.

 

[Twiglett]

11.3.6 consistently failed this intersection for me - I need to turn right and it starts the turn, then gets confused mid-turn and turns left (note the path tracing in the 3rd picture.) I think it gets confused by the lane markings. There’s no turn lane, even though there’s room. FSD starts to enter the area where the turn lane would be, then sees the lines and veers to the left again. My guess is it’s thinking it’s in the wrong land and so can’t turn right? Either way it’s a straightforward intersection for a human.

View attachment 941928View attachment 941932View attachment 941933

I've had it do that as well. Driving on a dead straight road, 300ft from making a left at a 4way stop, it suddenly decides to turn right into a neighborhood.

 

[FSDtester#1]

Thanks for the tip. I wonder how many others are experiencing this without knowing it. Seems like something Tesla should be aware of and maybe have the system check and alert the driver. Iv’e seen plenty of posts here about bad USB drives so it’s certainly not a rare event.

Agree with both of you. I had to pull out my USB to send a recording to the police for a hit and run. I was blown away at how many clips are in there that we don't see in the car's UI. I thought I had plenty of space and no issues, wrong.
I just copied the entire USB to my laptop and then formatted the USB.
World of difference. Maybe that why I drive fsdH, or "hopeful" they are getting closer.

Edit: Also after a reformat it adds track mode recording to the available folders

 

[kabin]

I went through a detoured construction zone and 11.4.2 wouldn't proceed forward even through signage was clear and cones were tightly spaced. FSD kept trying to turn off the roadway. It seemed to be relying heavily on map data and couldn't reconcile.

I had a new behavior. In very light traffic at a right turn intersection FSD had an abrupt left, right, left steering input after the turn. Usually that happens when stopped at an intersection.

Turns signals remain a PITA.

Not sure how if more cowbell requests will ever be taken to heart but more regen use needed when slowing for traffic lights, stop signs, ...! Gotta wonder if this is a conspiracy plan for early brake servicing.

 

[sleepydoc]

Gotta wonder if this is a conspiracy plan for early brake servicing.

I share the view that FSDb should do a better job of using regeneration rather than friction braking but when I compared energy usage on my drive to work I found that FSDb really didn’t use appreciably or noticeably more energy than I would if I drove and avoided friction braking completely. My conclusion was that the use of friction braking likely appears to be worse than it actually is.

 

[Ramphex]

I share the view that FSDb should do a better job of using regeneration rather than friction braking but when I compared energy usage on my drive to work I found that FSDb really didn’t use appreciably or noticeably more energy than I would if I drove and avoided friction braking completely. My conclusion was that the use of friction braking likely appears to be worse than it actually is.

The Junk uses friction brakes more than I do for sure in my MS, I purposely keep my foot close to the pedal to see if it moves

 

[Reddy Kilowatt]

Downloading 2023.7.10.

 

[rlsd]

11.3.6 consistently failed this intersection for me - I need to turn right and it starts the turn, then gets confused mid-turn and turns left (note the path tracing in the 3rd picture.) I think it gets confused by the lane markings. There’s no turn lane, even though there’s room. FSD starts to enter the area where the turn lane would be, then sees the lines and veers to the left again. My guess is it’s thinking it’s in the wrong land and so can’t turn right? Either way it’s a straightforward intersection for a human.

View attachment 941928View attachment 941932View attachment 941933

Nice neighborhood. It has the T sign for Tesla.

 

[LowlyOilBurner]

11.4.2
55mph road, cyclist spotted, car slams on the brakes and paces the cyclist, does not move around to pass them like it did back on, oh I don’t know…10.3??? No oncoming traffic either. Very poor behavior, this was happening to me 11.3.6 as well, sometimes.

Unmarked road intersections, hogs the lane way too much. If I’m making a left, it’s hogging the left side way too much, as if another car was turning right, they wouldn’t be able to. I mentioned this yesterday, todays drive confirmed this, it wasn’t a fluke, it’s a “new feature”.

 

[sleepydoc]

11.4.2
55mph road, cyclist spotted, car slams on the brakes and paces the cyclist, does not move around to pass them like it did back on, oh I don’t know…10.3??? No oncoming traffic either. Very poor behavior, this was happening to me 11.3.6 as well, sometimes.

Unmarked road intersections, hogs the lane way too much. If I’m making a left, it’s hogging the left side way too much, as if another car was turning right, they wouldn’t be able to. I mentioned this yesterday, todays drive confirmed this, it wasn’t a fluke, it’s a “new feature”.

I came across a couple cyclists a week or two ago. They were off to the side but there was only about a foot of paved shoulder so they were still in the driving lane. FSDb (11.3.6) slowed down appropriately and straddled the center line to give them an appropriate amount of space, exactly as I would have done. Hopefully this isn’t a regression.

 

[scottf200]

The Junk uses friction brakes more than I do for sure in my MS, I purposely keep my foot close to the pedal to see if it moves

Can you measure it on the same drives to know for real vs anecdotal? ala trips data

 

[Ramphex]

Can you measure it on the same drives to know for real vs anecdotal? ala trips data

The trip data to me is anecdotal. The amount of power the regen provides on a regular road coming to a red light is nothing compared to the 1000s of watts it takes me to get places.

What matters is the friction brake use. I would personally prefer for the Junk to use full regen prior to reaching for my brake pedal, even if it only applying slight pressure to the friction brakes. As a lot, maybe most on here, I do a lot of one pedal driving. If the car cannot come to a complete stop with full regen, then sure the Junk should use friction brakes, but only then and not before it exhausts full regen in regular driving situations.

 

[FSDtester#1]

11.4.2
55mph road, cyclist spotted, car slams on the brakes and paces the cyclist, does not move around to pass them like it did back on, oh I don’t know…10.3??? No oncoming traffic either. Very poor behavior, this was happening to me 11.3.6 as well, sometimes.

Unmarked road intersections, hogs the lane way too much. If I’m making a left, it’s hogging the left side way too much, as if another car was turning right, they wouldn’t be able to. I mentioned this yesterday, todays drive confirmed this, it wasn’t a fluke, it’s a “new feature”.

I just completed a 22 mile round trip residential/city loop on 11.4.2.
I was specifically checking 2 areas where fsd would slow from 55 to 35 mph every time I drove past those areas. @Ramphex has a similar area where his MS does the same thing and even slows to 0. To my surprise both areas are pretty much fixed.
1 was non existent, 2nd it went down 2 mph, but not noticeable enough to make me or drivers behind me uncomfortable. It also completed a U turn on a 4 lane divided highway, where in the past would force me to take over. I then tested residential streets with cul-de-sacs which also in the past would be a forced takeover because the car couldn't "complete the maneuver". It successfully completed that multiple times and kept the speeds down due to "residential area detected ", which overrides any speed offset you have programmed. All this testing took place after taking the recommendations of other members to clear out the Teslacam USB. Which others have stated that the full or corrupt files in the USB may interfere with FSDb and it's potential processing power.

 

[Ramphex]

I just completed a 22 mile round trip residential/city loop on 11.4.2.
I was specifically checking 2 areas where fsd would slow from 55 to 35 mph every time I drove past those areas. @Ramphex has a similar area where his MS does the same thing and even slows to 0. To my surprise both areas are pretty much fixed.
1 was non existent, 2nd it went down 2 mph, but not noticeable enough to make me or drivers behind me uncomfortable. It also completed a U turn on a 4 lane divided highway, where in the past would force me to take over. I then tested residential streets with cul-de-sacs which also in the past would be a forced takeover because the car couldn't "complete the maneuver". It successfully completed that multiple times and kept the speeds down due to "residential area detected ", which overrides and speed offset you have programmed. All this testing took place after taking the recommendations of other members to clear out the Teslacam USB. Which others have stated that the full or corrupt files in the USB may interfere with FSDb and it's potential processing power.

My one well known spot is fixed. Not sure if it was an FSDj issue or a mapping issue. I’ll also never know since the beta team never responded to my email when I submitted the details for that specific issue.

I like how the Junk now prompts what it’s doing on the dash screen.

 

[FSDtester#1]

My one well known spot is fixed. Not sure if it was an FSDj issue or a mapping issue. I’ll also never know since the beta team never responded to my email when I submitted the details for that specific issue.

I like how the Junk now prompts what it’s doing on the dash screen.

Agreed, the more it tells us what the hell it's thinking, the less you freak out when it starts that maneuver.
Also, it is super confident in merging on 2 and 3 lane roads to find gaps to change lanes. A couple of them I was gripping the yoke and had feet ready to intervene.
They were all safe, but way more aggressive then prior versions.

 

[Rikster]

I just completed my first drive with 11.4.2, coming from 11.3.6 yesterday. Virtually no change for my drive, unfortunately. Still mostly just fine, but the problem areas on my normal commute remain exactly as they were in 11.3.6 (I've identified these errors via the microphone probably 50-60 times over the past couple of months). If anything, I think the drive might have been slightly less smooth than 11.3.6, with a bit harsher and later stopping. Still several instances of the car trying to move into a turning lane when the route was planned to keep it going straight ahead.

So not really any improvement that I've noticed so far, unfortunately. Still fairly good, and I'm still happy to use it on pretty much every drive, but I was hoping for some minor improvements in the rough edges that I'm just not seeing yet. Maybe v12?

 

[fasteddie7]

I almost didn’t publish the video since 11.4.1 and 11.4.2 are almost identical for me. Had issues in all the same places, with the exception of a few solid left turns and some really aggressive braking. If anyone is interest in a HW4 comparison between versions:

 

[FSDtester#1]

I almost didn’t publish the video since 11.4.1 and 11.4.2 are almost identical for me. Had issues in all the same places, with the exception of a few solid left turns and some really aggressive braking. If anyone is interest in a HW4 comparison between versions:

Good video, well done. I think you are one of the few with hw4 so your feedback is much appreciated. I just subscribed to your channel. Hopefully others will also.