FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[EVNow]

Had my first drive on 11.4.2 …. No major changes from 11.3.x.

- The turns and roundabout navigation seems slower and more ”measured”
- Still had lane selection issues resulting in a disengagement
- Still drives too slowly in construction zone forcing another disengagement

 

[totoro722]

I almost didn’t publish the video since 11.4.1 and 11.4.2 are almost identical for me. Had issues in all the same places, with the exception of a few solid left turns and some really aggressive braking. If anyone is interest in a HW4 comparison between versions:

Nicely made video. FSD looks fairly competent except for the STOP sign that is attached to the utility pole and way higher than normal placement. Hard to see the fork in the road, but that appears to be somewhat difficult to detect until you are pretty close to it. I hope we see more videos from you, as this one is quite good.

 

[Ramphex]

Weird stuff happened on my trip to CVS for hemorrhoid cream today.

Visualizer got tired of drawing and went to sleep for the last 1/3rd of the trip

Then while inside I got a notification on my phone that the sentry mode shutdown due to a system error.

 

[fasteddie7]

Weird stuff happened on my trip to CVS for hemorrhoid cream today.

Visualizer got tired of drawing and went to sleep for the last 1/3rd of the trip

View attachment 942080

Then while inside I got a notification on my phone that the sentry mode shutdown due to a system error.

Happened to me once. Had to park at home where sentry wasn’t active and shut down to reset the computer, started working next drive.

 

[Ramphex]

Weird stuff happened on my trip to CVS for hemorrhoid cream today.

Happened to me once. Had to park at home where sentry wasn’t active and shut down to reset the computer, started working next drive.

First time either issues happen, never experienced on 2022.X

 

[fasteddie7]

First time either issues happen, never experienced on 2022.X

I started with factory firmware 2023.6.1 so I don't know about 2022.x software but it happened on my way to pickup dinner one night, I noticed after I tried to engage autopilot and it wouldn't engage, then I noticed it.

 

[frankvb]

Just updated to 11.4.2. I hoped it would fix the speed limit issues that started with 11.3.6, but it doesn't. Close to where I live, it often has 20 as speed limit for 25 mph roads. Changing it manually to 25 lasts only until the next turn where it drops back to 20 again

On Mira Mesa Blvd. it doesn't drop down from 45 to 40 until 1.5 block too late, and also, when turning onto that road from a 25 mph speed limit side road, it takes a long time before it changes the limit to 40 mph.

I'm not really sure if this is due to the FSDb updates, or to incorrect map data, or a combination of both. Map is at 2022.44 and I everything worked much better before 11.3.6 with the same map data (as far as I can recall map data hasn't updated for ages).

BTW: left turn (traffic light) onto MM Blvd. it swerved to the right out of the (dedicated) left turn lane, crossing over the line maybe 1 or 2 ft, no other vehicles around. That never happened with earlier versions.

 

[Dutchie]

Just had a longer drive with 11.4.2. I found it handelde the four-way STOP intersections much better.. In the old version the car was sometimes Giving way to cars that were clearly arriving later at the intersection than me. So far the new version handled these situation perfect.

I had one BIG intervention as two pedestrians wanted to cross the street using a crosswalk. The car did not see these people at all. I could barely brake and i had to apologize for me coming in so fast…

 

[fasteddie7]

BTW: left turn (traffic light) onto MM Blvd. it swerved to the right out of the (dedicated) left turn lane, crossing over the line maybe 1 or 2 ft, no other vehicles around. That never happened with earlier versions.

I see this on a lot of left turns, at a light, stop sign, or even just a regular left from one road to another, it does this weird jerk the wheel to the right one or two times cutting ever so slightly to the right before committing to the left.

 

[powertoold]

Just did a second 16 mile rt with 4.2 through highway, downtown with peds, city strs, etc.:

1) With a lead car, the slowing down is smoother and more responsive

2) Ride is overall more jolty vs 3.6, 4.2 overreacts to any sort of cross traffic (people pulling out of parking lot, etc).

3) Response around peds is improved, drove past homeless dude with dogs on side of road, also dealt with jaywalker intending to pass after my car (by just driving past him rather than freaking out)

4) Lane selection is improved, lane changing is improved (more responsive and was able to "skip" 3 lanes to get to a left turn lane)

5) Overall 4.2 is better than 3.6, but it needs some more polish to reduce the jolty driving

 

[EVNow]

Weird stuff happened on my trip to CVS for hemorrhoid cream today.

Visualizer got tired of drawing and went to sleep for the last 1/3rd of the trip

View attachment 942080

Then while inside I got a notification on my phone that the sentry mode shutdown due to a system error.

Sentry mode has clashed with FSD related stuff in the past. Could be related.

I’ve briefly lost visualization once or twice, IIRC.

BTW, reformat your USB and check. Some are claiming it improves FSD ….

 

[powertoold]

Seems like 4.2 solved a lot of downtown Seattle issues. Lane selection seems to be very complicated in dtsea:

 

[aronth5]

After a few drives here are my initial impressions. Prior new builds seem to improve over time so I'll wait a couple of weeks to see how 4.2 is going.

The Good

• Overall a smoother ride.
• Handled a secondary road curve without crossing over the yellow line. First time ever. In the past FSD would not slow down until it was halfway into the turn.
• Really nice turn into my street which needs to be taken below 8mph. Today FSD handled this turn twice beautifully.
• Not as jerky especially approaching intersections.
• Did some really nice turns.
• Tesla appears to have fixed the problem when you try to re-engage.

The Bad

• Tried to turn right on a No Turn on Red sign. This was fixed in previous v11 builds
• Tried to exit the highway 1 exit early.
• Lane selection getting off the highway was just as bad as v11.3.6
• Still stops at the main roundabout I use daily but once in the roundabout FSD was smoother.
• Disengaged twice when FSD tried taking a left hand turn with vehicles too close to turn safely. Worst I've ever seen.
• Approached a Stop Sign and instead of creeping just started to turn. Had to disengage. This happens to me from time to time at several intersections. Most intersections FSD creeps appropriately but it still takes far too long.
• Failed to identify a narrow residential street as a reduced speed zone (25mph). This really surprised me since other streets that aren't nearly as dangerous were down graded to 25mph.

 

[D Good]

What is the current status of the rollout? Is it the 20% rollout right now? No update here.

 

[jebinc]

25 mile round trip to the movie theater on FSDj
Partly sunny
No highway
MS Plaid

Summary: When it was not screwing up, it was much smoother than 11.3.6. It did run a stop sign at a T intersection (at speed), went crazy when one lane turned
Into two at a T intersection stop light, and slows down drastically when transitioning from one road to another via a sweeping right turn.

I might try one more FSDj drive with this version before mothballing it for the next couple of releases.

@WilliamG

 

[scottf200]

What is the current status of the rollout? Is it the 20% rollout right now? No update here.

~33% on TeslaFI

 

[EVNow]

Seems like 4.2 solved a lot of downtown Seattle issues. Lane selection seems to be very complicated in dtsea:

I think monorail is a special case. Otherwise not very different from other hilly downtowns.

 

[sleepydoc]

I think monorail is a special case. Otherwise not very different from other hilly downtowns.

I haven’t seen the video but how is a monorail different from a train?

 

[jebinc]

I haven’t seen the video but how is a monorail different from a train?

One rail instead of two!

 

[MP3Mike]

I haven’t seen the video but how is a monorail different from a train?

It is above ground, so there are pillars supporting the monorail track that you sometimes have to change lanes between.