FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Dennisis]

Here is the snapshot of a drive on a very windy country road today using V11.4.2.
@28 and @48 you can tell that I had to disengage and reposition the car to the right to avoid cars coming from the opposite direction. Speed was reduced to 20mph but at these turns no faster than 15mph is required. You can see other sections where the car is way too far to the left.
So the bottom line is FSD CANNOT drive this road. And it's a fairly popular road since it goes to Great Brook Farm.
Emailed Tesla.

North Road, Billerica, Ma
Google Maps

You're supposed to actually DRIVE on those windy country roads, not let the car do it! I love driving those roads in the Catskills when visiting every couple of years (raised there), winding through the hills. FSD is for the monotonous stuff IMO, can't take all the fun away...

 

[PACEMD]

When push comes to shove, I still ain't got no updates, and my maps were made by Eratosthenes........

 

[rlsd]

Here is the snapshot of a drive on a very windy country road today using V11.4.2.
@28 and @48 you can tell that I had to disengage and reposition the car to the right to avoid cars coming from the opposite direction. Speed was reduced to 20mph but at these turns no faster than 15mph is required. You can see other sections where the car is way too far to the left.
So the bottom line is FSD CANNOT drive this road. And it's a fairly popular road since it goes to Great Brook Farm.
Emailed Tesla.

North Road, Billerica, Ma
Google Maps

Too much expectation from FSD. Besides the road is narrow, probably the dense trees and shades gave FSD a hard time to figure out the pitfalls.

 

[rlsd]

One bad move by FSD this afternoon. FSD beautifully entered the freeway and exited to the road using the right most lane. This was the correct lane selection because I needed to go to a store on the right about 1/4 mile from turn. But when the car moved near to the end of the turning lane, it crossed the right lane marker. I disengaged when the car was about 2 feet out of the lane.

 

[powertoold]

But when the car moved near to the end of the turning lane, it crossed the right lane marker. I disengaged when the car was about 2 feet out of the lane.

You're saying it crossed over the right-most lane marked edge and 2 feet into the shoulder? I don't see how this is a problem.

 

[aronth5]

Too much expectation from FSD. Besides the road is narrow, probably the dense trees and shades gave FSD a hard time to figure out the pitfalls.

Too much expectation. No.
Is it a priority now? Probably not but it is a safety issue. If FSD cannot drive this then remove the road from navigation.
You just have to stay tight to the right on sharp curves and slow down.

Here is a similar problem with a blind curve that FSD does need to solve now.
@1:00

 

[rlsd]

You're saying it crossed over the right-most lane marked edge and 2 feet into the shoulder? I don't see how this is a problem.

Yes. The car should be inside the lane. There was no wide shoulder. If it continued to go it would hit the dirt and street barrier.

 

[powertoold]

Too much expectation. No.
Is it a priority now? Probably not but it is a safety issue. If FSD cannot drive this then remove the road from navigation.
You just have to stay tight to the right on sharp curves and slow down.

I don't see how it's a requirement to be biased on the right side of that road, as it doesn't completely fit 2 cars.

It's similar to driving on this road, where you just move over when you see a car:

 

[aronth5]

I don't see how it's a requirement to be bias on the right of that road, as it doesn't completely fit 2 cars.

It's similar to driving on this road, where you just move over when you see a car:

The problem is you cannot see the car until it's too late since the corners are so tight and they are hilly corners. On this road you have to disengage at every blind corner since you're too far left and cannot chance a car coming around the corner. That is why I haven't even tried FSD for almost a year knowing it's dangerous but figured I'd try with V11. On a road like this just stay far to the right on curves and you're fine.

I've shared several clips with Tesla and hopefully they will add this type of clip to their data sets. For sure it's an edge case but eventually FSD needs to address it or just remove the road from navigation which is certainly ok with me.

 

[FSDtester#1]

Some poster above said the penetration was 33% or thereabouts.

Sounds like like a night of rejection at @PACEMD 's house !

 

[powertoold]

problem is you cannot see the car until it's too late since the corners are so tight and they are hilly corners. On this road you have to disengage at every blind corner since you're too far left and cannot chance a car coming around the corner. That is why I haven't even tried FSD for almost a year knowing it's dangerous but figured I'd try with V11. On a road like this just stay far to the right on curves and you're fine.

Not sure if this is a preference or requirement, I guess an alternative is to go slow around corners?

How much extra visibility are others getting if you're in the middle of that narrow lane vs 4 feet to the right?

 

[swedge]

Here is the snapshot of a drive on a very windy country road today using V11.4.2.
@28 and @48 you can tell that I had to disengage and reposition the car to the right to avoid cars coming from the opposite direction. Speed was reduced to 20mph but at these turns no faster than 15mph is required. You can see other sections where the car is way too far to the left.
So the bottom line is FSD CANNOT drive this road. And it's a fairly popular road since it goes to Great Brook Farm.
Emailed Tesla.

North Road, Billerica, Ma
Google Maps

Nice video! Very generous of you to let FSD drive that road, we seldom get such lovely drives. And brave of you too, what with all those trees standing ready in case your car should glitch and dent itself.

I'm still waiting (with [im]patience) for 11.4.2. A quick question for you 11.4.2'ers: does it sill have the "can't engage" bug? My 11.3.6 after disengaging flickers the grey steering wheel icon, and re-engaging is a try and try and ty again exercise. Very distracting. Have they fixed this???? That has been a big annoyance.

Thanks!

 

[jebinc]

I just got my update so they’re letting some miscreants through.

Also a handful of malcontents!

 

[powertoold]

Cone placement on the bird's eye view is still hit and miss.

I think car placement on the bird's eye view is so good because Tesla can use radar to train car distances on the forward facing cameras and then use that data to better estimate vehicle distances on the other cameras.

But cones and other "soft" objects (like trash bins) don't have a strong radar signal, so the estimation isn't good.

I see robotaxis as impossible unless Tesla can figure out how to accurately place cones on the bird's eye view.

Btw, I'm not sure if there was a calibration period or maybe I'm driving in the daylight, but 4.2 seems smoother at stopping today, almost perfect.

 

[loquitur]

The problem is you cannot see the car until it's too late since the corners are so tight and they are hilly corners. On this road you have to disengage at every blind corner since you're too far left and cannot chance a car coming around the corner. That is why I haven't even tried FSD for almost a year knowing it's dangerous but figured I'd try with V11. On a road like this just stay far to the right on curves and you're fine.

I've shared several clips with Tesla and hopefully they will add this type of clip to their data sets. For sure it's an edge case but eventually FSD needs to address it or just remove the road from navigation which is certainly ok with me.

Thanx for the tree-lined road and the Laurel Canyon vids. I can imagine that something like a Kalman filter has to be applied to get a smoothish path thru all those obstacles (trees, garbage cans, parked cars on corners) and that going down the middle is a good basic strategy while the lane-edge detection networks are being fine-tuned.

Without occupancy networks in place there'd be little hope at all. Further, although Tesla showed in simulation that road shadows and temp lighting artifacts are well-handled, you simply can't
argue with a tree or on-coming car!

Although the two scenarios are not in the Waymo/Cruise drive-hailing domain, I wonder whether
they "bias center" or "bias right" in these narrow lane situations.

 

[sleepydoc]

I don’t know if this is a common problem but 11.3.6 seems to be unable to navigate a left-hand turn where two lanes are turning at the same time in both directions. If I am in the far left lane, it swings wide and cuts off the car in the near left lane. If I am in the near left lane, it swings wide and threatens to ram the car turning left in the opposite direction.

I’ve experienced this, too but note that It wasn’t until the latest version that FSD was able to reliably take the right lane of a double left turn and calculating a path for 2 left turns (yours and the car to your left) while also figuring out where the oncoming car is going to be driving is not an easy task, even for a human.

 

[fisherman]

Got 11.4.2 Friday ahead of a long weekend drive. Used it mainly for busy interstate driving, but anyone else experiencing it playing the loud alarm beep as if you're going to collide with something randomly? Had it happen a few times across a few hundred miles of straightforward driving in good conditions.

Also had a few phantom braking episodes on the interstate. I have absolutely no idea how this is not their first priority.

 

[Dewg]

Northbound I5 between Ortega Highway and Culver. I hope that is fixed by now.

There is major construction going on in that area (widening the I5), with significant lane shifting as well, and in some areas the lane markers are gone, so it can get a little dicey.

 

[Dewg]

Too much expectation. No.
Is it a priority now? Probably not but it is a safety issue. If FSD cannot drive this then remove the road from navigation.
You just have to stay tight to the right on sharp curves and slow down.

Here is a similar problem with a blind curve that FSD does need to solve now.
@1:00

That slow down seemed like it was a T-Junction, perhaps it thought it was an intersection that it had a stop sign for?

 

[powertoold]

It doesn't look like FSDb will be any good at parking lots until V12 (if ever) lol

Also, what do you guys think, did 4.2 turn into the correct lane after this turn? I think so: