FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[powertoold]

How come your rear wheel hit the curb but your front wheel get scratched?

The pictures don't tell the whole story, the pictures inside the car and visualization seem accurate, but the pic outside the car with the scratch doesn't match

 

[mtndrew1]

The pictures don't tell the whole story, the pictures inside the car and visualization seem accurate, but the pic outside the car with the scratch doesn't match

The car showed that the rear wheel was closest to the curb and that there was plenty of space near the front wheel. Realty was exactly the opposite; there was a foot or more of space between the rear wheel and the curb and the front wheel hit it.

Tesla Vision FTW.

 

[WilliamG]

The pictures don't tell the whole story, the pictures inside the car and visualization seem accurate, but the pic outside the car with the scratch doesn't match

The pictures do tell the whole story. Park assist got it simply wrong.

 

[swedge]

And yet another Tesla fail. I’ll keep my USS, thank you very much.

On our MY, the USS does not see curbs up close either, I assume because the sensors are well above typical curbs, and are looking over them.

Someone suggested Tesla was using USS to train the vision version. Perhaps not a great idea...

 

[powertoold]

The car showed that the rear wheel was closest to the curb and that there was plenty of space near the front wheel. Realty was exactly the opposite; there was a foot or more of space between the rear wheel and the curb and the front wheel hit it.

Tesla Vision FTW.

Well, based on the pictures, I'm not convinced. You may be right. But we need more information, especially the front camera view or view of the whole curb.

About 95% of all Teslas I see have some curb rash, so it's clear to me that most Tesla drivers will be bad with curbs.

This is a case where extraordinary claims require more evidence.

The picture below seems to show the front of the car is angled away from the curb, but the scratch picture shows it well-angled into the curb.

 

[Todd Burch]

The car showed that the rear wheel was closest to the curb and that there was plenty of space near the front wheel. Realty was exactly the opposite; there was a foot or more of space between the rear wheel and the curb and the front wheel hit it.

Tesla Vision FTW.

So you profess to have no faith in anything the Autopilot team does, yet you immediately go out and test it by pulling right up to the curb without verifying its accuracy first? And this despite the warning in the release notes?

 

[mtndrew1]

Well, based on the pictures, I'm not convinced. You may be right. But we need more information, especially the front camera view or view of the whole curb.

I just did a couple hundred dollars of damage to my car because of Tesla’s lousy software. Believe what you like.

 

[mtndrew1]

So you profess to have no faith in anything the Autopilot team does, yet you immediately go out and test it by pulling right up to the curb without verifying its accuracy first? And this despite the warning in the release notes?

Correct. I had no intention of purposely testing this feature but I went to park and this very confident graphic showed up and I gave it a go. I know better than to trust Tesla‘s software and I paid for my mistake, financially. Lesson learned.

 

[powertoold]

The pictures do tell the whole story. Park assist got it simply wrong.

Show us the Google map view of that specific curb

 

[mtndrew1]

On our MY, the USS does not see curbs up close either, I assume because the sensors are well above typical curbs, and are looking over them.

Someone suggested Tesla was using USS to train the vision version. Perhaps not a great idea...

I never trusted curbs on my USS 3 for that reason and now I know not to trust Vision parking either.

 

[mtndrew1]

Show us the Google map view of that specific curb

Weirdly combative but ok. The first pics were me first seeing the graphic and I took them (stopped) having been excited at something that was finally impressive for USS-less cars. You can see in the photo that the car indicated that the rear was quite close and there was plenty of space in the front. As I parked, the reality was the opposite and the front wheel curbed even as it continued to show plenty of space remaining.

34.06434° N, 118.21683° W

 

[mtndrew1]

11.4.9 on the holiday update has some notable improvements on my routes. The car slams on the brakes far less often for traffic in adjacent lanes when using FSDb on the freeway. It feels ”skittish” in the same areas where it would normally hit the brakes but it’s an improvement.

I’ve only got about a hundred miles on this build so far but city driving is less bad. Of course it still makes a lot of the same dumb mistakes but it’s an improvement. I’ll try my test route soon.

 

[powertoold]

34.06434° N, 118.21683° W

Is this the curb? I searched it on Google maps as typed

 

[mtndrew1]

Is this the curb? I searched it on Google maps as typed

Is this the curb? I searched it on Google maps as typed

Yep, I parked where the forward-most car on the bottom of the image is.

 

[TK211X]

I was kind of excited seeing the new curb graphic and thought “huh maybe they’ll actually pull this off” and then…View attachment 1000435View attachment 1000436
View attachment 1000437View attachment 1000438

I see uuuuuu

 

[mtndrew1]

View attachment 1000503
I see uuuuuu

Aren’t I a vision?

 

[TK211X]

It seems like a concurrent release.

.1 pending count is topping off by 200 whenever it falls too much

.2 appears to have something similar happening.

 

[jebinc]

That is absolutely awful. It’s clear what the park assist is saying exists is disconnected from reality. Pretty much Tesla these days.

More junk… served up cold, and undercooked.

 

[jebinc]

The pictures do tell the whole story. Park assist got it simply wrong.

Never let the facts ruin a good story….

 

[Scott Chapman]

.3 is out