FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[EVNow]

I am at .086 here in rural NC. Fuel is around $3.20 per gallon here. I do not miss living on the west coast.

0.10 on the west coast here ... and completely emission free. Who cares what the gas price is

 

[EVNow]

A thousand pardons sir. It won't happen again.

You should be sorry - you are literally quoting TSLAQ.

 

[Knightshade]

And you claimed to care about the traffic in your previous comment but clearly you were just running off at the mouth again.

It'd be great if you could substantively address the actual discussion instead of the usual personal attacks.

I pointed out the system tends to do better- and I tend to have far less need to intervene when there's little or no traffic, and even gave specific examples/reasons why that's the case. The only thing that's running off is a description of real world experiences.

 

[archae86]

A post on my experience driving 2023.7.5/FSDb 11.4.1:

Summary: for my personal observations--good news. Some clear improvements. No clear new shortcomings.

I drove our model 3 yesterday on a trip which took me through three separate lane-threading spots for which previous versions of FSDb have consistently had poor behavior. While it could be luck of the draw, I felt that in all three cases it did enough better than before that it was outside the previous range of behavior. Mostly this meant better lane choice, less inclination to attempt awkward lane changes late while surprising other drivers, and doing better at getting in the correct lane for the next exit enough earlier to reduce challenges from that.

Also, I drove it in light rain in the dark, on a road where previously the steering was noticeably erratic in that condition. Even though the "FSDb might be degraded" alarm chimed early on and stayed on for the whole drive, the actual steering experience seemed normal. I think it was probably an improvement--though no two rain experiences can ever be quite the same for perfect comparison. This improvement may have come a release or two ago.

Just to cut through the goods new with a bad observation (but not new): just after doing a good set of lane changes to get in the correct turn lane after a freeway dismount, the car did an annoying creep and twitch while looking for a traffic hole to turn in to while stopped at a major intersection. The movements were distracting to me, and very likely visible to other drivers. When it finally chose a hole and went ahead, the acceleration was not quite enough for best courtesy. I'll mellow that bad observation on this drive with the overall comment that across my usual drive locations both the twitchiness while creeping and the failure to accelerate enough to fit the merge problems are much better now on average for me than they were a few revisions ago. They both still need improvement.

 

[kabin]

You should be sorry - you are literally quoting TSLAQ.

Pot meet V11 kettle.

I have no idea what TSLAQ is and don't care to fall into that rabbit hole. Back on topic.

 

[EVNow]

I have no idea what TSLAQ is and don't care to fall into that rabbit hole.

TSLAQ - Wikipedia

en.wikipedia.org

According to the Los Angeles Times in 2019, TSLAQ members believe Tesla is a fraudulent company and its stock would eventually crash, while also specifically claiming that Tesla was experiencing a "demand cliff" for its products and has had to regularly distort its sales numbers.[6] Their self-reported main goal as of 2019 was to "change the mind of Tesla stock bulls and the media."[6] Tesla was the most shorted stock in the U.S. in December 2020, with over $34.5 billion in shorted share value at its peak.[7] Business Insider described TSLAQ member activity in 2019 as consisting of "exchang[ing] research, news articles, and sometimes outlandish conspiracy theories about the company" and that members were "betting on the company's death and have found much success in irritating the billionaire executive."[8]​

 

[EVNow]

You bet! I've only done Alki in West Seattle, so keen to do more. Post them to this thread in the order you want me to test them and I'll start on them this upcoming weekend. I'll follow up to make sure I'm clear on the routes. If you have specific times you want them tested, add that too please. Weekdays are tough, as I have a job - but weekends are fairly open before 7pm.

@WilliamG Did anything happen with this ? @jebinc

Or did #TeamJunk get cold feet

 

[aronth5]

For those more familiar with the technical specs on the fender and front facing cameras a HW3 question for you.
What degree does the car have to turn at a stop sign/intersection for a left or right turn so that the fender camera can fully see the lane approaching you and the front camera can fully see the opposite lane approaching you?

One of the biggest changes I've seen in creep mode over time is the degree to which FSD now aligns the car towards the direction it's going to turn. This behavior has given me enough confidence, although not always, that the car is actually seeing the crossing road traffic well enough to avoid a disengagement. Especially in obstructed view turns.

 

[uscbucsfan]

For those more familiar with the technical specs on the fender and front facing cameras a HW3 question for you.
What degree does the car have to turn at a stop sign/intersection for a left or right turn so that the fender camera can fully see the lane approaching you and the front camera can fully see the opposite lane approaching you?

One of the biggest changes I've seen in creep mode over time is the degree to which FSD now aligns the car towards the direction it's going to turn. This behavior has given me enough confidence, although not always, that the car is actually seeing the crossing road traffic well enough to avoid a disengagement. Especially in obstructed view turns.

It's not the fender camera that looks left/right, it's the pillar camera behind your door.

 

[kabin]

We'll have to get @SeattleFSD some views and subscriptions. I like the your dialog box issue indicators. Hopefully you don't have a BSOD.

It's nice for viewers to see the pedals. Some youtubers try to keep the pedal area clear which admittedly adds risk.

One of the youtubers was talking about adding a honk counter. Probably a good idea as it's a peer driver indicator of acute FSDj failure. At the very end, that last lane change was too close for my comfort level given the state of FSDj.

Looks like the FSDj path planner was wigging out prior to the first intersection as well as turning at the intersection otherwise it hung in there for those FSD drivers willing to put it all on the line.

 

[EVNow]

Yesterday I drove through one of the streets I normally disengage - very narrow with a center line - but cars parked on either side. FSD did quite well.

So, the current top issues for me are
- Bad lane changes / planning
- Roundabouts
- Slow to make decisions (esp. flashing yellow turn signal) <-- this might be a lot more difficult to solve

 

[jebinc]

@WilliamG Did anything happen with this ? @jebinc

Or did #TeamJunk get cold feet

I was never part of this. Opted out, if you recall. This was a @WilliamG thing.

 

[sleepydoc]

Roundabouts

You keep complaining about roundabouts. I think you just need to move to a different part of the country with fewer of them!

 

[EVNow]

You keep complaining about roundabouts. I think you just need to move to a different part of the country with fewer of them!

That's my plan - just need to get my family to agree.

I'm getting the feeling that would be faster than Tesla fixing roundabouts.

 

[WilliamG]

@WilliamG Did anything happen with this ? @jebinc

Or did #TeamJunk get cold feet

I didn’t follow up quite yet. Been a little busy/out of town. Also Chuck caught on and this specific issue is now bigger/getting more press than it was (though it’s hilarious that Chuck believes it’s only a 11.4.1 issue when it’s been a problem on 11.3.6, too).

We’ll see if Tesla can address it in the next update.

 

[GSP]

We'll have to get @SeattleFSD some views and subscriptions. I like the your dialog box issue indicators. Hopefully you don't have a BSOD.

It's nice for viewers to see the pedals. Some youtubers try to keep the pedal area clear which admittedly adds risk.

One of the youtubers was talking about adding a honk counter. Probably a good idea as it's a peer driver indicator of acute FSDj failure. At the very end, that last lane change was too close for my comfort level given the state of FSDj.

Looks like the FSDj path planner was wigging out prior to the first intersection as well as turning at the intersection otherwise it hung in there for those FSD drivers willing to put it all on the line.

I really like the format of his videos, with excellent non-verbal communication, very strategic fast forwarding and slow-mo. The background music reminds me of Tesla’s “Paint-it-black” video. This is one of the best FSD channels out there, and definately deserves more subscribers.

GSP

 

[EVNow]

I didn’t follow up quite yet. Been a little busy/out of town. Also Chuck caught on and this specific issue is now bigger/getting more press than it was (though it’s hilarious that Chuck believes it’s only a 11.4.1 issue when it’s been a problem on 11.3.6, too).

We’ll see if Tesla can address it in the next update.

Still would be very interesting to see the difference if any between S/X and 3/Y. Looking forward to any tests you guys do.

 

[uscbucsfan]

Still would be very interesting to see the difference if any between S/X and 3/Y. Looking forward to any tests you guys do.

May be out of context and certainly anecdotal, but my 22 MS performs much better on the same local roads than my 21 MY.

 

[EVNow]

May be out of context and certainly anecdotal, but my 22 MS performs much better on the same local roads than my 21 MY.

I wonder whether S/X need more clearance to move i.e. they will stop when a car comes from the other side more so than 3/Y which are smaller.

In your experience - are there specific ways / scenarios where S is better?

 

[WilliamG]

Still would be very interesting to see the difference if any between S/X and 3/Y. Looking forward to any tests you guys do.

We’ll probably get to it before too long if things don’t resolve with the next update.