FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[powertoold]

Seriously? It broke early, didn't pull fully into the left turn lane (admittedly a short left turn lane), and left the rear hanging out to dry. Even Chuck reports the butt pucker event. Check out the UI to see how close that initial vehicle was. By the graphics it looked like it made contact and that was a potential chain reaction as following vehicles were traveling fast and close. Not safe and not considerate which is the theme for the day apparently.

I agree it was a close call, but when I look at the speedo, the slowdown was gradual. It didn't slam on the brakes. Its only "mistake" is that it initially stopped too far behind the lead car, but there very well may have been another car there in another instance.

The biggest issue in this case is the tailgating car behind. But yes, I agree with you that FSDb didn't do this "perfectly." You say, "it broke early," but to me, that is the best defensive move in this situation.

The funniest theme in this thread is that some people here expect FSDb to be perfect from the getgo, judging from all the nitpicky criticisms.

 

[sleepydoc]

Seriously? It broke early, didn't pull fully into the left turn lane (admittedly a short left turn lane), and left the rear hanging out to dry. Even Chuck reports the butt pucker event. Check out the UI to see how close that initial vehicle was. By the graphics it looked like it made contact and that was a potential chain reaction as following vehicles were traveling fast and close. Not safe and not considerate which is the theme for the day apparently.

yes, seriously. I've seen other human drivers drive conservatively and brake early. If the driver behind would have hit him it would have 100% been his fault.

 

[powertoold]

Here's another case of an unsafe inconsiderate FSDj action. This is the new v11.4.3 rollout.

This looks like a new "bug" behavior with 4.3 (stopping too far behind the lead car on certain short left turn lanes):

 

[EVNow]

Has the random lane change behavior improved in 4.3 ?

Today FSD chose the changed the lane some 50 ft before the turn and changed back. I think because the map showed the road section in orange …. even though the road was empty ahead.

BTW, at roundabouts FSD visualization does not show the creep wall … so it’s difficult to figure out whether the car is just creeping for better visibility or will enter the roundabout. This results in half my roundabout disengagements.

 

[EVNow]

According to conversations on TMC, Teslas:

• Can't maintain speed correctly
• Can't make a right turn correctly, protected or unprotected
• Can't make a left turn correctly, protected or unprotected
• Turns into oncoming lanes
• Stops at green lights
• Won't go after a light turns green
• Stops way before a stop sign and creeps forward
• Stops in the middle of intersections
• Waits for others at a 4-way stop, even though it got there first
• Slams on the brakes for no reason
• Runs red lights
• Leaves way too much of a gap to a lead car
• Jerks the steering wheel back and forth randomly
• Turn on no-turn-on-red
• Doesn't slow down for school zones
• Doesn't stop for school busses
• Can't stay centered in the lane
• Drive too close to parked cars
• Drive over white lane lines
• Drive over yellow/double-yellow lane lines
• Drives in the shoulder
• Can't change lanes properly
• Doesn't accelerate smoothly
• Doesn't decelerate smoothly
• Runs over children and animals
• Panics around pedestrians, except when running over children and animals
• Changes lanes for no reason
• Changes lanes away from a turn
• Runs through turn lanes instead of turning
• Doesn't slow down for train tracks
• Doesn't slow down for speed bumps
• Doesn't avoid pot holes
• Smells like wet feet when AC is on
• Has terrible rattling noises when driving
• Goes through tires like they're going out of style
• Bursts into flames randomly
• Dry wipes the windshield frequently
• Cleans an already clean windshield
• Highbeams other drivers constantly
• Fails to turn on wipers when misting/raining

Did I miss anything?

Seriously, many of these are valid and should be converted to a new wiki thread and maintained …

 

[powertoold]

Seems like UPLs are improved in 4.3:

 

[powertoold]

I'm liking what I'm seeing in 4.3: better UPLs, better parking lots, smoother stopping and turns

It's still timid changing lanes between cars, but that's an inevitable fix.

Improvement is accelerating!1!1!!!

 

[JHCCAZ]

Seriously, many of these are valid and should be converted to a new wiki thread and maintained …

Please don't. It's better to have one main thread. Already the one that you started and made a Sticky-Wiki is working. Better than a whole bunch of separate threads, ostensibly to track separate issues, that don't really accomplish much to "organize" the discussion - and only fracture the continuity of issue tracking.

I'm not saying your idea isn't theoretically better, but in actuality it just perpetuates the problems mentioned above. FSD development encompasses so many issues, ongoing issues, that can't be separated out from a general discussion of what's happening with the latest release.

Finally, I suggest tgat next time (v12 thread?) It should be created as a Sticky but not a Wiki. I suppose the letter is meant to be helpful by having the topic-orientation posts at the beginning, but in reality those just become totally out of date, and it requires extra work to scroll down and find the page number selector that is buried below the wiki stuff.

Please note all these comments are meant constructively, not just grousing. I appreciate the intent of organizing information, but I don't think it actually facilitates the free-flowing discussion.

 

[Dave EV]

So, to imply anyone using FSD is *inconsiderate* is insulting.

I use FSD on a daily basis, so now you're purposely misreading what I'm saying on purpose just to argue.

My statement is that when FSDb does something stupid that can put you in a position where the car is acting erratically, delaying traffic, or otherwise acting like a drunk student driver, you should disengage and take over.

I dislike drivers who do this, the same as when FSDb does this. So when FSDb does this, I take over to keep it from doing so. I don't wait until it figures it out, or until others figure out how to go around my car. Driving is a privilege where mistakes have real consequences - not a playground for you to play with FSD.

And how do you think the NHTSA would react to that?

I'm not suggesting that it speed - I'm primarily suggesting that it do a better job of going SLOWER than the speed limit when it doesn't make sense, like in the example I quoted. There are tons of bad speed limit signs out there.

Why are are multiple people purposely misreading my statements and ignoring the context of what I'm saying?

 

[GhostMY]

Got 4.3 last night. Unfortunately I won't be able to test it properly until Monday. My 30 mile drive each way to and from work allows me to see what's better, what's the same, and what's worse with each update. Like others, 4.2 was a mixed bag but did fix some really annoying slow downs and lane changes that appeared to be geographically related. I was happy! And then last week, for reasons I cannot comprehend, those same slow downs and lane changes reappeared.

 

[GhostMY]

I use FSD on a daily basis, so now you're purposely misreading what I'm saying on purpose just to argue.

My statement is that when FSDb does something stupid that can put you in a position where the car is acting erratically, delaying traffic, or otherwise acting like a drunk student driver, you should disengage and take over.

I dislike drivers who do this, the same as when FSDb does this. So when FSDb does this, I take over to keep it from doing so. I don't wait until it figures it out, or until others figure out how to go around my car. Driving is a privilege where mistakes have real consequences - not a playground for you to play with FSD.


I'm not suggesting that it speed - I'm primarily suggesting that it do a better job of going SLOWER than the speed limit when it doesn't make sense, like in the example I quoted. There are tons of bad speed limit signs out there.

Why are are multiple people purposely misreading my statements and ignoring the context of what I'm saying?

Same reason texts are often misunderstood. We're better at verbal communication than written, most of the time. Patience and objectivity helps!

 

[EVNow]

My statement is that when FSDb does something stupid that can put you in a position where the car is acting erratically, delaying traffic, or otherwise acting like a drunk student driver, you should disengage and take over.

The only difference we have is in this statement about "delaying traffic". I don't think an extra second at the stop sign or taking a right turn to be delaying traffic. Do you ?

Other than that - duh, if we don't dosengage when FSD is erratic we are not even helping test the software since there is no other way to send bug reports.

 

[EVNow]

Please don't. It's better to have one main thread. Already the one that you started and made a Sticky-Wiki is working. Better than a whole bunch of separate threads, ostensibly to track separate issues, that don't really accomplish much to "organize" the discussion - and only fracture the continuity of issue tracking.

Nah ... Not what I meant. Just to keep track of know bugs and which release solves it and if it regresses.

Kind of like the release history wiki thread.

Good point on this thread, we should remove the wiki part of it and make it a plain sticky thread - @Krash ?

 

[EVNow]

https://twitter.com/x/status/1667355544967950336

 

[rlsd]

Got 4.3 last night. Unfortunately I won't be able to test it properly until Monday. My 30 mile drive each way to and from work allows me to see what's better, what's the same, and what's worse with each update. Like others, 4.2 was a mixed bag but did fix some really annoying slow downs and lane changes that appeared to be geographically related. I was happy! And then last week, for reasons I cannot comprehend, those same slow downs and lane changes reappeared.

I wish I have 11.4.3 now. I prefer testing new releases with the routes I go to work daily on early weekend mornings. Traffic during weekdays is too heavy for me to test. If 11.4.3 can reliably makes left/right turns on designated short turn lanes and does not swerve the car to the shoulder on ramps to freeway then I will be happy.

 

[drtimhill]

The funniest theme in this thread is that some people here expect FSDb to be perfect from the getgo, judging from all the nitpicky criticisms.

Exactly.

 

[nvx1977]

Seriously? It broke early,

ok I used to think people casually misused "break" for "brake" because they are pronounced exactly the same. But now it appears some people actually think "break/broke" is the correct word.

It's "brake/braked".

 

[sleepydoc]

ok I used to think people casually misused "break" for "brake" because they are pronounced exactly the same. But now it appears some people actually think "break/broke" is the correct word.

It's "brake/braked".

Yeah, I just left that one!

 

[sleepydoc]

Have you not experienced cut ins and outs on the freeway yet? Vastly superior to 3.6

I don’t do much freeway driving so I haven’t noticed that but the other regressions are more than enough to cancel that benefit out, IMO

I was on a highway driving home from work this morning and FSD actually crossed over a dotted white line swerving around a bus in the adjacent lane. I thought it was changing lanes or avoiding an obstacle at first but it promptly went back in its lane afterwards (and then signaled to merge in front of the bus.) So from my experience it isn’t better.

 

[jebinc]

According to conversations on TMC, Teslas:

• Can't maintain speed correctly
• Can't make a right turn correctly, protected or unprotected
• Can't make a left turn correctly, protected or unprotected
• Turns into oncoming lanes
• Stops at green lights
• Won't go after a light turns green
• Stops way before a stop sign and creeps forward
• Stops in the middle of intersections
• Waits for others at a 4-way stop, even though it got there first
• Slams on the brakes for no reason
• Runs red lights
• Leaves way too much of a gap to a lead car
• Jerks the steering wheel back and forth randomly
• Turn on no-turn-on-red
• Doesn't slow down for school zones
• Doesn't stop for school busses
• Can't stay centered in the lane
• Drive too close to parked cars
• Drive over white lane lines
• Drive over yellow/double-yellow lane lines
• Drives in the shoulder
• Can't change lanes properly
• Doesn't accelerate smoothly
• Doesn't decelerate smoothly
• Runs over children and animals
• Panics around pedestrians, except when running over children and animals
• Changes lanes for no reason
• Changes lanes away from a turn
• Runs through turn lanes instead of turning
• Doesn't slow down for train tracks
• Doesn't slow down for speed bumps
• Doesn't avoid pot holes
• Smells like wet feet when AC is on
• Has terrible rattling noises when driving
• Goes through tires like they're going out of style
• Bursts into flames randomly
• Dry wipes the windshield frequently
• Cleans an already clean windshield
• Highbeams other drivers constantly
• Fails to turn on wipers when misting/raining

Did I miss anything?

If a refreshed S/X, it vibrates like a MoFo!