FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Mark II]

You may have disagreed, but as I said, it didn’t know what it was doing, and using it in significantly degraded conditions is dangerous:

“Mitigated hydroplaning risk by making maximum allowable speed in Autopilot proportional to the severity of the detected road conditions. In extreme cases, Autopilot may use the wetness of the road, tire spray from other vehicles, rain intensity, tire wear estimation or other risk factors that indicate the vehicle is near the handling limit of the surface to warn the driver and reduce speed.”

We’ll see if this speed adjustment is concomitant with increased following distance and other adjustments needed for inclement weather. It seems unlikely that this safety improvement is complete (thus it likely remains unsafe except in highly curated situations).

Remember that Tesla provides lots of latitude, and assumes the driver will only use the system when it is safe, and will take care to avoid any instances where FSD will have a collision. This is a tricky balance, but Tesla will typically err on providing more latitude, even if it means an accident caused by the driver.

The point I was trying to make, and why I disagreed, is that I did not have an issue before with the old freeway stack, unless it rained really hard. In a light drizzle it worked just fine in the past. Now with combined stack it complains with a slightest bit of rain. This is a regression.

 

[aronth5]

The point I was trying to make, and why I disagreed, is that I did not have an issue before with the old freeway stack, unless it rained really hard. In a light drizzle it worked just fine in the past. Now with combined stack it complains with a slightest bit of rain. This is a regression.

My experience is the opposite. Just another example of how people's experiences with FSD are so different.

 

[Knightshade]

The point I was trying to make, and why I disagreed, is that I did not have an issue before with the old freeway stack, unless it rained really hard. In a light drizzle it worked just fine in the past. Now with combined stack it complains with a slightest bit of rain. This is a regression.

My experience is the opposite. Just another example of how people's experiences with FSD are so different.

Yeah mine's the opposite too.... and part of why I never believed Tesla would get L4 with this HW because nobody would accept an ODD of "not raining"


Previously NoA shut off in even moderate rain, let alone heavy. Dropping you back to basic AP.... while city streets still operated in moderate rain (it'd sometimes give a degraded message, but still actually navigate, make turns, etc).

Haven't driven single stack highway in moderate rain to see what's up with it yet, but I'd expect more city streets type behavior where it retains more function than just giving up like NoA did.

 

[mhan00]

Personally I won't use those gaps unless I can see all of the lanes of travel I will be crossing, as I have seen too many collisions from people doing that while someone comes barreling down the outside/turn lane.

Yeah, that is called the wave of death. Lots of accidents caused by people stopping in one lane and waving someone waiting to turn onto or off of the street, directly into a car in an adjacent lane that was driving at speed. If you’re going to take those gaps, always always always do so cautiously. Just because one lane is waving you through doesn’t mean the rest of the lanes know what’s going on. Flip side, if you’re driving next to a lane of stopped cars, always assume they’re stopped for a reason and slow Down at least a bit and be ready to stop, or you run into situations like JHCCAZ had where the car was barreling forward too fast.

 

[AlanSubie4Life]

The point I was trying to make, and why I disagreed, is that I did not have an issue before with the old freeway stack, unless it rained really hard. In a light drizzle it worked just fine in the past. Now with combined stack it complains with a slightest bit of rain. This is a regression.

I see. Let us know how it goes with wiper setting 3 to clear your vacillating detection state (making poor conditions always detected).

 

[AlanSubie4Life]

run into situations like JHCCAZ had where the car was barreling forward too fast.

Car needs to reliably understand context and what it cannot see (and respond appropriately). And of course have a very long memory and extended prediction capabilities over that timeframe. Limited evidence of this being reliable and extended to anything other than specific situations, so far.

 

[Therealm80]

Just saw that 11.4 is dropping hope its better!

 

[aronth5]

Yeah mine's the opposite too.... and part of why I never believed Tesla would get L4 with this HW because nobody would accept an ODD of "not raining"

Previously NoA shut off in even moderate rain, let alone heavy. Dropping you back to basic AP.... while city streets still operated in moderate rain (it'd sometimes give a degraded message, but still actually navigate, make turns, etc).

Haven't driven single stack highway in moderate rain to see what's up with it yet, but I'd expect more city streets type behavior where it retains more function than just giving up like NoA did.

I've driven in moderate and heavy rain several times with V11 and FSD has yet to drop back to basic AP. Kinda surprised me.

 

[Supcom]

An interesting artifact that I noticed several times on today's highway drive was the presence of a phantom car that would show up in the visualization behind and to the left of the car in a wide grassy median. The phantom car was always oriented in my direction and appeared in the visualization with about half of it outside of the display. It tended to move left/right a bit, but not forward/aft relative to my car. It was not in a position that would be expected to influence my car and there was no indication that it did. It was unrelated to traffic going the other way, which was quite a ways to the left and would happen with no traffic anywhere near me going in my direction.

This happened travelling both directions on US287 and always on the left side. I've never seen it on this, or any other version before. Anyone else have a phantom car pace them on the highway?

 

[GranularHail]

There's a spot on my daily commute where I can't enable now on v11 (both.3 & .6). What's interesting and counter to your statement is that if it's enabled before I hit that section, no problem and stays on.

Road is not narrow and clearly marked with yellow center and white edge.

Never had any issue on 10.x.

So Tesla does allow FSD to be enabled there, just not allowing to be initiated.

A spot???? Like only one spot where it doesn't work???? Must be nice. There's only a few spots where I can enable it unless I'm stopped or going slow (also if it was already on it stays on). It does flicker the icon occasionally, so I now spend most of my commute trying over and over to engage it and listening to the fail beeps instead of music.

Based on my experience, it's absolutely related to width of physically drivable space.
- I have no issue enabling it on multi-lane roads
- The times it flickers on when on a road with only one lane each direction almost always happens approaching an intersection with a side street (where the road widens slightly and/or it would see the side street as drivable space)

Putting those observations together, I believe it only enables when the physically drive-able space is a certain minimum width.



Anyway, I'm pretty well upset with v11 (both 11.3 and 11.6). It's an improvement almost everywhere I've driven except for the area that matters to me most: my daily commute. I've only got it active for maybe 30% of my commute now vs. 95% on almost all previous versions (and I'm talking 95% since October 2021 - I was an original 100 SS).

I can't get it to engage on single-lane roads, and on multi-lane roads it ALWAYS gets in the right lane as soon as possible, which on my commute route always becomes a turn-only lane soon, and I need to go straight. It also has the same random turn signals as previous versions, only now my brief disengagement to stop the errant turn signal becomes minutes of trying and trying and trying to get lucky and have it re-engage.

Ugh.

 

[animorph]

FSD was fine in moderate rain on the Interstate for about 10 hours of driving. Much of that was with the FSD is degraded warning, but it didn't quickly toggle or kick us out of FSD at all. At one point I was close to zero forward visibility while passing a large truck spraying lots of water. I had a hard time with that one, and maybe FSD did too, but it made it through just fine. It can see lane lines in back of the car and may have a good memory of the next short stretch of road that would keep it in it's lane even if the front camera is momentarily blinded.

My big problem has been coming too close to cars it's passing if speed-based or manual lane change is started too late. It seems to have only one lane change program. It takes its sweet time to change lanes and if it is speeding up as well to match the faster lane it leaves very little space between its right front bumper and the left rear of the car in the original lane. Seems like a version of the cut-in response problem, or longitudinal and lateral planners not talking with each other.

 

[powertoold]

https://twitter.com/x/status/1648123295831822336

 

[willow_hiller]

https://twitter.com/x/status/1648123295831822336

I imagine verbal feedback isn't fully disabled, it's just no longer enabled on every disengagement. Might need to meet certain campaign criteria before FSDb asks for feedback.

 

[powertoold]

I imagine verbal feedback isn't fully disabled, it's just no longer enabled on every disengagement. Might need to meet certain campaign criteria before FSDb asks for feedback.

Good point, it doesn't always pop up for me on 3.6

 

[aronth5]

For anyone that has the updated maps, NA-2022.44-14515 have you noticed any FSD improvement?

 

[MP3Mike]

Anyone else have a phantom car pace them on the highway?

Did you check and make sure your cameras are clean on that side?

 

[bhakan]

Finally, I saw the feedback request for autopilot disengagement. My car went to a bus stop lane which was not the right turn lane which was another 50 yards away. I did not provide a feedback as I was busy paying attention to the road, traffic, and pedestrians. At least, it exists.

 

[Twiglett]

had chance to use 11.3.6 on my regular weekend run (55 miles each way)
One of the two biggest difference I noticed was with the dreaded flashing yellow warning lights - it still tries to come to a complete stop on 70mph divided highway but at least it didn't set off the alarm bells and flashing screen when I forced it through with the accelerator pedal.
Interesting difference when using TACC, it still goes through the "take over immediately" histrionics.
The second one is that for the very first time it took this 270 degree turn to follow the route and it took it at a decent speed, complete with merge onto the next road. Very impressed with that.

One regression was that there were many more unneeded lane changes, even with the "reduced lane change" option selected. Most of them didn't show the changing lanes to follow route banner. Reported all the silly gotchas. Down to 20-25 disengages on this route, which is quite an improvement.

 

[Mark II]

Yeah mine's the opposite too.... and part of why I never believed Tesla would get L4 with this HW because nobody would accept an ODD of "not raining"


Previously NoA shut off in even moderate rain, let alone heavy. Dropping you back to basic AP.... while city streets still operated in moderate rain (it'd sometimes give a degraded message, but still actually navigate, make turns, etc).

Haven't driven single stack highway in moderate rain to see what's up with it yet, but I'd expect more city streets type behavior where it retains more function than just giving up like NoA did.

Overall it is better when it is raining than NoA. The problem is the warning shows up much sooner than with NoA, even thought it does not seem to impact the ability to change lanes etc. The issue I am having is the warning kept making a ding sound every minute to the point it was so annoying I stopped using FSD, even though it was working fine.

I did not notice the issue on normal roads, even in the rain. Not sure if it was speed related or what. Only driven once on the freeway since getting 11.3. Hopefully it was an anomaly.

 

[Ramphex]

How are these shills getting 11.4 if no one on TeslaFi got it yet?