FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[jebinc]

I don’t remember asking for your opinion of my opinion. You’re welcome to keep providing it though. Anything that makes my back come off the seat and my body move towards the steering yoke by me is considered as aggressive braking. You can throw in the synonyms of slam on the brakes.

It is important to remember that one can slam on the brakes WITHOUT holding down the pedal until complete stop, or wheel lock, or skid. Stomping the brakes even for a second is still stomping the brakes. I can see you be the guy who argues the semantics of how many seconds it would have to be to be considered as slamming on the brakes, so just don’t bother if that was going to be your next move. Normal people do not do that while driving in normal conditions and nor should the junk.

The “j” in “junk” should be capitalized. Geezzz

 

[JHCCAZ]

v11 4.2 Improvement: Today for the first time, it made a right turn at a major intersection where there is a bus stop immediately on the right. I take this route once a week, and previously it would always turn into the bus lane and try to drive there. Today it properly negotiated the turn into the rightmost through lane, properly bypassing the bus stop lane.

It seems to be slightly better with the transient phantom lane-change blinks near intersections. This is when you're traveling straight through but it momentarily threatens to move into or away from a turn lane. Quite common previously, still happens on 11.4.2 but less often.

It's still far too anxious to move out of the current lane in moderate-to-heavy traffic, when it should stay put in anticipation of a turn coming in a mile or even less. I had one spot like this get fixed in the last few weeks of 11.3.6 - so probably a map hints Improvement - but this general problematic behavior still needs attention.

 

[powertoold]

It's still far too anxious to move out of the current lane in moderate-to-heavy traffic, when it should stay put in anticipation of a turn coming in a mile or even less.

Is it moving out to get into a faster lane? If so, chill mode would help with that

 

[jebinc]

That is exactly what mine says except depending on the aggressiveness of the braking, I get a few more expletives with the words "turn it off now", lol

My wife refuses to let me turn on FSDj (aka, “the Junk”) when we are in the Plaid. She prefers it when we take the Rivian, as she knows I can’t silently and secretly turn on the Junk in the Rivian, as I have tried to do in the Plaid. I say, “tried” as the Junk gives it’s self away with erratic moves and abrupt braking. I get scolded and she now asks to take the Rivian as a result.

 

[jebinc]

I wanna buy an S/X just to see how much of a difference FSD is in one.

If you do, expect Tesla to gift you the “normal” vibration in the front end — free of charge.

 

[JHCCAZ]

Is it moving out to get into a faster lane? If so, chill mode would help with that

Chill mode doesn't seem to help as much as Minimal Lane Changes. However, even with these settings it's simply not smart enough to recognize when it's inadvisable. The lane to the left is not significantly faster in the rush hour traffic, the turn is coming up soon enough, and FSD isn't terribly talented at executing the lane changes in traffic.

If anything, Chill mode makes it worse because it's less aggressive at getting back once it's made the change. Being overanxious to change lanes is bad to begin with, and it's especially incompatible with timid execution of the maneuvers.

I'm not saying this is the worst problem ever, but I'm keeping track of it as a consistently undesirable behavior that needs work. Give FSD credit where it's due and there's plenty of that, but also point out behavior that any observer on the road would say is poor driving. It's part of the testing and feedback.

 

[powertoold]

If anything, Chill mode makes it worse because it's less aggressive at getting back once it's made the change. Being overanxious to change lanes is bad to begin with, and it's especially incompatible with timid execution of the maneuvers.

Interesting, have you tried chill mode with 4.2? I find the behavior is tweaked.

 

[JHCCAZ]

Interesting, have you tried chill mode with 4.2? I find the behavior is tweaked.

Thanks. My comments about Chill mode are admittedly based on 11.3.6, so I'll try it specifically on the latest release.

However, my other comments still stand, as anyone watching the car change left into an equally crowded lane, make no significant progress and then have to attempt an inexpert change back half a mile later, would rightly have an unkind comment about the competency of the driver. No one would say, " oh that's just an [Average | Aggressive] everyday driver".

 

[LowlyOilBurner]

If you do, expect Tesla to gift you the “normal” vibration in the front end — free of charge.

Well, if I do, I was also hoping to opt for the excessive rear tire wear package, if that’s possible?

 

[rlsd]

I had a really bad phantom breaking yesterday morning. The car exited the freeway then it made a hard breaking when the car in front was at least 100 yards away.

 

[EVNow]

No one would say, " oh that's just an [Average | Aggressive] everyday driver".

Reminds me - I used to commute a few years back for just a few miles, a couple of exits away. I'd just stick to the right lane, even if slow. There would be cars trying all kinds of aggressive lane changes but finally end up behind me at the exit

 

[Reddy Kilowatt]

Im finding 11.4.2 a little more polished than the previous version. Almost a week driving but not my long weekend runs yet.
-Cured one of my 4 way stop sign issues where it crept all the way across the intersection.
-Smoother turns, when close together. They are not as mechanical feeling, now starting to have a natural flow.
-Not seeing the car slamming on the brakes when approaching a red light, relying on regen more, and less braking.
-Have not seen any real regressions on this version yet.
-Acceleration was to quick out of a turn. reduced car to sport from Ludicrous mode, solved that issue.

Had an issue this morning when a car cut me off next to an exit ramp to run across three lanes to turn left at next intersection. my car slowed and followed it over the three lanes also, instead of simply maintaining distance.

Has changed lanes turning in a double left turn twice.

 

[Scott Chapman]

I have to say, it feels super weird running the latest version. I have no idea what to do with all the free time I have since I am no longer checking to see how many cars are getting the update while I am waiting!

That being said, I am generally enjoying using FSDb, I do definately notice the general improvements that have come with each release. There are still situations that can use improvement, but overall I find it pretty amazing.

 

[jebinc]

Well, if I do, I was also hoping to opt for the excessive rear tire wear package, if that’s possible?

Very (possible)! 100% if you get the $4500 21” wheels!

 

[Ramphex]

The Junk got me good today. As I was driving slowly through my community, I figured I’ll allow it to drive for a bit while I use a microfiber cloth to clean my sunglasses. So as we pull up to the gate where it “brakes aggressively” and stops like 20 feet before the sensor to trigger the gate to open, it does the usual spill your coffee braking and stops.

Not a problem, I do what I always do and give it a bit of accelerator to move up to the sensor. 11.3 handled that fine, it moved up to the sensor and waited for the gate to open and then proceeded through. NOT TODAY JUNIOR. The Junk threw up red take over hands, turned on hazards, and a message came up saying autosteer is no longer available due to a system error.

I wish the car that was behind me was a Tesla, I would ask for a recording to see how this debacle looked like from the outside view

 

[Ramphex]

Well, if I do, I was also hoping to opt for the excessive rear tire wear package, if that’s possible?

I got that package, it’s pretty cool, worth the money

 

[EVNow]

I figured I’ll allow it to drive for a bit while I use a microfiber cloth to clean my sunglasses.

User error - so you wanted to be doing something else at known problem sites. No wonder you have all kinds of issues with FSD.

 

[jebinc]

The Junk got me good today. As I was driving slowly through my community, I figured I’ll allow it to drive for a bit while I use a microfiber cloth to clean my sunglasses. So as we pull up to the gate where it “brakes aggressively” and stops like 20 feet before the sensor to trigger the gate to open, it does the usual spill your coffee braking and stops.

Not a problem, I do what I always do and give it a bit of accelerator to move up to the sensor. 11.3 handled that fine, it moved up to the sensor and waited for the gate to open and then proceeded through. NOT TODAY JUNIOR. The Junk threw up red take over hands, turned on hazards, and a message came up saying autosteer is no longer available due to a system error.

I wish the car that was behind me was a Tesla, I would ask for a recording to see how this debacle looked like from the outside view

We have a gate also... The Junk does fine going out, but it now (since 11.4.2) throws up the "red heads of impending death," while slamming on the brakes, when returning.

 

[Olle]

I have learned that the only sure way is to heavy double tap on the brake to disengage when warnings (any warnings) comes up. This would give brake light flash warning to the car behind, your car stay on course with foot on accelerator. This would also hopefully avoid a penalty jail.

On the topic of disengagement: After a go pedal nudge, FSD beta often decelerates to a fraction of the set speed. It seems like if it’s set to 70 and parks itself in somebody’s blind spot while blocking the left lane, I nudge it momentarily, to 75 to get out and ease off the pedal, it will go down to 30 and then slowly accelerate back to 70, even on the highway with cars whooshing by at 70 mph. The only way I’ve found out of this is to disengage, accelerate and re engage. Anyone found a way around this?

 

[archae86]

I nudge it momentarily, to 75 to get out and ease off the pedal, it will go down to 30 and then slowly accelerate back to 70, even on the highway with cars whooshing by at 70 mph. The only way I’ve found out of this is to disengage, accelerate and re engage. Anyone found a way around this?

In the circumstance you describe, I choose to do my nudge with the speed scroll, rather than the accelerator.