FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Pdubs]

How is it that FSD can operate without a navigation plan? Why is that a thing?

don't think of it as navigating, you need a destination to navigate to. think of it as maintaining.

 

[Pdubs]

In regards to the wiper function, is anyone using RainX on their windshield? Perhaps this has something to do or not with the wiper function?

good thinking but in my experience, what makes the wiper function so poorly is changes in light.

 

[Pdubs]

Phantom braking still occurs, way more during city driving. For me it's not actually braking, just a quick regen slow down of 1 to 5 mph.
All my highway slow downs seem map related, since they are always in the same spot.

around here, with these stupid middle sections of road that people have to come into from the opposing lanes to cross your road, freak the ever loving sht out of my car. I'm talkin spill your drink hard braking. I'm anticipating them now and have my foot on the accelerator to over ride, same as when I come up on blinking yellows, but christ is it a hard brake the first time you're not expecting it.

 

[aronth5]

don't think of it as navigating, you need a destination to navigate to. think of it as maintaining.

I used to always say "navigate to" or "drive to" but now just say the destination name and FSD goes there.

 

[Joboo7777]

It is odd that some of us have had minimal or no dry wipe or no-wipe-in-rain symptoms. Clearly there is a recent software bug which make these worse, but only much worse for some yet not for others.

Regarding the spot on the windshield theory, it makes sense that quick changes in lighting might cause a false raindrop perception: A mote of dust or small bit of condensation on the windshield, being so close to the camera would be way out of focus, and so would appear as a low contrast but stationary shadow on the image. Image processing would need to ignore this sort of image flaw. But bright light coming from in front would illuminate that mote, turning its' image from a shadow to bright, making it appear much like a water droplet landing on the windshield. A windshield with fewer such spots this would be less likely to be be perceived as a rain storm. A windshield with many such spots might make it hard for image processing to distinguish real rain. All speculation on my part, of course.

But with all the reports of varying results, ArchHamster had a very interesting idea, certainly worth someone here trying. If it works as it did for him, it might offer some relief while we wait for Tesla to fix it's bug:


That link leads to a video showing how to clean the inside of the glass for the front cameras:

I wonder if tinting the windshield helps to reduce dry wiping events? I have a 23M3LR with a clear tint on the windshield to reduce UV and have no issues at all. Once in a blue moon I’ll get a wipe but very rare and does not comitnue.

 

[Pdubs]

I used to always say "navigate to" or "drive to" but now just say the destination name and FSD goes there.

I actually like that feature. I really like the way they integrated all the options of say, "Publix", then you can click the map Letter to go to the one you want. It's the little things...

 

[rlsd]

Can we tell FSD to resume route to destination after cancelling navigation?

 

[Pdubs]

I wonder if tinting the windshield helps to reduce dry wiping events? I have a 23M3LR with a clear tint on the windshield to reduce UV and have no issues at all. Once in a blue moon I’ll get a wipe but very rare and does not comitnue.

Negative Ghost Rider. Mines tinted and I still get them. I put ceramic on the wind screen as well and thought that may have been causing it, but after reading about that shtty RainX people are using, I don't think either have anything to do with it. Convinced it's a light processing issue.

 

[Supcom]

Can we tell FSD to resume route to destination after cancelling navigation?

No. But the last destination is likely at the top of the suggested destination list.

 

[Pdubs]

Can we tell FSD to resume route to destination after cancelling navigation?

If you cancel your navigation, there'd be nothing to resume?

 

[sleepydoc]

It will be very useful if we have command to Pause FSD, resume FSD,... to the last set destination.

They have that. You push up on the stalk. Then you double tap down to resume.

 

[rlsd]

They have that. You push up on the stalk. Then you double tap down to resume.

I forgot that. Doing that every day. My head is mixed up with android auto.

 

[Corey Alemand]

FSD v11.4.7.3 downloaded late last night. Took it out for a drive this morning. I think the FSD has been getting worse and not better. Things that had improved in prior releases are now back and causing me fits. First, once again when the car needs to take a right turn at a stop sign, the wheel turns left and so when the car starts to move it goes left before turning right. It can be extremely concerning when there are multiple cars at an intersection and a car is turning to go into the lane to the left of me, and when my car goes it does this little left turn. Second, in my neighborhood some of the streets do not have curbs which causes the Tesla to treat them like narrow roads when in fact they are a 2-way street. This results in the Tesla driving down the middle of the road. It kind of gets back into its lane when it approaches a stop sign but not quite. I get nasty stares and a periodic horn when drivers coming toward me see me in their lane. It is ridiculous that the car does this. Third, the random braking -- worse than it has ever been. I never really had issues with it before the last couple of updates. Fourth, once again if the car is in the far right lane and you get close to an exit off the highway, when the lane widens the car drifts to recenter. It had stopped doing that a few updates ago. Now it is back again.

 

[drtimhill]

It does not otherwise know what to do at intersections. It ignores my signal input at a T intersection and turns right instead of left. It takes an exit when I want to go straight ahead. It just makes up a path if there is no nav. I’ve only seen this when testing or when canceling and remaking a trip while already in FSDb. Just don’t get the point of Random FSD.

Think of it as an "I feel Lucky" button.

 

[aronth5]

Can we tell FSD to resume route to destination after cancelling navigation?

There is no "resume" voice command that I'm aware of but just use the right scroll wheel to trigger a voice command and just say the name of the destination. FSD will simply drive there. Or you could select navigation and pick the prior destination. First option is easier.

 

[Wnuk]

They can usually force a new update to clear it

It got pushed again and installed this time, didn’t need to request service. That was first time an update did that on my car, strange

 

[syncword]

around here, with these stupid middle sections of road that people have to come into from the opposing lanes to cross your road, freak the ever loving sht out of my car. I'm talkin spill your drink hard braking. I'm anticipating them now and have my foot on the accelerator to over ride, same as when I come up on blinking yellows, but christ is it a hard brake the first time you're not expecting it.

Jumping in on this thread for the first time after subbing to FSDb a couple days ago for a month, have lots of thoughts/discoveries already. My car is currently running 11.4.4.

You got that right. I noticed something about it today: if the car in the middle is moving at all - like if it's creeping (just like FSDb does at a stop light, ha!) that's where the hard braking happens, every time so far. I guess it's extrapolating that if the creeping car keeps going, there's a collision potential - which I suppose there is. However if the car is stopped when it comes into the field of attention, FSDb seems OK with it and keeps going.

This is FL secondary hwy 1 along the E. coast, central part. I wouldn't dream of using FSDb during busy morning and late afternoon weekdays. So many of those cars waiting to turn left in that middle lane, plus people darting across the entire highway - unsafely of course, I'd snag a rear-ender lickety split. I sure hope someone at Tesla is listening to the "why did you disengage" feedback, because they're getting a LOT of it from me so far. They should be paying *me* $200/mo for doing this, not the other way around.

Oh well, how's this for another: There is a section of the same highway with small traffic lights above the road that are usually off, in front of a fire station. Not lit at all. All us locals pay them no attention because 99.99% of the time they're OFF. Imagine my surprise when I saw "slowing for traffic light" as the car literally tried to haul itself down right in the middle of 55 mph traffic as it passed those. Disengage and accelerate to avoid being rear-ended... And yesterday approaching a clear rail crossing when the sun was shining (from behind me) a little through the red crossing lights... yep, "stopping for traffic light" just as it went over the tracks.

With all this, I do think Tesla is generally on the right track with FSD and going full steam ahead with E2E NN and the Dojo for training. It's just a long road - no pun intended. I dabble in LLMs and was today thinking - with the right context of the training videos for v12 - particularly location, but also weather and time of day, it could end up quite good. Think of a prompt in a well-trained LLM - you tell it approximately where you are, and it'll do a good job generalizing how people drive, common traffic features/road designs, etc.

 

[Pdubs]

Jumping in on this thread for the first time after subbing to FSDb a couple days ago for a month, have lots of thoughts/discoveries already. My car is currently running 11.4.4.

You got that right. I noticed something about it today: if the car in the middle is moving at all - like if it's creeping (just like FSDb does at a stop light, ha!) that's where the hard braking happens, every time so far. I guess it's extrapolating that if the creeping car keeps going, there's a collision potential - which I suppose there is. However if the car is stopped when it comes into the field of attention, FSDb seems OK with it and keeps going.

This is FL secondary hwy 1 along the E. coast, central part. I wouldn't dream of using FSDb during busy morning and late afternoon weekdays. So many of those cars waiting to turn left in that middle lane, plus people darting across the entire highway - unsafely of course, I'd snag a rear-ender lickety split. I sure hope someone at Tesla is listening to the "why did you disengage" feedback, because they're getting a LOT of it from me so far. They should be paying *me* $200/mo for doing this, not the other way around.

Oh well, how's this for another: There is a section of the same highway with small traffic lights above the road that are usually off, in front of a fire station. Not lit at all. All us locals pay them no attention because 99.99% of the time they're OFF. Imagine my surprise when I saw "slowing for traffic light" as the car literally tried to haul itself down right in the middle of 55 mph traffic as it passed those. Disengage and accelerate to avoid being rear-ended... And yesterday approaching a clear rail crossing when the sun was shining (from behind me) a little through the red crossing lights... yep, "stopping for traffic light" just as it went over the tracks.

With all this, I do think Tesla is generally on the right track with FSD and going full steam ahead with E2E NN and the Dojo for training. It's just a long road - no pun intended. I dabble in LLMs and was today thinking - with the right context of the training videos for v12 - particularly location, but also weather and time of day, it could end up quite good. Think of a prompt in a well-trained LLM - you tell it approximately where you are, and it'll do a good job generalizing how people drive, common traffic features/road designs, etc.

Yes. That is one of the roads I was referencing. Have to keep your feet by the pedals.

 

[Supcom]

11.4.7.3 installed today. Not expecting any significant difference and, so far, meets expectations. an xx.x.x.3 should have very minor changes. Maybe just changed the password for Elon mode...

One thing I did notice was a tendency to slow way down when entering an intersection after having been stopped for a red light. Even with a follow car, ego braked on entering some intersections and needed to be prodded along so as not to be too annoying to the guy behind me. Seemed worse than usual, but I have found the car to be worse on the first couple drives. Or, so it seems...

 

[FSDtester#1]

If you cancel your navigation, there'd be nothing to resume?

View attachment 984425

I actually forgot one time on my way home to put in a destination.
It was about an 8 mile trip. The car just assumed I wanted to go home and took me there. It wasn't until I was about a half a mile away that I noticed what a d bag I was.
So I just let it finish....