FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[rlsd]

Have yo ever tried singing in the rain, with Gene Kelly?

Nah. Only this one

 

[DrGriz]

I'm wid youse, FUBARed beyond all recognition.........

You can say that again...

 

[heapmaster]

Here's a challenging scenario with a highway crossing of a very lucky deer.

From the side/back camera views one can see FSD steers left well after deer crosses the road. Unfortunately there's no visible instrumentation but it's mostly dark out and what appears to be a brake light reflection is seen via the side camera view - it appears to initiate about the same time as the left steering.

https://twitter.com/x/status/1714074764711706639

Nice, still dont trust TX with my car, I lost two cars to deer in that state, I drive around it now.

 

[Dewg]

Nice, still dont trust TX with my car, I lost two cars to deer in that state, I drive around it now.

Or, from a different point of view, TX lost two deer because of your cars.

 

[PACEMD]

You can say that again...

It's like a Yogi Berraism, all over again..........

 

[JulienW]

So much for there being even a 0.00000001% chance of 11.4.7.3.x.x.x.x.x.x.x.x.x. (enough with the micro point updates) fixing the Dry Wiping problem. And naturally I just checked and it is available for me to download. Funny all the updates I was excited to get I was always in the last group. With a meh update I get right away.

 

[rlsd]

So much for there being even a 0.00000001% chance of 11.4.7.3.x.x.x.x.x.x.x.x.x. (enough with the micro point updates) fixing the Dry Wiping problem. And naturally I just checked and it is available for me to download. Funny all the updates I was excited to get I was always in the last group. With a meh update I get right away.

View attachment 983941

Drive to a car wash station without activating car wash mode to see if the wipers work .

 

[sleepydoc]

We just took a trip from our home, west of Minneapolis to Madison, WI. 287 miles, approximately 4 ½ hours of driving. Stopped at the EauClaire, WI super charger. The entire trip was on FSD and the only intervention I had was on highway 12, west of Madison when it started entering a turn lane instead of going straight. It may have been able to 'right' itself but I didn't feel like giving it the chance. Other than that, all I did was adjust the speed as necessary (e.g. to more quickly pass a truck so I could get out of the way of other cars behind me.) I don't count these because they were strictly personal preference and were in no way safety related.

The drive included getting from our home to the interstate, navigating through Minneapolis & St. Paul, exiting at navigating to the supercharger, navigating back to the interstate then onto a U.S. highway and finally navigating through Madison to our Hotel.

Oh, and I also only had a single phantom wipe. (clear, sunny weather the entire trip.) My car updated to 11.4.7.2 recently, so maybe that's the good version?

 

[Ramphex]

So what’s new in this 27.7 update?

 

[swedge]

everyone is having the same issue

Not a valid assumption. I have had almost no dry wiping. One time when the sun was low, straight ahead. I and others have reported having little or no dry wiping. But generally, folks don't tend to post about a problem that they don't have. So, selection bias makes it seem like everyone has the problem.

It could be that with the new version it became much more sensitive...

The way to test the dirty windshield theory is to clean your windshield.

 

[PR0FESSOR]

With a perfectly clean windshield, I exited from my shaded driveway on to the road, pulling directly towards a rising sun and got a dry wipe. As someone else mentioned, pulling from shade to sun, especially a rising or setting sun, seems to set off the wipers. I have about one dry wipe every day, and the windshield wipers NEVER come on when it rains. I have to move from auto mode to manual mode after it has started raining, and then back to auto mode, and the wipers will work properly… until it stops raining and then the whole process starts over again.

Joe

 

[kabin]

Not a valid assumption. I have had almost no dry wiping.

That sounds like a dry wiping problem.

 

[swedge]

With a perfectly clean windshield, I exited from my shaded driveway on to the road, pulling directly towards a rising sun and got a dry wipe. As someone else mentioned, pulling from shade to sun, especially a rising or setting sun, seems to set off the wipers. I have about one dry wipe every day, and the windshield wipers NEVER come on when it rains. I have to move from auto mode to manual mode after it has started raining, and then back to auto mode, and the wipers will work properly… until it stops raining and then the whole process starts over again.

Joe

It is odd that some of us have had minimal or no dry wipe or no-wipe-in-rain symptoms. Clearly there is a recent software bug which make these worse, but only much worse for some yet not for others.

Regarding the spot on the windshield theory, it makes sense that quick changes in lighting might cause a false raindrop perception: A mote of dust or small bit of condensation on the windshield, being so close to the camera would be way out of focus, and so would appear as a low contrast but stationary shadow on the image. Image processing would need to ignore this sort of image flaw. But bright light coming from in front would illuminate that mote, turning its' image from a shadow to bright, making it appear much like a water droplet landing on the windshield. A windshield with fewer such spots this would be less likely to be be perceived as a rain storm. A windshield with many such spots might make it hard for image processing to distinguish real rain. All speculation on my part, of course.

But with all the reports of varying results, ArchHamster had a very interesting idea, certainly worth someone here trying. If it works as it did for him, it might offer some relief while we wait for Tesla to fix it's bug:

I haven't seen this posted in this thread yet. You guys with wiper problems, have you tried this: Reddit - Dive into anything

Cleaning the windshield under (over?) The camera cluster from the side inside the cabin (requires removal of camera box)?

11.4.7.2 still trying to murder me btw. I'm starting to think I'm on some kind of list.

That link leads to a video showing how to clean the inside of the glass for the front cameras:

 

[PACEMD]

So what’s new in this 27.7 update?

I've heard it dry wipes with a certain je ne sais quoi. Pardon my French.........

 

[cbdream99]

Not a valid assumption. I have had almost no dry wiping. One time when the sun was low, straight ahead. I and others have reported having little or no dry wiping. But generally, folks don't tend to post about a problem that they don't have. So, selection bias makes it seem like everyone has the problem.

It could be that with the new version it became much more sensitive...

The way to test the dirty windshield theory is to clean your windshield.

I am wondering if this only impacts newer Tesla, I have a 17 S and I don't experience much of the dry wipe problem (not completely zero instance though), may be certain angle of lights, shades, or timing combination which is hard to predict, or may be it is just Texas has only sun, no shades, and no rains, no clouds, and my windshield always stay clean.

 

[jebinc]

Black Tesla,” - One of the most objective FSDj testers (re re re refreshed Model S, not 3/Y). He just released his 11.4.7.3 review. MORE and increasingly more JUNK!!!

 

[fasteddie7]

Could it be that the cameras are too good? I grabbed several dashcam clips while the wipers weren’t going and I could kind of see while driving, and barley see anything while in a parking lot, and to my surprise, the view of the dashcam did not represent what I could see through the windshield, you could barely tell it was raining.

 

[legendsk]

So what’s new in this 27.7 update?

The release notes were changed to reflect the new version numbers.

 

[legendsk]

Not a valid assumption. I have had almost no dry wiping. One time when the sun was low, straight ahead. I and others have reported having little or no dry wiping. But generally, folks don't tend to post about a problem that they don't have. So, selection bias makes it seem like everyone has the problem.

It could be that with the new version it became much more sensitive...

The way to test the dirty windshield theory is to clean your windshield.

I believe it goes back to this summer when the update changed the visualization so much. The camera images were no where nearly as good. Instead of 3/4 mile away, semi trucks don't appear until 150 feet or so, trash cans not until about when I can see them out the passenger window. The top 1/3 of the display is whited out. It's as though someone decided that if the computer had fewer pixels to process, it would be faster so the resolution was reduced from all the cameras and the images just aren't as good as they were, prior to that update. I keep hoping an update will bring them back. But not yet 2023.7.7 and still no better. This could also be the cause for dry wipes - can't see as good as it used to, therefore perhaps the windshield needs wiping? That didn't help, well let's try it again. Nope, I give up.

 

[fasteddie7]

With a perfectly clean windshield, I exited from my shaded driveway on to the road, pulling directly towards a rising sun and got a dry wipe. As someone else mentioned, pulling from shade to sun, especially a rising or setting sun, seems to set off the wipers. I have about one dry wipe every day, and the windshield wipers NEVER come on when it rains. I have to move from auto mode to manual mode after it has started raining, and then back to auto mode, and the wipers will work properly… until it stops raining and then the whole process starts over again.

Joe

I was wondering if you had dry wipes since you’re HW4. I have not had a single dry wipe yet, but I also don’t have any wet wipes either. If I keep it on auto, the entire windshield will be covered and I’ll have to manually start the wipers or they never come on. FSD seems to still be able to function without the wipers but I can’t babysit it to make sure it doesn’t hit a curb since I can’t see.