FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[aronth5]

I don’t think an obstructed view is the problem. My HW4 Model X repeatedly attempts to pull out into a street with cross traffic even when there are no obstructions at all. I think it’s a v11/software behavior problem, not a camera placement problem.

If the car properly and slowly creeps forward at that intersection, the maneuver is possible. The problem now is that after stopping, it either creeps forward aggressively (giving me and other drivers the appearance that it’s going to go), or actually pulls out without even checking.

Both software issues. That’s my take on it.

I partially agree but I have many intersections where you cannot creep any further because there is no shoulder and your affecting cross traffic. This is actually one of tbe least obstructed view intersections I have.

 

[PACEMD]

I'm not a software engineer...........

Have you at least played one on TV? Asking for a friend...........

 

[PACEMD]

I didn't realize FSD Beta could make U-turns.

Only stolen European expats can make U-turns........

 

[PACEMD]

Repeatability, dependability and reliability. Yep, not with this product.

Almost like it's close to sometimes being like junk?

 

[bhakan]

At least, in one of my trips last week the car did not stay on the left lane which was "exit only" lane and had always been staying there and in fact speeding also on that lane. It used to get stuck as the car on FSD cannot move to the traveling lane. Finally, the car stayed calm and carried on in the lane it is supposed to. I haven't driven my car while we are in frozen tundra and my backyard is like Antartica minus the penguins. With so many pot holes I don't think the FSD will slow down on those lanes.

 

[JulienW]

First I will UNcomplain about dry wiping. Haven't had a single dry wipe for a while now and very happy.

Oddly FSD Beta still drives on the wrong side of the road at this particular construction site. It is near 100% at always doing this for almost a year now (since the lane was diverted for a high rise construction). Keep thinking it will be fixed since it seems so obviously WRONG. Today I slowed to 20MPH to give it plenty of time to figure it out. Seems it was happy switching to British driving and no sight of correcting.

Also oddly my Rearview camera (and Front place holder) are missing. The Left and Right Fender camera feeds are there (on the bottom) but the top of the video doesn't have this. Has anyone else noticed this or is this just an anomaly?

 

[Todd Burch]

It appears 11.4.9 is still pretty heavily-dependent on map data.

On Saturday my car was driving through a neighborhood. There was one intersection where the stop sign was not shown on the map. This is the one intersection where the visualization showed the stop sign (so the NN was seeing it) but the planning code didn’t try to stop for it.

 

[kabin]

It appears 11.4.9 is still pretty heavily-dependent on map data.

Yep! I've experienced that with a newly installed roundabout. FSD consistently throws me into the roundabout as if it isn't there.

 

[rlsd]

I drove 200 miles round trip on Saturday. On the way out during the day the wipers was slow (about level 1) when it rained hard. I had to manually set it to level 2 or 3. On the way back at night the wipers automatically go to level 3 or 4 when the rain was not very heavy. I have to manually set to level 1 or 2. Maybe FSD took into account of the day/night situation or the wind direction?

 

[Pdubs]

It appears 11.4.9 is still pretty heavily-dependent on map data.

On Saturday my car was driving through a neighborhood. There was one intersection where the stop sign was not shown on the map. This is the one intersection where the visualization showed the stop sign (so the NN was seeing it) but the planning code didn’t try to stop for it.

And conversely, there’s a road going to my development that no map data has a stop sign on, but there is a small stop for pedestrians crosswalk they just put in. Every time I come up on it, the car says, “Stopping for Stop Sign”. There is no stop sign. (Just had a flashback to The Matrix)

 

[enemji]

I just did a crazy experiment. I applied RainX to just the portion of the windshield where the camera is.

Today is raining bad. Took it out for a ride on the highway. The speed of the car kept the windshield clean for the camera but the rest of the windshield was so bad I could not even see through it. The result was FSD was super confident, and it did not even use the wipers. Made it scary for me as I could not see anything

 

[EVNow]

I just did a crazy experiment. I applied RainX to just the portion of the windshield where the camera is.

Today is raining bad. Took it out for a ride on the highway. The speed of the car kept the windshield clean for the camera but the rest of the windshield was so bad I could not even see through it. The result was FSD was super confident, and it did not even use the wipers. Made it scary for me as I could not see anything

I've always wondered whether to apply Rainx in front of the camera or not. This winter I've been very slow to apply Rainx ... will do so soon (including the camera).

 

[Supcom]

I've always wondered whether to apply Rainx in front of the camera or not. This winter I've been very slow to apply Rainx ... will do so soon (including the camera).

I've always left the area in front of the camera untreated on the assumption that the auto wiper function is tuned with a bare windshield.

 

[EVNow]

I've always left the area in front of the camera untreated on the assumption that the auto wiper function is tuned with a bare windshield.

Yes - thats what I did last year. I'm ready to try another way this time !

 

[Twiglett]

I've always left the area in front of the camera untreated on the assumption that the auto wiper function is tuned with a bare windshield.

as it seems the auto wiper is tuned to something other than rain, I'm willing to give it a try

 

[FSDtester#1]

as it seems the auto wiper is tuned to something other than rain, I'm willing to give it a try

I put rainx on everything including all cameras... the rear camera works way better with it.

 

[bhakan]

I put rainx on everything including all cameras... the rear camera works way better with it.

A Tesla employee told me when I got my first MS in 2017 to use Rainx on the rear camera and the entire front windshield.

 

[Yelobird]

A Tesla employee told me when I got my first MS in 2017 to use Rainx on the rear camera and the entire front windshield.

As it actually says in the manual Not to use windshield additives I never use them. If you want auto wipers to never go on sure coat the screen in front of the camera and while you won’t see the cameras won’t and the wipers will never activate.

 

[Twiglett]

As it actually says in the manual Not to use windshield additives I never use them. If you want auto wipers to never go on sure coat the screen in front of the camera and while you won’t see the cameras won’t and the wipers will never activate.

I got out of the habit of using RainX during the time the wipers worked well, this reminds me to get a new bottle. But yes, I used to do the windshield but leave the camera area alone. Especially now that the new (new, new) autowipers are supposedly using all cameras to figure out if its raining or not.

 

[Pdubs]

I put rainx on everything including all cameras... the rear camera works way better with it.

I ceramic coated the windscreen but did nothing with the cameras. I can’t offer an opinion on how good or bad it is as I don’t really pay that much attention. It amazes me how everybody is so detailed n their experiences with everything. It makes me extremely envious and makes me want to be a better man. Mostly it’s because I forget how good or bad it is. What do I need to do?