FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Porkalletta]

Had a very impressive, no intervention (11 pm) return drive from downtown L.A. last night to Ventura. Intense traffic at speed limit or above and slowing traffic rerouting to Highways 110-10-405-101 with FSDb 11.3.6. If you have ever driven these in SoCal you know how intense this corridor is to drive. Some SoCal drivers are nuts. Weaving at 100 mph not uncommon. Construction...

Only low was 3 light phantom breaks (73 to 68 mph) before overhead bridges to which I pushed the accelerator due to intense traffic behind me.

Highs:
1. Auto lane changes were more efficient than I could have done due to heavy 5 lanes of traffic and darkness.
2. Changed following distance and lane change aggressiveness automatically depending on traffic density.
3. Merged from one highway to the other without intervention in very heavy traffic at high speeds. It automatically moved over 5 lanes from the left HOV lane in time for the merge several times despite very heavy fast moving traffic.
4. It moved from the far left, non-HOV lane when faster cars were approaching from behind. Usually doing about 85 mph. I was at 76 mph. Never annoyingly.
5. Even my copilot wife was impressed and will start using FSDb.

It's getting there. Now we just need SoCal big city to work better.

 

[kabin]

I watched that clip a dozen times and I don't see any problem with that highway exit. The car was a bit off center, but certainly not onto the lane markers.

The linked video shows how FSDb badly handled a truck pulling out of the corner gas station while the Tesla was trying to make a left turn through an intersection. Watch the planner. FSDb sees the truck, considers it an obstruction, tries to route around to the right of it, realizes that there is a car to FSDb's right, and decides to go straight ahead a bit and stop in the intersection. Definitely not the right behavior, but I didn't see anything about trying to cut a corner.

The highway snafu was more than off-center. It was out of the lane as evidenced by tires thumping over roadside reflectors. This isn't the first time this has happened. Recall a previous bandaid fix was to add lane center offsets for higher speed turns. No surprise that kludge fix isn't working.

Left turn cuts remain an ongoing issue. 11.3.6 does the same thing regularly for me even if a vehicle occupies the oncoming left turn lane. The truck was a nonissue - even if it was close it could easily be handled with slight deceleration versus steering far left and heading into oncoming traffic as suggested. No. Notice the poorly lined path from early on - that's why he disengaged. It required quick and substantial manual steering input to get back on proper path after disengagement.

 

[dgatwood]

Yeah idk the thing about this is it's hard to know what is "the final product"...how does one measure such a thing?

The same way as you do with Google products: when they remove the word "beta".

 

[Supcom]

I consider this to be an accident mitigation strategy. By staying out of the right lane, there will be fewer interactions with cars that are changing lanes, speeding up, slowing down, etc. All of those are opportunities for FSDb to make a mistake - or for another driver to do something idiotic against which few could defend.

You mentioned a lack of need to stay out of the right lane. I doubt their code worries about context that much. I think it just says Right Lane Bad and goes from there.

In urban areas, sitting in the rightmost interstate lane leads to a lot of slowing for vehicles entering the highway before attaining the normal highway speed. So, if there are three, or more lanes, moving out of the rightmost lane is good practice, and one that I have used long before there were self-driving cars.

In rural areas, this is less of an issue as exists are far apart and there is less traffic.

 

[PVia]

It's getting there. Now we just need SoCal big city to work better.

Exactly...

 

[JB47394]

In urban areas, sitting in the rightmost interstate lane leads to a lot of slowing for vehicles entering the highway before attaining the normal highway speed. So, if there are three, or more lanes, moving out of the rightmost lane is good practice, and one that I have used long before there were self-driving cars.

Sure. I learned the same rules 45 years ago. I noticed a while back that FSDb would place the car in the left lane when there were only two lanes. I considered that a clear attempt to keep the car away from too many challenging encounters with other traffic. It sounded like the prior poster was running into that sort of behavior. I just got home after driving a road with two lanes to choose from and, unlike past versions, FSDb chose the right lane. Progress is progressing.

 

[EVNow]

First time FSD went out of the turn lane near home. There was another vehicle in front in the left turn lane.

BTW, I'm disengaging more often now so that I can send voice reports. Wonder whether it will make any difference....

 

[Therealm80]

how do I fix a wrong speed limit? There are several places where either on a street is wrong or in a roundabout it is 25 and doesn’t change the speed after exiting the roundabout

 

[EVNow]

I just got home after driving a road with two lanes to choose from and, unlike past versions, FSDb chose the right lane. Progress is progressing.

This is a bad change for me in one of the city roads. It's a winding road, so being on the left lane is bad ... Esp. at night. Lots of PB because of the oncoming vehicles and cresting.

Even when I put on the indicator to change to right lane, FSD will change back after a while. I use the road 3/4 times a week - so, not progress

 

[Supcom]

how do I fix a wrong speed limit? There are several places where either on a street is wrong or in a roundabout it is 25 and doesn’t change the speed after exiting the roundabout

Dial the set speed up or down with the right hand scroll wheel.

 

[aronth5]

This is a bad change for me in one of the city roads. It's a winding road, so being on the left lane is bad ... Esp. at night. Lots of PB because of the oncoming vehicles and cresting.

Even when I put on the indicator to change to right lane, FSD will change back after a while. I use the road 3/4 times a week - so, not progress

Are you using the minimal lane change in this driving scenario to stay in the right lane?

 

[JB47394]

Are you using the minimal lane change in this driving scenario to stay in the right lane?

Ooo. Good point. You didn't ask me, but I turned that on for my latest drive, and I absolutely prefer to tell the current FSDb when to change lanes for optional stuff. Now I know why I can stay in the right-hand lane.

 

[mhan00]

good news is that I’m no longer cancelling inane lane changes nearly as much as I used to on versions prior to 11.3.6(?). Bad news is that now my car occasionally takes waaaaaay too long to change into a turn lane while slowing down. This impedes people traveling behind me. It’s not really super dangerous, but it is driving like a jerk, imo. It doesnt help that the car tends to wait until AFTER it has started slowing down in preparation for the lane change into the turn lane to signal, which is not good. The car should be signaling a second or two in advance of slowing down to let the person behind me know what to expect. I’ve been taking over to complete lane changes into turn lanes way more than I have had to in the past.

 

[JHCCAZ]

good news is that I’m no longer cancelling inane lane changes nearly as much as I used to on versions prior to 11.3.6(?). ...

Interesting. For me, I think 11.3.6 has somewhat reduced but not eliminated) the transient false turn signal blinks that mostly occur approaching intersections. But this version displays a strange and very frustrating insistence to change out of the lane it needs to be in, prior to arriving at a real turn laying on the route. I'm guessing it's a mapping issue but I'm not sure.

In places where it happens consistently, it starts trying to exit the proper (leftmost or rightmost) travel about half a mile (2000 to 3000 ft) from the upcoming turn. If I don't completely disengage, I'll have to cancel the attempted lane change two to three times, then once the nav lady announces the turn upcoming in 1,000 ft, I can relax and it will keep the travel lane, enter the turn bay and execute the turn properly.

 

[jebinc]

First time FSD went out of the turn lane near home. There was another vehicle in front in the left turn lane.

BTW, I'm disengaging more often now so that I can send voice reports. Wonder whether it will make any difference....

Unlikely.

 

[Ramphex]

First time FSD went out of the turn lane near home. There was another vehicle in front in the left turn lane.

BTW, I'm disengaging more often now so that I can send voice reports. Wonder whether it will make any difference....

Dang, it’s like watching a son I never had growing up

 

[Jeff N]

Have you ridden in a Tesla with FSDb in the backseat ?

There is a huge difference when you are in the drivers seat and backseat.

I keep giving examples where that guy who uses FSDb for Uber professionally has dozens of videos but you won't believe it. A lot of people don't even know they are in a FSDb Tesla. They think the guy is driving.

No, but I rode in the front passenger seat of a Waymo twice. I hadn’t realized before then that Waymo was allowing people to sit up front with a driverless car. There is a sign on the steering wheel that warns that the car will stop if you try to mess with the steering wheel. Cruise only lets you ride in the back seat.

I did not notice much difference in ride comfort/smoothness between Waymo sitting in the front passenger seat and sitting in the back seat of a Cruise.

It’s true that I haven’t sat in the back seat of a Tesla driven by FSD beta. However, I’m doubtful that it feels that different between front and back.

What I can believe *does* feel different is sitting in the driver’s seat versus being a passenger. As a driver, even if you aren’t actively driving, you can still take control while as a passenger you are at the mercy of someone else’s driving and have no sense of control.

 

[aronth5]

good news is that I’m no longer cancelling inane lane changes nearly as much as I used to on versions prior to 11.3.6(?). Bad news is that now my car occasionally takes waaaaaay too long to change into a turn lane while slowing down. This impedes people traveling behind me. It’s not really super dangerous, but it is driving like a jerk, imo. It doesnt help that the car tends to wait until AFTER it has started slowing down in preparation for the lane change into the turn lane to signal, which is not good. The car should be signaling a second or two in advance of slowing down to let the person behind me know what to expect. I’ve been taking over to complete lane changes into turn lanes way more than I have had to in the past.

This reflects the FSD behavior of turning on blinkers too late especially when someone is behind you. If the car is starting to slow down the blinker should be on so I often turn the blinker on myself to warn the driver behind me.

 

[Ramphex]

iCloud

Sign in to iCloud to access your photos, videos, documents, notes, contacts, and more. Use your Apple ID or create a new account to start using Apple services.

share.icloud.com

Who needs traffic lights lol

 

[EVNow]

Are you using the minimal lane change in this driving scenario to stay in the right lane?

No .... can't remember to do it every drive

But the point is .... there is no reason to change. There are no slow vehicles in the front or anything like that. Just that apparently FSD no longer likes right lanes, even when there are only two and its a city street.