FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Joboo7777]

you're making the assumption that it is being withheld, when it could be something entirely different.
Its entirely possible that 20% of owners have the download set to "Standard" instead of "Advanced"
Could also be the percentage of cars that aren't on wifi very often, or some other reason etc etc.

This is most likely the reason. I know many Tesla owners who simply do not connect to wifi or have Tesla’s located in second homes that are not used very often, etc.

 

[JulienW]

.....there probably is 20% of folks who are clinging to old versions or never connect to wifi.....

As someone who has never had WiFi in almost 4 years I know it is irrelevant to getting in the queue for an update. Notification that you have an update is sent over Cell and you are prompted to connect to a WiFi to download the update. Most people don't know this because update notifications are usually sent out over night while their car is connected to WiFi.

Plus during the early Beta days us without WiFi would get the Beta update downloaded over Cell (got a little spoiled).

Just to add this is a NHTSA recall and recalls MUST be sent over cell if there is no WiFi connection.

Also the Advanced Button doesn't seem to have any discernible effect on the Beta updates.

I have had Advanced On since it was first introduced but I'm still on 69.

 

[zoomer0056]

He passed in between me and the car an my right and my car jerked back and forth after he got to the front right bumper area. FSDb had no idea what to do with that and almost hit him.

That's a good corner case. I can't imagine how an artificial brain can deal with this in a cool and calm manor.

 

[Mardak]

I brake disengaged a couple times this morning for crosswalks. These probably could benefit from some real world reasoning abilities, but plain pattern matching could be sufficient to train and learn for the neural networks. I suppose if one considers pedestrians walking along the sidewalk an edge case, these could be corner cases:

1. Parent walking with 2 kids along a sidewalk past a parked vehicle just before a crosswalk. It was obvious that they were wanting to get to school directly across the street instead of disappearing behind the vehicle.

2. Pedestrian walking towards a roundabout crosswalk where the sidewalk only feeds into the sidewalk instead of continuing along the road. Most likely this person wasn't going to walk through the dirt and plants.

Most likely FSD Beta 11.3.4 would have reacted in time, but everyone involved would probably be feeling pretty uncomfortable.

 

[kabin]

Just got and tried 11.3.4 in the L.A. area with wife as shotgun. We agreed that the update did make significant upgrades to 10.69.

In town we had the usual problems. Wrong lane for turning. There are many places where 3 lanes have cars parked in the right lane all of a sudden. If you are in the right lane, manual driving means this has to be realized way in advance or you will get stuck waiting a LONG time for an opening to move over. FSDb, of course would wait until the last minute. Scary in L.A. Did very well as long we used non-commercial residential areas.

On highway it was very impressive. On a 130 mile drive only had 1 phantom break which was soft and easily corrected. Merging worked fine.The biggest shock was when we were in the left lane, the traffic was moving about 55 mph and a lane splitting motorcyclist was on our right side passing between cars. He was swerving in and out (like they do) and my 2022 MX refresh with USS hit the brakes very hard and jerked the wheel left and right not knowing where to go. I thought the car behind me would hit me but he had kept enough distance. It is expected in SoCal to move over to allow these lane splitters room to go. Idk how Tesla can ever address this? Anyone else had this happen to them?

I would guess Tesla doesn't have enough training data sets to adequately address motorcycle lane splitting. Add to that FSD's lane swerve to maximize lateral distance to adjacent lane trucks and trailers and it's probably only a matter of time. It's one of those things that happen so fast the driver may not have time to make the correct response.

I predict the team plan will be their usual kludge overfit - whenever a motorcycle is seen the FSD overreacts by driving on the opposite lane line. Yuck!

 

[Twiglett]

As someone who has never had WiFi in almost 4 years I know it is irrelevant to getting in the queue for an update. Notification that you have an update is sent over Cell and you are prompted to connect to a WiFi to download the update. Most people don't know this because update notifications are usually sent out over night while their car is connected to WiFi.

Plus during the early Beta days us without WiFi would get the Beta update downloaded over Cell (got a little spoiled).

Just to add this is a NHTSA recall and recalls also MUST sent over cell if there is no WiFi connection.

Also the Advanced Button doesn't seem to have any discernible effect on the Beta updates.

I have had Advanced On since it was first introduced but I'm still on 69.

I'm not sure on the Advanced slider - it seems like everyone who "tests" it is starting from Advanced and already wants the most up to date release, so most of us are wanting something faster than Advanced. You and I have it set on Advanced but you're on 69 and I'm on 11.3.3 - so neither of us is on the release we want
But there does seem to be a subset of owners who don't want immediate updates and prefer to wait, otherwise why would Tesla leave an option in the interface that does not affect anything - I think its more likely that it does have an effect, just not one that we can easily test because of the vagaries of the update process. I'm not willing to wait a year to properly test it though

 

[AlanSubie4Life]

About five seconds after highway engagement, someone mildly cut in and of course the car over-reacted.

Wife: “Is the car driving?” “Why is the car driving?”

Still completely unable to anticipate predictable cut ins and do slowing in advance. Gradually. Though in this case there was so much margin it was a non-issue. Grabbed some TeslaCam, maybe will post it.

Also on interchange 163S to 52W, all seemed to be going well, then it did aggressive regen completely inappropriately ant the curve even with a lead car. That got an even more upset reaction from the VIP.

So it seems that interchanges will still require disengagement.

I don’t understand why they cannot completely prohibit jerky acceleration by rule, except in emergency. Would make it so much more usable.

The dream has not yet been realized.

Several other disengagements to avoid further drama for example with slowing traffic 52W to I-5N. Seems I will have to disengage at all interchange curves of any significance (I would guess at full speed this interchange would have been bad).

But some good smooth lane changes and impressive exit at Nobel Drive, threading into a line of traffic at 0.5miles (seems to be the programmed trigger point).

 

[Mardak]

I think part of the reason why it feels like FSD Beta isn't merging is that it gets really really really close to the new lead vehicle. Here when the merging lane ends, 11.3.4 was following at about ¼ seconds behind the black vehicle:

However it was obvious to me that it was actively following this vehicle as it slowed down to match their 70mph speed instead of speeding up. This pacing started before the dashed lines between the merging lanes disappeared and continued for 15 seconds before the combined lanes reduced to 1 lane width.

This was on Chill where it would normally follow around 1.5 seconds behind the lead vehicle or ~150 feet at 70mph. So dropping to ¼ seconds is about 30 feet, which is the distance between the 10' dashed lines on the interstate.

 

[Dewg]

It is expected in SoCal to move over to allow these lane splitters room to go. Idk how Tesla can ever address this? Anyone else had this happen to them?

California is the only state in the union where lane splitting is allowed. Hopefully not much longer before that unsafe maneuver is abolished here too. They probably don't have it as a high priority to train on this data as the ODD is small.

 

[JulienW]

.....but you're on 69 and I'm on 11.3.3 - so neither of us is on the release we want ...

You know what THEY say: 11.3.3 is a LOT closer (infinitely) to V11 than 10.69.x.

 

[AlanSubie4Life]

part of the reason why it feels like FSD Beta isn't merging is that it gets really really really close to the new lead vehicle.

Also because it waits until the end of the merge lane to merge, even when inappropriate, does not signal, and pulls into inappropriately small gaps (as you outlined). In California it is fine to use all the merge lane if no one is there but earlier opportunities should be safely used when traffic is around, to maintain as long a following distance as possible given the situation.

I don’t have much doubt that it is tracking other vehicles and will merge in behind them, but the merge is totally broken.

Signaling is most definitely required even if it is clear your lane is ending. (Apparently maybe not - if no one is around…but that would be negligent and irresponsible for obvious reasons. Know the Road with the CHP: When a lane ends and you're merging, do you have to signal? | abc30.com )

Your example is just one demonstration of the consequences.

 

[powertoold]

Merging on 3.4 isn't much different than 25.2 (turned off no-confirm lane change) for me. It works reasonably well, better than many drivers.

 

[AlanSubie4Life]

It works reasonably well, better than many drivers.

Lol. Definitely not the metric.

 

[kabin]

California is the only state in the union where lane splitting is allowed. Hopefully not much longer before that unsafe maneuver is abolished here too. They probably don't have it as a high priority to train on this data as the ODD is small.

I would think FSD needs to handle lane splitting cases even if it's not legal. Motorcyclists can be seen lane splitting during freeway rush hours even though it isn't legal in other states.

 

[thewishmaster]

Surface street. Haven’t tried canceling lane changes on the freeway yet

Though today I canceled a couple more lane changes on surface streets and it worked fine. I wonder if there's some cutoff if the car starts moving over where canceling the blinker no longer cancels the lane change.

One v11 improvement for streets with light rail tracks- previous versions would split up the road into 2 lanes - the portion between the right rail and the yellow line and the portion between the right rail and the parked cars on the right, forcing the car into bad lane positioning. 11.3.4 appears to handle this in a more natural way, treating it as a single lane, and the car staying in a reasonable position within the lane.

 

[jabloomf1230]

As someone who has never had WiFi in almost 4 years I know it is irrelevant to getting in the queue for an update. Notification that you have an update is sent over Cell and you are prompted to connect to a WiFi to download the update. Most people don't know this because update notifications are usually sent out over night while their car is connected to WiFi.

Plus during the early Beta days us without WiFi would get the Beta update downloaded over Cell (got a little spoiled).

Just to add this is a NHTSA recall and recalls MUST be sent over cell if there is no WiFi connection.

Also the Advanced Button doesn't seem to have any discernible effect on the Beta updates.

I have had Advanced On since it was first introduced but I'm still on 69.

I suspect that there are many more owners just like you who are on Advanced and do not have 11.3.x. The release was stopped for some unknown reason. It's got nothing to do with the Standard/Advanced setting IMO and I also think that this setting is deprecated altogether.

 

[EVNow]

Lol. Definitely not the metric.

Not the metric, but a metric

 

[EVNow]

Surprise .... yesterday's drive was much better. No jerky lunge to the left to leave space on the right .... even stop & creep were ok.

Almost as if, it either got some extra map data to help with exact stop line location or .... just random behavior. A repeat of better driving today (and a few more days) will confirm either way.

Unsurprisingly - the jerky lunge was back today. But not as bad as the other day.

But - the drive itself was good. Zero disengagements and one intervention to accelerate.

- On protected right turn (using a right turn only lane), a car in front just drove without slowing down ... yet, FSD wanted to "creep" and check before moving. A few seconds of pushing the accelerator needed ...
- On a different protected right turn (using a right turn only lane) at a multi-lane roundabout I've talked about in the past, FSD drive nicely without stopping or slowing too much. On 25.x it would stop or almost stop forcing me to use the accelerator (or disengage).

 

[Dennisis]

FSD wanted to "creep" and check before moving. A few seconds of pushing the accelerator needed ...

Hopefully this early stop and creep behavior will get corrected soon, very annoying. Should be able to stop, look, and smoothly proceed - can shave 3-5 seconds off each turn easily.

 

[Dewg]

I would think FSD needs to handle lane splitting cases even if it's not legal. Motorcyclists can be seen lane splitting during freeway rush hours even though it isn't legal in other states.

Oh I'm not saying they won't handle it. Just lower on the priority list.