FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[zoomer0056]

It does. I’m sure it would stop, but it would be an extremely hard stop, or end up halfway in the middle of the intersection by time the vehicle came to a complete stop.

I don't have FSD. Is there a setting that would essentially define driving style? Like aggressive or sane or mild.

Tongue in cheek:
Other settings could be in a hurry, Sunday drive, road rage, not paying attention, ditz, student driver, grandparent ....

 

[rlsd]

They are posted 55mph roads, and have stop signs where another road bisects said road. Also, there are parts where the speed limits change from 55 to 30, which is also odd, it’s a big change in speed quickly.

Either 11.4.7.2 or 2023 map fixes the speed limit change from 50 to 35 (only about 50 yards of the road) on the 3-lane road in my area.
I got 11.4.5.2 and 2023 map at the same time.

 

[sleepydoc]

Small update, not sure why but I noticed my map data updated after it attempted to download 2023.32.9 but abandoned it. Now I have NA-2023.20

So odd. Has anyone from 32.7 successfully downloaded 32.9?

So does anyone know exactly what the Nav updates do? I’m still on a 2022 nav version and my experience doesn’t seem any different than others’ and may actually be better.

I’ve also seen updates that I would assume to be included in the Nav data occur without any Nav update.

 

[sleepydoc]

Less impressed that it’s still tries “stopping for traffic control“ on a 70mph divided highway when it sees flashing yellow lights.

FSD’s approach to flashing yellow traffic lights seems to be to treat them as ’stop, then proceed if able’ rather than ’stop only if necessary,’ so your description isn’t surprising. My question is why are there flashing yellow lights on a 70 MPH divided highway?

They are posted 55mph roads, and have stop signs where another road bisects said road. Also, there are parts where the speed limits change from 55 to 30, which is also odd, it’s a big change in speed quickly.

Many rural highways are like this.

 

[jebinc]

The 2023 map update did not fix one rural road near me that has a few stop signs. Car just wants to blow through them at 60. I rarely travel this road, maybe twice a year, maybe it will be fixed the next go around.

Many rural roads here as well. Many a blown stop sign as well

So does anyone know exactly what the Nav updates do? I’m still on a 2022 nav version and my experience doesn’t seem any different than others’ and may actually be better.

Nope. Just a lot of wild speculation and placebo effect.

 

[JHCCAZ]

For me, on my second MY, the wipers have always performed acceptably in the rain. However now I have a problem. Sometimes, but not always, when I start to drive out of my garage they will make two passes and stop. The windscreen is totally dry (Arizona, no rain in memory!). Any suggestions?

I've been complaining about the dry wipes for a long time. These were fixed on an earlier version of FSDb 11, in that they would give a spray on the first wipe of the drive. So even if the wipe is unnecessary, at least it isn't grinding desert dust into the windshield.

Unfortunately, this fix has been lost with 11.4.7 or maybe a bit before. There are also more system-initiated wipes, as you say often at the very beginning of the drive when the sun hits the windshield.

Another indication that this is not being handled well in a software, is that the dry wiping activity also happens while the car is in Park! Pretty clearly, this code needs some attention.

My suggestion to you in the meantime: try to remember to give a cleaning wash wipe cycle before you pull into your garage, or immediately when you get in the car before you start, or otherwise manually use a sponge handle / squeegee like they have at the gas station, to get the abrasive dust off before the car has a chance to dry-wipe it.

 

[Supcom]

FSD’s approach to flashing yellow traffic lights seems to be to treat them as ’stop, then proceed if able’ rather than ’stop only if necessary,’ so your description isn’t surprising. My question is why are there flashing yellow lights on a 70 MPH divided highway?

Texas highway 24 has flashing lights above it's intersection with TX-19 between Commerce and Paris. This is a 75 mph divided highway. But, it is not a limited access highway so there are many surface level intersections. The flashing lights at this intersection serve only to alert drivers to the intersection, which is likely to have cross, or merging traffic. However, vehicles on TX-24 have right-of-way and generally do not need to slow down. The flashing lights are only an alerting mechanism.

 

[sleepydoc]

Texas highway 24 has flashing lights above its intersection with TX-19 between Commerce and Paris. This is a 75 mph divided highway. But, it is not a limited access highway so there are many surface level intersections. The flashing lights at this intersection serve only to alert drivers to the intersection, which is likely to have cross, or merging traffic. However, vehicles on TX-24 have right-of-way and generally do not need to slow down. The flashing lights are only an alerting mechanism.

Having a 75 MPH highway with stopped traffic needing to cross or merge doesn’t seem like the best idea, but that’s TXDOT’s decision.

FSD needs to fix its handling of yield situations in general. Currently it tends to treat many/most of them as ‘stop-check-proceed’ rather than ‘check and only stop if necessary.’ Otherwise it’s going to view that as a controlled intersection with a flashing yellow light rather than the warning that it is. (I’m also curious as to why they don’t just have a flashing yellow light over a sign at the side of the road)

 

[Supcom]

Having a 75 MPH highway with stopped traffic needing to cross or merge doesn’t seem like the best idea, but that’s TXDOT’s decision.

FSD needs to fix its handling of yield situations in general. Currently it tends to treat many/most of them as ‘stop-check-proceed’ rather than ‘check and only stop if necessary.’ Otherwise it’s going to view that as a controlled intersection with a flashing yellow light rather than the warning that it is. (I’m also curious as to why they don’t just have a flashing yellow light over a sign at the side of the road)

Yields are slow because FSDb still has horrendous lag. This results in a yield becoming a stop because FSDb cannot determine that the way is clear quickly enough so has to stop until the determination can be made.

 

[arnolddeleon]

Yields are slow because FSDb still has horrendous lag. This results in a yield becoming a stop because FSDb cannot determine that the way is clear quickly enough so has to stop until the determination can be made.

Latency through that system is my guess for a top source of issues. I see FSD "twitch" for a pedestrian or bicyclist long after the event/situation where action was either no longer warranted or it was going to be too late anyway. As humans, we can see something late and probably "skip" the reaction because we can infer that it was not going to make a difference in the outcome.

Annoying long standing "habit" is FSD will happily drive alongside another car even if that car is positioned a little close to our lane. I would speed up or slow down if can. If I can't do that, I will at least adjust my lane position to give more room assuming I have the space on the other side.

As with previous releases the behavior seems to be evolving with presumably only map updates. I don't know how much the navigation data reflects what FSD is driving on but in this image:

New item by Arnold de Leon

photos.app.goo.gl

it shows a slight jog to right to make a left turn. The left turn lane is to the unsurprisingly to the left. I don't know if this contributes to the wrong lane selection. Is this just bad map data? This kind of error seems common.

 

[rlsd]

Please go to the poll:

FSD 11.4.x bug fix priority poll

Which 3 FSD bugs you want to be fixed in the next FSD V11?

teslamotorsclub.com

 

[Supcom]

Latency through that system is my guess for a top source of issues. I see FSD "twitch" for a pedestrian or bicyclist long after the event/situation where action was either no longer warranted or it was going to be too late anyway. As humans, we can see something late and probably "skip" the reaction because we can infer that it was not going to make a difference in the outcome.

Annoying long standing "habit" is FSD will happily drive alongside another car even if that car is positioned a little close to our lane. I would speed up or slow down if can. If I can't do that, I will at least adjust my lane position to give more room assuming I have the space on the other side.

As with previous releases the behavior seems to be evolving with presumably only map updates. I don't know how much the navigation data reflects what FSD is driving on but in this image:

New item by Arnold de Leon

photos.app.goo.gl

it shows a slight jog to right to make a left turn. The left turn lane is to the unsurprisingly to the left. I don't know if this contributes to the wrong lane selection. Is this just bad map data? This kind of error seems common.

It is common for FSDb to hard brake AFTER a crossing car has cleared the lane even though ego isn't even close to the crossing car. This is the most obvious indication of unacceptable latency that I see. It's so predictable that I automatically press the accelerator as soon as a car starts across my path. I suspect that it's a major factor in Tesla's willingness to chuck away years of work to pursue end-to-end AI on V12.

 

[heapmaster]

They are posted 55mph roads, and have stop signs where another road bisects said road. Also, there are parts where the speed limits change from 55 to 30, which is also odd, it’s a big change in speed quickly.

Do the roads have stop sign ahead signs so even a human would know there is a stop coming at that speed? If it does and FSD could read those then it wouldnt have to rely on map updates, like it should be reading the "no right on red" signs. You would think this would be so simple having OCR for the last 20 years, just add it in at the top level of the control code.

 

[heapmaster]

Latency through that system is my guess for a top source of issues. I see FSD "twitch" for a pedestrian or bicyclist long after the event/situation where action was either no longer warranted or it was going to be too late anyway. As humans, we can see something late and probably "skip" the reaction because we can infer that it was not going to make a difference in the outcome.

Annoying long standing "habit" is FSD will happily drive alongside another car even if that car is positioned a little close to our lane. I would speed up or slow down if can. If I can't do that, I will at least adjust my lane position to give more room assuming I have the space on the other side.

As with previous releases the behavior seems to be evolving with presumably only map updates. I don't know how much the navigation data reflects what FSD is driving on but in this image:

New item by Arnold de Leon

photos.app.goo.gl

it shows a slight jog to right to make a left turn. The left turn lane is to the unsurprisingly to the left. I don't know if this contributes to the wrong lane selection. Is this just bad map data? This kind of error seems common.

My car when the car to either side is to close to my lane will hit the breaks on city streets and a few times on freeway at full speed. Another thing that needs to be fixed, I thought it would just move over a bit, not slow down to match their speed and stay behind them even when they are not in our lane.

 

[sleepydoc]

Annoying long standing "habit" is FSD will happily drive alongside another car even if that car is positioned a little close to our lane. I would speed up or slow down if can. If I can't do that, I will at least adjust my lane position to give more room assuming I have the space on the other side.

Agreed. Another bad habit is that it has no qualms about hanging out in another driver’s blind spot. Even with the prevalence of blind spot monitoring systems that’s poor practice. I teach my kids to either speed up or slow down so the other car will not inadvertently merge into them.

 

[scottf200]

It is common for FSDb to hard brake AFTER a crossing car has cleared the lane even though ego isn't even close to the crossing car. This is the most obvious indication of unacceptable latency that I see. It's so predictable that I automatically press the accelerator as soon as a car starts across my path. I suspect that it's a major factor in Tesla's willingness to chuck away years of work to pursue end-to-end AI on V12.

FYI, this has gotten noticeably better for me over the several OTAs. It used to be predictable for me to. I would hover over the accelerator but it has gotten to a point where I think about it but don't physically do anything and just wait.

 

[dgatwood]

It is common for FSDb to hard brake AFTER a crossing car has cleared the lane even though ego isn't even close to the crossing car. This is the most obvious indication of unacceptable latency that I see. It's so predictable that I automatically press the accelerator as soon as a car starts across my path. I suspect that it's a major factor in Tesla's willingness to chuck away years of work to pursue end-to-end AI on V12.

It was amazingly common with highway autopilot, too. It has gotten a lot better, but it is still reacting, just not as strongly. By the time it gets around to reacting, it shouldn't even be reacting anymore. I can't imagine the pipeline latency still being measured in seconds in HW3, so I feel like at this point it must be something else, like persistence failing to recognize that the vehicle is genuinely gone and not just invisible, or something like that.

 

[mtndrew1]

The new maps fixed an issue on a back road in the Antelope Valley of Southern California where FSDb used to slam on the brakes in a 55 MPH zone down to an erroneous 25 MPH limit, then veer hard right off the road.

The maps also fixed a lane choice across a very weird intersection near my house.

Still makes a dozen other significant errors on both routes but progress is progress.

 

[Ramphex]

I don't have FSD. Is there a setting that would essentially define driving style? Like aggressive or sane or mild.

Tongue in cheek:
Other settings could be in a hurry, Sunday drive, road rage, not paying attention, ditz, student driver, grandparent ....

The latter summarizes FSD’s general driving styles.

 

[Pdubs]

For me, on my second MY, the wipers have always performed acceptably in the rain. However now I have a problem. Sometimes, but not always, when I start to drive out of my garage they will make two passes and stop. The windscreen is totally dry (Arizona, no rain in memory!). Any suggestions?

Park outside?