FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[sleepydoc]

The curve under the overpass is slow to negotiate. Listen to the thump, thump, thump as the tires run over the edge bumper/reflectors. That's a good warning for non-attentive drivers but not something anyone should expect for FSD.

Let me see if I can wrap the irony of those thoughts. It some how makes sense for FSD to shift within a lane to maximize distance from a safely driven truck in an adjacent lane and yet be unable to maintain a safe distance from the road edge in a turn with stationary fixed concrete obstacles at near freeway speeds. Nope, that doesn't come close to passing the sniff test.

There's nothing wrong with critiquing the slop of an effort that's been FSD.

I fail to see the issue with that curve. The car slowed from about 58 to 55 MPH and it was not perfectly centered in the lane, but neither of those are major issues. Human drivers routinely drift towards or even over the yellow lines when negotiating curves to flatten them out and reduce the lateral forces.

 

[sleepydoc]

I'm going to venture a guess that 2023.2.12 contains at lest some of the fixes NHTSA is requesting since it seems like Tesla is just flooding download with it at the moment. If that's the case then I guess I would expect the release of V11 broadly to be based on that release.

Since all of the items the NHTSA listed were known bugs that Tesla was at least theoretically working on I think that’s A pretty safe guess.

 

[EVNow]

In this case, FSDb’s tendency to reduce the speed too slowly would actually be a benefit. Areas where the expectation is to break the law pose a quandary for FSD. On one hand it’s probably safer not to follow the law, on the other hand programming the system to intentionally break the law isn’t good, either.

The solution would be to add a flag which lets the driver control whether to drive at traffic speed or at speed limit (+/- offset).

 

[Mardak]

write code that predicts lateral gs and then adjusts speed accordingly. It’s just math. With that being said, why has it taken so long for the engineers at Tesla to fix this issue? It’s just such a simple issue.

Tesla does that math calculating lateral acceleration and jerk along with other things that feed into the total cost function. Tesla is likely heavily weighting safety such as collision risk with another vehicle or vulnerable road user much higher than comfort and timeliness, and based on Tesla's progress of 10.x and 11.x so far, it seems like the first priority is safety.

Here's AI Day 2021's "Optimizing for a Safe, Smooth & Fast Path" example:

So the trapezoidal S-curve in the top right lateral acceleration green line reflects a slightly negative value for slowly turning right to get into the left turn lane (instead of suddenly shifting left then back right strictly following the yellow road lines) then the larger positive curve with a maximum value around 3m/s² for the actual left turn. (For reference, that acceleration is 0.3g while Safety Score tracks turns that are at least 0.2g and negatively impacts the score when exceeding 0.4g.)

In this 11.3 example, it seems like FSD Beta did not notice or care about the raised yellow reflectors resulting in the thumps. Maybe the bright sunlight contrasted with overpass shadow did make it harder for the subsequent left-curving road to be identified early enough to avoid weaving as it somewhat seems like it was expecting a continuous right-curving exit. Maybe some future FSD Beta 11.x driving profile will be able to adjust the cost function to prefer smoother driving than speed as in the AI Day screenshot above, it seems to have worked quite hard to reduce traversal time from 14s to 10s even after collision risk was very low, so there could have been an even smoother path by giving up say 1 second.

 

[EVNow]

Human drivers routinely drift towards or even over the yellow lines ...

All the time.

I've plenty of videos of other cars regularly crossing lines when driving. When the roads are "empty" (i.e. low probability of collisions), humans don't care much about lane lines.

 

[jabloomf1230]

All the time.

I've plenty of videos of other cars regularly crossing lines when driving. When the roads are "empty" (i.e. low probability of collisions), humans don't care much about lane lines.

One thing that I noticed when I bought my Tesla is that most expressway drivers weave all around within their lane (and sometimes over the lines). This is noticeable because Tesla's drive in the exact lane center on highways.

 

[TRWhit]

Right? I became acutely aware of that too. Aware of the fact that I usually drive much closer to outside of the lane than the Tesla. It was unnerving at first.

 

[Gbills]

The solution would be to add a flag which lets the driver control whether to drive at traffic speed or at speed limit (+/- offset).

Let’s have a flag on the outside of the car as well.

 

[2101Guy]

Since this version is the latest/greatest/much improved...is it safe to assume the disclaimer of "will do the WORST thing at the WORST time", has disappeared?

Or is that STILL the case..even with this version?

 

[ddeflyer]

Since this version is the latest/greatest/much improved...is it safe to assume the disclaimer of "will do the WORST thing at the WORST time", has disappeared?

Or is that STILL the case..even with this version?

While it is classed as a driving assist you should always assume "will do the WORST thing at the WORST time"

 

[Twiglett]

Is it too to point out to Teslascope that almost 55% of cars are now on a version that currently has no avenue to FSDb
That could be wrong tomorrow, or March 2nd when they release V11 as part of 2023.6
We can live in hope
I say that but I live in Texas

 

[Mardak]

This obviously isn't for Tesla, but ChatGPT

Here's a combination of both. I suppose the last FSD Beta release notes were for 10.69.x and 11(.0) from before late November's launch of ChatGPT, so here's its attempt at simply explaining 11.3 by topic:

​

FSD Beta v11.3 has made some improvements to how the car drives on highways. In the past, there was a separate system for driving on highways, but now the same system is used for both on and off-highway driving. This new system uses more cameras and is smarter, so it can handle more complex driving situations. For example, it can drive through construction zones where the lanes are blocked, and it can navigate sharp curves on the highway better.​

​

The update has also added a long-range highway lanes network, which helps the car detect blocked lanes and curves from farther away. Additionally, when driving at high speeds or on sharp curves, the car will now move closer to the inner lane lines to help it stay centered and drive more smoothly.​

​

--​

​

The new Tesla software update, FSD Beta v11.3, has made several improvements to make driving safer.​

• One of the improvements is to the Automatic Emergency Braking (AEB) feature. AEB now works better to stop the car if another car runs a red light or turns in front of it. This improvement should help prevent about half of the accidents that happen in these situations. This feature works both when driving manually and when using Autopilot.
• Another improvement is to the Autopilot's reaction time when it comes to red lights and stop signs. It now reacts 500 milliseconds faster than before by using the object's speed and trajectory estimates.
• The software update also improves the lane guidance feature by making it better at detecting lines on the road and edges of the road. This helps the car stay in its lane and know where the road ends. The update also makes the feature better at predicting when other lanes cross over, reducing the likelihood of accidents caused by unexpected lane changes or collisions with oncoming traffic.

Overall, these updates should help Tesla cars drive more safely and avoid accidents.​

​

--​

​

The new version 11.3 of Tesla's Full Self-Driving (FSD) software includes several improvements to the way the car drives itself. Here are some examples:​

1. The car can now change lanes more smoothly and with better timing. For example, it can detect when it needs to change lanes at the same time as another car, and make the change without causing a problem.
2. The car can also follow other cars more smoothly, by taking into account the way they brake. For example, if the car in front of you suddenly slows down, the Tesla will be able to adjust its speed more smoothly, rather than jerking to a stop.
3. The car can now also make better decisions at crosswalks, by using a more advanced method of predicting where it needs to go. This means that the car will be better at stopping for pedestrians and other obstacles.
4. The car's ability to merge onto highways and other roads has also been improved. It now uses a new method of determining the best path to take, based on the layout of the lanes, rather than relying on an older, less reliable method.

--​

​

The v11.3 update for Tesla's autonomous driving system includes several improvements to the datasets that the car uses to "see" and understand the world around it.​

• One improvement involves making the car's predictions about where it will go more accurate and faster by improving the dataset with better optimization, more data, and better design. This means that the car will be able to plan its movements more precisely and quickly.
• Another improvement involves teaching the car to better recognize objects in challenging situations like when it's raining or there is debris on the road. This means that the car will be better equipped to handle unexpected obstacles in its path.
• The update also includes improvements to how the car handles situations where other cars cut in front of it. By adding more examples of this scenario to the dataset and improving how the car models the motion of other cars, it will be able to respond more smoothly and predictably when other cars cut in front of it.
• Other improvements include better detection of rare objects, like a small animal crossing the road, and better recognition of specific types of vehicles, like school buses and cars that are starting to move from a standstill. These improvements are made by adding more accurate and detailed data to the dataset.

All of these improvements mean that the car will be better able to navigate and react to its surroundings, making for a safer and more comfortable ride.​

​

--​

​

Aside from the driving improvements, v11.3 also includes some non-driving changes that are aimed at making the Autopilot system better. For instance, there's a new feature called voice drive-notes that lets you record a message after Autopilot has to stop or you have to take over control. This is a way for Tesla to gather feedback from drivers and improve the system.​

​

Another change is that longer fleet telemetry clips are now available, which means Tesla can analyze more data to make the system better. This is achieved by balancing compressed IPC buffers and optimized write scheduling across twin SOCs. Think of it like a security camera that captures footage for longer periods of time - this is like Tesla's way of capturing more information about how the Autopilot system is working in different situations.​

 

[2101Guy]

While it is classed as a driving assist you should always assume "will do the WORST thing at the WORST time"

Ok. So I’ll take that as a yes…despite all of the promised improvements at this point with version 11 and single stack and advanced code, etc.

It STILL..as stated by Tesla… will do the worst possible thing at the worst possible time. For $15k.

 

[kriskros91]

Teslascope seems to have a source that is feeding them some interesting information. Such as EAP is getting single stack very soon. I just don't understand this exactly. It doesn't seem that there is much fine manual control left based on the 11.3 videos from the insider publishing videos on YouTube/Twitter.

I'm certain I'm not alone in the fact that I don't always like highway AP lane selection at times and just sometimes don't want it to change lanes at all until I manually change lanes. I think we'll have to set alternative profiles without FSD on at all to keep this functionality.

 

[JulienW]

Teslascope seems to have a source that is feeding them some interesting information. Such as EAP is getting single stack very soon......

According to a reliable source (Elon) we were getting V11 "one Stack to rule them all" in January......January of 2022. So let me know what "very soon" on a Tesla timeline means.

 

[Dewg]

Ok. So I’ll take that as a yes…despite all of the promised improvements at this point with version 11 and single stack and advanced code, etc.

It STILL..as stated by Tesla… will do the worst possible thing at the worst possible time. For $15k.

Yup. And hundreds of thousands of Tesla owners are enjoying their purchase. Bummer a few aren't. I guess that's true for almost any product, or we wouldn't have the term "buyer's remorse".

 

[sleepydoc]

One thing that I noticed when I bought my Tesla is that most expressway drivers weave all around within their lane (and sometimes over the lines). This is noticeable because Tesla's drive in the exact lane center on highways.

Especially when they're on their phones!

 

[aronth5]

Since this version is the latest/greatest/much improved...is it safe to assume the disclaimer of "will do the WORST thing at the WORST time", has disappeared?

Or is that STILL the case..even with this version?

Since the single stack IMO affects highway driving more I suspect we will still see a lot of "Worst thing at the Worst time" events on city/streets.

 

[Gbills]

Ok. So I’ll take that as a yes…despite all of the promised improvements at this point with version 11 and single stack and advanced code, etc.

It STILL..as stated by Tesla… will do the worst possible thing at the worst possible time. For $15k.

I’ll buy a vowel (E) for $ 6k

 

[legendsk]

I'm a little disappointed. I didn't find anything in the release notes about fixing the fart noise. It just isn't quite ... . . . . . . . moist enough? And I'm a bit concerned that this time the one step backward will be some minor inconvenience like, Reverse doesn't work any more, except on Tuesdays and only if it's raining. Fortunately, I've learned to not bother to try to talk to service about it. Just figure out how to work around it and keep going.