FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Ravioli]

Getting back to 11.x - Lots of drives yesterday, mostly ok. But on one drive lots of lane selection issues (slow traffic blocking earth movers !).

For me now the top two issues are Roundabouts and planning / lane selection. Fix those and my disengagement rate will go down 5x.

My experience on local roads is very similar. How is your car doing on roundabouts when there is little traffic? Mine can handle 1 lane ones well. It starts to complete 2 lane ones as well but with an occasional weird lane change before or at the roundabout.

 

[EVNow]

Have seen this many times with 11.3.6 as most of the roads in neighborhoods around here are unmarked.

We have unmarked roads here too (like I guess most neighborhoods in US) - but FSD doesn't have that problem. I think there is a particular width of the road above which it can handle. Narrower than that cut off, it will stop. We'd slow down - but not stop.

 

[Twiglett]

We have unmarked roads here too (like I guess most neighborhoods in US) - but FSD doesn't have that problem. I think there is a particular width of the road above which it can handle. Narrower than that cut off, it will stop. We'd slow down - but not stop.

yes that seems right, it does seem better on the narrower roads that have edge markers but no center line. When it has to imply both edges and center it is much more unsure. Give it a similarly narrow road, but add road markings and it will do fine.
Overall I've been impressed by 11.3.6, its just that they manage to fix some annoying things while managing to add new ones.
Many fewer phantom braking events but those have now been replaced by ridiculously timed lane changes

 

[EVNow]

Checked it out, interesting that DreamDrive Pro is included with flagship models and lower end models come with DreamDrive and Pro is available as an OTA update.

Oh and no CEO spewing false timelines lol

Lucid Air EV Will Have Lidar and Driver-Assist Systems on Board

The upcoming EV sedan will have driver monitoring and can be updated for future Level 3 semi-autonomous driving capability.

www.caranddriver.com

The Air will launch with less tech than what Tesla currently offers, but within a year it will likely be on par with Cadillac’s Super Cruise hands-free system and within three years, will have Level 3 hands off and eyes-off capabilities, which no automaker currently offers.​

​

ps : Now the question is will Lucid still be around in 3 years ...

 

[totoro722]

Lucid Air EV Will Have Lidar and Driver-Assist Systems on Board

The upcoming EV sedan will have driver monitoring and can be updated for future Level 3 semi-autonomous driving capability.

www.caranddriver.com

The Air will launch with less tech than what Tesla currently offers, but within a year it will likely be on par with Cadillac’s Super Cruise hands-free system and within three years, will have Level 3 hands off and eyes-off capabilities, which no automaker currently offers.​

​

ps : Now the question is will Lucid still be around in 3 years ...

I notice that article was from 2020, 3 years ago. So where is the level 3 they promised?

 

[sleepydoc]

My experience on local roads is very similar. How is your car doing on roundabouts when there is little traffic? Mine can handle 1 lane ones well. It starts to complete 2 lane ones as well but with an occasional weird lane change before or at the roundabout.

IME, 11.4 still doesn't handle yields well. It tends to stop completely, whether necessary or not, then will proceed about half the time.

 

[uscbucsfan]

IME, 11.4 still doesn't handle yields well. It tends to stop completely, whether necessary or not, then will proceed about half the time.

You have 11.4?

How is it?

 

[JulienW]

Checked it out, interesting that DreamDrive Pro is included with flagship models and lower end models come with DreamDrive and Pro is available as an OTA update.

Oh and no CEO spewing false timelines lol

OT: Here is a former Tesla owner that bought an early Air. He talks about how the system and how limited it still is. TS 7:15

 

[Joboo7777]

Oh Boy feeling alot of negative vibes about FSDb!

On a positive note, Aside from the strike I received on my way back
From Iowa City in route to Dallas There was a good hour when I was traveling from Iowa city to Des Moines on Interstate 80 at around 9pm with significant rain. Not fun. So much so that I would have pulled over if it weren’t for FSDb. Although I received FSDb degraded message the car kept me in the lane and far enough away from cars in front of me. It certainly felt like the car was able to see better than I was at that time and It was a huge help! A little further along in the trip just before stopping at a hotel south of Kansas City, I was getting alittle tired. Going through Kansas City at 2am in the morning was a breeze with FSDb. There were so many interchanges and quick exits that if I didn’t have FSDb there is no doubt I would have made a wrong turn or exit.

It’s not perfect, but it was extremely helpful in keeping me safe during these more difficult and stressful situations.

 

[Tronguy]

yes that seems right, it does seem better on the narrower roads that have edge markers but no center line. When it has to imply both edges and center it is much more unsure. Give it a similarly narrow road, but add road markings and it will do fine.
Overall I've been impressed by 11.3.6, its just that they manage to fix some annoying things while managing to add new ones.
Many fewer phantom braking events but those have now been replaced by ridiculously timed lane changes

Took a drive to a State Park yesterday for a nice walk over hill and dale, about 30 miles from home. Since it was just me, decided to see how well 11.3.6 would run.

Interestingly, about 2/3 of the way there, NAV decided to turn off the 50 mph, yellow stripe marked, relatively fast road onto a much shorter path that went very much up hill and down dale, sans any stripes at all, and narrow. In two spots there were signs up warning one of one lane bridges (!), one of which was pretty nasty: What used to be an old water-driven mill that was right up against the road, with a sharp right (with no view) of oncoming traffic. Whee!

In any case, this has been the first time since 10.69 that I've had the car on steep, unmarked roads that have very definite crests over which one can't see oncoming traffic. On 10.69 FSD-b had the unfortunate habit of driving straight down the middle of the road while approaching a crest on a road like this. On 11.3.6, I'm happy to report that the car tended to hang somewhat over to the right when driving down these roads, especially when approaching a crest. So, glad to report that at least one thing got fixed.

As it happened, there wasn't too much traffic. (Maybe the signs that said, "Impassible for trucks" had something to do with that? ). At one point, though, there was an oncoming car on one of the straights. There was room for two to pass (remember, no center road stripes at all), but FSD-b didn't like the looks of things and came to a halt as the other car passed, then picked up speed and went on.

On the way to the park, at those one-lane bridges, the car went over the bridge without too much trouble (as it happens, there's a pretty good view of any oncoming traffic), but, at that old mill, came to a halt, then snuck around the building to the right before continuing on. Not completely out of line to do so, given the lack of visibility. On the way back, as the car swung wide right around the building, FSD-b clearly couldn't stand the country road vibe and went into the "Take over immediately!" panic.

At one point the road went from asphalt to gravel. On the way to the park there was a fork, with the main road going to the left and another road starting up on the right. Despite the map being clear about what was happening, the gravel, lack of boundaries, and extreme lack of signage fooled the car into taking the wrong fork, from which it didn't recovery. Took over, told the verbal bit that the car couldn't handle a fork in the road, and continued on.

All in all, not to bad for a Beta product. 25 miles or so each way, combination of limited access highway, on ramps, off ramps, local roads going from one lane forward to two and back to one, unprotected lefts, unprotected rights, traffic lights, and all that. The only real weirdnesses was running around in the woods on one lane roads, about three interventions each way, pretty much in the woods.

 

[jebinc]

When I bought my MX in 2020, I paid what I thought was a lot for a product that was not developed yet; I wanted to be part of its growth and see what happened. Since then I have experienced a lot of versions of FSD that were interesting but unusable. Recently on v11 I have found I can drive 50 miles each way to work in SoCal with few disengagements, and those mostly for lane changes in heavy traffic. Now I am back in Oregon, and find I can turn on FSDb 11.3.6 and drive 50 or more miles on Hwy 101 through towns, windy roads, etc. with no disengagements and very rare interventions. Last year this was impossible as there were numerous places along my routes that would consistently fail in a predictable way. Today all of those failure places no longer cause a problem for FSD. I find that I have now pretty close to what I was hoping for when I ponied up three years ago. The idea of roboTaxi has no appeal for me, but this is good. It is pretty close to set it and forget it. I understand some here have different experiences, but on this first gen Model X FSD is working well

Another X/S data point…

 

[jebinc]

I think Lucid cars are hideous. I would consider Rivian if I needed a truck

I did…

 

[sleepydoc]

You have 11.4?

How is it?

sorry - typo. I'm on 11.3(.6)

 

[oliphant428]

New version rolling out internally. Based on 2023.7.X

https://twitter.com/x/status/1656388766368677893

 

[jabloomf1230]

New version rolling out internally. Based on 2023.7.X

https://twitter.com/x/status/1656388766368677893

Interesting. Other than factory firmwares, Tesla software has not recently had a version for odd numbered weeks of the year (such as week 7).

 

[JulienW]

Interesting. Other than factory firmwares, Tesla software has not recently had a version for odd numbered weeks of the year (such as week 7).

Hopefully just an internal testing place holder version (11.4.1??) and when 11.4.x is released it will be on 23.12.x. Although it is odd (figuratively and literally) and seems a 23.6.16 would have been just as easy.

 

[Dewg]

I did…

View attachment 936484

How do you like the Rivian?

I was curious so I looked up a comparison thread on Rivian's forums - do you agree with their assessments?

https://www.rivianforums.com/forum/threads/driver-highway-assist-vs-tesla-autopilot-an-extensive-owners-review.11518/

 

[willow_hiller]

v11.4.1 release notes. At first glance they do appear different from v11.4:

https://twitter.com/x/status/1656396186532839424

FSD Beta v11.4.1

Autopilot Improvements

- Improved control through turns, and smoothness in general, by improving geometry, curvature, position, type and topology of lanes, lines, road edges, and restricted space. Among other improvements, the perception of lanes in city streets improved by 36%, forks improved by 44%, merges improved by 27% and turns improved by 16%, due to a bigger and cleaner training set and updated lane-guidance module.

- Added lane-guidance inputs to the Occupancy Network to improve detections of long-range roadway features, resulting in a 16% reduction in false negative median detections.

- Improved ego's assertiveness for crossing pedestrians in cases where ego can easily and safely cross before the pedestrian.

- Improved motorbike recall by 8% and increased vehicle detection precision to reduce false positive detections. These models also add more robustness to variance in vision frame-rate.

- Reduced interventions caused by other vehicles cutting into ego's lane by 43%. This was accomplished by creating a framework to probabilistically anticipate objects that may cut into ego's lane and proactively offset and/or adjust speed to position ego optimally for these futures.

- Improved cut-in control by reducing lane-centric velocity error by 40-50% for close-by vehicles.

- Improved recall for object partial lane encroachment by 20%, high yaw-rate cut-in by 40%, and cut-out by 26% by using additional features of the lane-change trajectory to improve supervision.

- Reduced highway false slowdowns related to underestimated velocities for faraway objects by adding 68K videos to the training set with improved auto-labeled ground truth.

- Smoothed in-lane offsetting for large vehicles by tuning the amount of lateral jerk allowed for the maneuver.

- Improved lateral control for upcoming high-curvature merges to bias away from the merging lane.

 

[jebinc]

How do you like the Rivian?

I was curious so I looked up a comparison thread on Rivian's forums - do you agree with their assessments?

https://www.rivianforums.com/forum/threads/driver-highway-assist-vs-tesla-autopilot-an-extensive-owners-review.11518/

I received the Rivian last October, and often walk past the MS Plaid when needing to go somewhere. If I was forced down to one vehicle, it would likely be the Rivian that stays (more versatile with the bed, tonneau, and gear tunnel).

Re the two autopilots, Telsa is farther ahead, but I don’t dislike the Rivian system. Like Telsa in the early days, it’s always improving. I do have less “anxiety” using the Rivian system, vs FSD(j) on the highway.

If you need a truck, I would recommend the R1T over the Lightning and the R1S over the Model X, for a full size SUV.

 

[Dewg]

I received the Rivian last October, and often walk past the MS Plaid when needing to go somewhere. If I was forced down to one vehicle, it would likely be the Rivian that stays (more versatile with the bed, tonneau, and gear tunnel.

Re the two autopilots, Telsa is farther ahead, but I don’t dislike the Rivian system. Like Telsa in the early days, it’s always improving. I do have less “anxiety” using the Rivian system, vs FSD(j) on the highway.

If you need a truck, I would recommend the R1T over the Lightning and the R1S over the Model X, for a full size SUV.

You don't have all the weird stop/starts on highways/freeways? I wonder if it's location based - like people in SoCal having a better experience with Tesla AP/FSD Beta?

HA starts and stops. A lot. It stops at most highway splits. It cut off multiple times telling me a tollbooth was ahead when it wasn’t. It cut off telling me the lanes weren’t marked, the road was narrowing, and the camera was blurred. Every time it cuts off, alerts you to take over you either have to click the drive stalk up turning off both HA and AC or wrest control away (more on this in a moment). Or in some cases, like when it determines that the camera is blurred, it just stops. With the cruise set at 77, by the time it cuts out, alerts you, and you look down, your speed has already dropped to 72 and you have to go back to the accelerator to regain speed. There’s no traditional cruise control, it’s AC only so when the camera decides it’s done, you’re completely without cruise control at all. And once it kicks off, it’s at least 30-60 seconds minimum before it’s available again. The availability and consistency of AP are big wins for Tesla over Rivian’s HA.