FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[AlanSubie4Life]

Regression in Chuck Cook's UPL

Way better success rate though!

If you use a very low standard with no failures due to excessive wait time, I think it was 4/6! Much better than ~1/8 (best possible grading on prior version with free passes was 3/8 before).

Stopping in middle of the road not ideal though. To be fair, it was doing this before, effectively (it would proceed across the road at 9-12mph, which is of course stopped).

FSD beta has literally never worked on ULTs. Way better than not even attempting them (prior state of the art), but it is still broken. No changes recently. Same old broken results.

People like to pretend that somehow it worked at some point but they were just fooling themselves. It’ll make an effort to cross the road without being hit, of course, but it just isn’t functional in any normal sense of the word. Fortunately all of Chuck’s videos prove it.

 

[jebinc]

No update here.

I’m on the banned list, probably, which is just fine for now.

 

[rlsd]

Does anyone know how TeslaFi and Teslascope get the update numbers?

 

[jebinc]

Does anyone know how TeslaFi and Teslascope get the update numbers?

Their enrolled customer’s cars report into the system. I use TeslaMate (running 24x7 on a Raspberry Pi 4 8GB) and self host locally. No need to link your car to an unknown third party database in the cloud. YMMV.

 

[Yippydog]

Totally expecting to hear this, this week. I treat Elon kind of like a dead beat dad. You know the kind that says they'll totally be at your sports game or show up for the science fair or whatever but never does and always has an excuse. It's always, 'work ran late today,' ("haha guys, solving the ai problem engineers have been struggling decades to solve ended up being hard, who knew, lul") but really he was out cheating on your mom with a mistress (twitter) but you still love him because he's your dad (of tesla), but eventually the lies grow so big that you finally work up the courage to confront him and instead of him taking responsibility to do better he cripples hw3 by removing radar and uss and blames you for his failed marriage because you didn't love him enough and how dare you hold him responsible for his past actions and words after everything he's done for you and hw4 is around the corner and it's totally going to solve his marriage problems. Then he excuses himself to spend time with his mistress (tweeting) and forgets about hw4.

Ha ha ha! When I read this, it makes me think the Elon movie will be coming out soon! After all, all the the investor media pundents seem to think Tesla’s success or failure hinges on one man and his tweets or dumb decisions or how much time he alone wastes on that mistake called twitter.

Sorry fer gettin off topic. Back on…I am also frustrated and want my gosh darn v11.2.X. I am guessing it is not wholly in Elon’s control. Tesla probably has become institutionalized or less “informal” about releases and reducing any more liability.NHTSA is on em like stink on s$&@t

 

[legendsk]

I got 11.3.3 (2022.44.12) this morning. I went out to Teslafi and there it was (timed 2 minutes ago). So went out and checked the app on my phone and sure enough.

I'll confess that all the delays, combined with the fud had eroded my expectations of FSDb. It wasn't completely gone, but I was beginning to question my purchase of a lottery ticket, based on vaporware.

It took about an hour to install, a little longer than the usual update but went smoothly. It's all good folks.

It used to insist on taking its' lane out of the center on streets with poorly painted lanes. Fixed. Stays in the appropriate lane, with the occasional swerve around chuckholes, smooth swerve btw. On curves, hugs the inside to give traffic in the next lane more clearance - so much more comfortable.

It is still very cautious at stop signs. Nice informative blurbs appear, Stopping for traffic control - Creeping for better visibility. . . Then a very human like smooth left turn and accelerate to the speed limit. The wild whipping the steering wheel back and forth and excess acceleration in the turn is GONE.

Ignores exit lanes and unmarked street widening and stays comfortably and smoothly in the traffic lane like a human would.

It does show a tendency to want to get in the passing lane, even when there is no other traffic. (The blurb says use the turn signal to cancel the lane change.) Tapping the turn signal doesn't cancel, it is necessary to fully press the turn signal into latched mode - then it will cancel the signal and stay where it belongs. This will take a little getting used to, because it's so different from how it used to behave, but works well.

On navigate it once failed to move to the left lane to take an entrance ramp and I had to override it and reengage. But overall, a good trip.

The steering wheel nags every 10 - 15 seconds are still there and a nuisance. And the required torque is high enough to be a big problem if autopilot drops out, a full second of that much torque, without the resistance of autopilot would put us in the ditch. So the requirement is to continuously torque the wheel enough to crash, but be always ready to stop torquing INSTANTLY if ego stops pulling against you.

I have described the experience previously as like being the driving instructor for a student on their second lesson. It would keep the passenger's heart rate above 'resting' - maybe an average of 75 bpm? With excursions to 120 bpm?

With 11.3.3 I really believe my passengers will after 10 -15 minutes relax to a resting heart rate. And for the driver? Well, there are a lot of crazies out there and you have to be alert and prepared for anything, but now FSDb will be a nice experience, making driving easier and safer.

Oh, and although I am guilty of making fun of fart noises and light shows, when I wanted the software efforts focused on a smooth comfortable driving experience, now BOOMBOX IS BACK. I can play good music from USB through my external speakers, which is a nice touch. My faith in eventually being able to take a nap in back while it drives to New York is restored. Good job, Tesla software engineers.

 

[JHCCAZ]

It varies by state in the US. It's legal where live, but FSD really needs to avoid making unforced lane changes in intersections... It's like today it's logic doesn't even try to avoid it when it could just... wait until it passes the intersection.

Interesting, because I have a place in my nightly drive that pretty clearly shows the car is trying to avoid changing lanes in the intersection.

Right turn into a turn/service lane, then the car needs to merge into the rightmost of three through lanes, then twice more to to get across all lanes and end up in the left turn lane at the next main light. But there is an intermediate stoplight intersection a few hundred yards from the initial turn.

It's very clear that the car will not change lanes across that intermediate intersection. It always signals to change into the middle lane as it's approaching the stoplight, and sometimes completes that before the intersection - but if it decides it can't complete it before the light, then it cancels, waits until it has just crossed the intersection, and immediately reactivates the blinker and completes the change into the middle lane right after the light.

I've probably crossed this intersection 100 times on FSDb, at night with little or no traffic. It has achieved the lane change ~20 times before the light, but canceled/deferred ~80 times until after passing through. In no case did it ever get caught changing while actually crossing the intersection.

Perhaps this is because of following specific local traffic laws, I'm not sure. But its intent to respect the intersection is very clear.

 

[archae86]

I had an oddly contrasting pair of drives to and from church running 11.3.3 this morning. There was a small amount of large-flake snow falling as I started up, and perhaps that somehow toggled a mode change. Whatever the reason, there were two very clear differences from normal:
1. very, very placid acceleration from stops, both in my low-speed neighborhood and on the 55 mph 4-lane with median main road
2. I had LOTs of trouble with attention nags. I actually wondered whether the torque requirement had been considerably altered, as I generally have very little trouble on the same journey in the same car.

Yet coming home in the sunlight two hours later both behaviors were gone. So, tentatively, these two are conditions that 11.3.3 running on our Y can slip into, somehow, then recover on a subsequent drive.

Overall I find 11.3.2 and 11.3.3 to be quite good, and definitely the best of the dozen releases of FSDb I've driven.

 

[rlsd]

I think Tesla should detect the pressure on the steering wheel according to the driver's weight. I believe I get nagged often because I don't have big and heavy hands.

 

[Supcom]

Does anyone know how TeslaFi and Teslascope get the update numbers?

Aggregated from their subscribers' cars.

 

[tivoboy]

Do firmware upgrades require Wifi connectivity or can it be downloaded via cellular service? I have the premium connectivity package. Does it also require a minimum battery level?

Thanks

Some require WIFI and some will come over cellular, more the necessary, required or import NHTSA or other updates can come over cellular. I think for a shift to FSDb one is going to need a WIFI connection, but it COULD be that the latest NHTSA recall update might be included and therefore it COULD possibly come over wifi.

I have found that updates don't come when the battery is low, ~ <20%, so when one knows there is an update looming it might be best to make sure the car has maybe >50%.. I don't think we have any real evidence of this, but I personally have several instances where AFTER I added power to the car, from low 20's to 75% or so, within an hour an update appeared. Could be completely coincidental, but it's more than once.

 

[JHCCAZ]

I think Tesla should detect the pressure on the steering wheel according to the driver's weight. ...

Uh oh, discriminatory fat-shaming nags "triggered" by heavy drivers? Not sure how well that would go over in these oversensitive times!

Seriously though - comfortably getting to the right torque can take some adjustment of habits. It bothered me for months, then one day I realized that I'd kind of stopped thinking about it, and was getting very few nags or accidental disengagements.

It would have been nice if they had a better method, like capacitive sensing as people often mention. But I think Tesla (rightly or wrongly) looks at this as a stopgap along the way to a completely hands-off system, and they don't want the cost or complexity of specialized monitoring features in the steering wheel. Your idea wouldn't require that, but I'm guessing that the "right" torque detection threshold, if there is one, varies from person to person due to several factors aside from body build.

 

[rlsd]

Uh oh, discriminatory fat-shaming nags "triggered" by heavy drivers? Not sure how well that would go over in these oversensitive times!

Seriously though - comfortably getting to the right torque can take some adjustment of habits. It bothered me for months, then one day I realized that I'd kind of stopped thinking about it, and was getting very few nags or accidental disengagements.

It would have been nice if they had a better method, like capacitive sensing as people often mention. But I think Tesla (rightly or wrongly) looks at this as a stopgap along the way to a completely hands-off system, and they don't want the cost or complexity of specialized monitoring features in the steering wheel. Your idea wouldn't require that, but I'm guessing that the "right" torque detection threshold, if there is one, varies from person to person due to several factors aside from body build.

I found the solution for myself: Amazon.com : metal wrist brace

 

[LowlyOilBurner]

On narrower unmarked roads, it still almost comes to a complete stop when a vehicle is approaching. There’s enough room to move over for the other vehicle and maintain the same speed.

It’s also not giving enough room for pedestrians walking on the road. I had two instances Friday and today where I had to take over, the people looked at me like I was going to run them off the road. This has to get fixed.

 

[dflowerz]

My Model Y updated this morning. Have not tested it yet and not in much of a hurry because of what I have read here. Still very much beta. FSD aside, hoping for some Apple music improvements because so far it has been a little flaky about working consistently. Also hoping for some sort of 360 parking view on par with all the other cars that have it.

 

[Dennisis]

I haven't experienced the pedestrians yet but have passed bicyclists and was impressed by the slight move to the left to give them room without being too aggressive or impeding the passing lane - very human-like.

 

[aronth5]

I found the solution for myself: Amazon.com : metal wrist brace

On narrower unmarked roads, it still almost comes to a complete stop when a vehicle is approaching. There’s enough room to move over for the other vehicle and maintain the same speed.

It’s also not giving enough room for pedestrians walking on the road. I had two instances Friday and today where I had to take over, the people looked at me like I was going to run them off the road. This has to get fixed.

I feel the same way for pedestrians and cyclists. The law is 3 feet but most drivers move over even further. FSD also IMO waits to long to move over and then moves back over too quickly. The former often scares pedestrians wondering if I'm moving over for them the latter can be a real problem for fast cyclists.

 

[Scott Chapman]

My car's been parked in my garage for a few days. So far no disengagements!

Any ideas why FSDb is not on the main branch of software? If I were to venture a guess it would be to keep existing cars, not part of the FSDb program, to continue to use AP as is. Maybe when AP uses the same stack as FSDb THEN maybe they will be able to share the code branch?

 

[D Good]

I got 11.3.3 (2022.44.12) this morning. I went out to Teslafi and there it was (timed 2 minutes ago). So went out and checked the app on my phone and sure enough.

I'll confess that all the delays, combined with the fud had eroded my expectations of FSDb. It wasn't completely gone, but I was beginning to question my purchase of a lottery ticket, based on vaporware.

It took about an hour to install, a little longer than the usual update but went smoothly. It's all good folks.

It used to insist on taking its' lane out of the center on streets with poorly painted lanes. Fixed. Stays in the appropriate lane, with the occasional swerve around chuckholes, smooth swerve btw. On curves, hugs the inside to give traffic in the next lane more clearance - so much more comfortable.

It is still very cautious at stop signs. Nice informative blurbs appear, Stopping for traffic control - Creeping for better visibility. . . Then a very human like smooth left turn and accelerate to the speed limit. The wild whipping the steering wheel back and forth and excess acceleration in the turn is GONE.

Ignores exit lanes and unmarked street widening and stays comfortably and smoothly in the traffic lane like a human would.

It does show a tendency to want to get in the passing lane, even when there is no other traffic. (The blurb says use the turn signal to cancel the lane change.) Tapping the turn signal doesn't cancel, it is necessary to fully press the turn signal into latched mode - then it will cancel the signal and stay where it belongs. This will take a little getting used to, because it's so different from how it used to behave, but works well.

On navigate it once failed to move to the left lane to take an entrance ramp and I had to override it and reengage. But overall, a good trip.

The steering wheel nags every 10 - 15 seconds are still there and a nuisance. And the required torque is high enough to be a big problem if autopilot drops out, a full second of that much torque, without the resistance of autopilot would put us in the ditch. So the requirement is to continuously torque the wheel enough to crash, but be always ready to stop torquing INSTANTLY if ego stops pulling against you.

I have described the experience previously as like being the driving instructor for a student on their second lesson. It would keep the passenger's heart rate above 'resting' - maybe an average of 75 bpm? With excursions to 120 bpm?

With 11.3.3 I really believe my passengers will after 10 -15 minutes relax to a resting heart rate. And for the driver? Well, there are a lot of crazies out there and you have to be alert and prepared for anything, but now FSDb will be a nice experience, making driving easier and safer.

Oh, and although I am guilty of making fun of fart noises and light shows, when I wanted the software efforts focused on a smooth comfortable driving experience, now BOOMBOX IS BACK. I can play good music from USB through my external speakers, which is a nice touch. My faith in eventually being able to take a nap in back while it drives to New York is restored. Good job, Tesla software engineers.

Hold up. Boombox in drive is back? That might actually be the best news out of this release considering the absolute failure of chucks turn.

 

[black knight 3]

11.3.3 is noticeably smoother and lane changes are more refined and human like. However, I witnessed a regression for right turn at stoplight. A cyclist was crossing the intersection and the car lunged forward just as the cyclist was in the middle of the intersection and the car attempted to make a right turn as if the cyclist wasn't even there. I intervened and applied the brakes immediately. I will check my dashcam for this event. I could care less about small things like erratic turn signal flashing at this point in time. I am more concerned about the dangerous decisions that may cause severe injury or death. I've been in the pilot FSD testing for almost 2 years and this still shows that it is not close to anything but a stressful L2 driver assist at this time. Maybe L4 in 4 years when 21.x.x is being released...