FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[Mardak]

Mine went directly to FSD 10.69.25.1 back in December

Just to confirm, your new Y without Ultrasonics Sensors has FSD Beta but not Summon (basic or Smart)? If so, presumably some 11.x will restore those missing features by replacing the legacy summon stack with FSD Beta's similar to the replacing of legacy highway stack.

So it'll be interesting to see if new S/X without USS will follow your experiences.

 

[sleepydoc]

It says that "...some features may be "temporarily limited or inactive at delivery including [a few that are part of FSD capability] and Full Self Driving capability features..."

This language is extremely similar, though not quite the same, as the language in the original press release regarding USS removal, last autumn well before any HW4 news. So why do they add the somewhat redundant mention of FSD capability, to a list that already names some FSD capability features? I think partly just due to imprecision, but also possibly to emphasize the point.

It’s also incredibly vague and impossible to interpret accurately. Whether that’s intentional or not is up for debate but the service adviser’s response was not exactly helpful or encouraging.

Given Tesla’s history of wildly inaccurate time estimates and the fact that so many features are crippled without ultrasonic sensors I would be very hesitant to take delivery of a car without them. It’s dumbfounding to me why Tesla takes the approach of ‘I assume we can figure out another way so let’s break it, then figure out how to fix it’ rather than ‘figure out a better way, then make the switch.’

 

[JB47394]

It’s dumbfounding to me why Tesla takes the approach of ‘I assume we can figure out another way so let’s break it, then figure out how to fix it’ rather than ‘figure out a better way, then make the switch.’

If Tesla knew that they didn't want to keep using ultrasonics and their supplier contract was expiring, they may have just decided to chuck it and accept that some customers were going to get mad. They save a bunch of money, and customers will eventually get their functionality back. It's a monopolistic/privileged attitude, but as the front runner, that's where Tesla is.

 

[AlanSubie4Life]

and customers will eventually get their functionality back.

Haha. That is a good one you snuck in there!

 

[Mardak]

To me, it's not even so clear that this "Proceed with Delivery" agreement language is any different from that of non-USS cars delivered since November or so. I found only one other screenshot example, and that had quite similar language but was shorter, with a "Learn More" button suggesting another page that wasn't shown.

Here's both for those wondering what changed:

Where the new text moves the last sentence "is no longer equipped with ultrasonic sensors" to be in the first sentence: "will not include ultrasonic sensors." And instead of "which will temporarily impact some features" linking to Replacing Ultrasonic Sensors with Tesla Vision support page, it directly lists the features and future update.

The change here seems to be more upfront instead of hiding things behind a link that might have resulted in lower customer conversion rate. People have gotten FSD Beta on 2023 3/Y without ultrasonic sensors, so presumably 11.x will also be available to the larger group of new S/3/X/Y if USS-removal is the difference from before.

It's possible that HW4 computer/sensors will be introduced with the updated S/X without USS, but that would probably mean Autopilot team needs to get all the other features listed on the support page working: Forward Collision Warning, Automatic Emergency Braking, Lane Departure Warning / Avoidance, Emergency Lane Departure Avoidance, Pedal Misapplication Mitigation, Auto High Beam, Autowiper, Blind Spot Collision Warning Chime, AutoSteer, Auto Lane Changes, Navigate on Autopilot, Traffic Light and Stop Sign Control.

 

[JHCCAZ]

It’s also incredibly vague and impossible to interpret accurately. Whether that’s intentional or not is up for debate but the service adviser’s response was not exactly helpful or encouraging.

Yes, this is extremely common from Tesla and from Elon. I don't blame anyone for having a hard time getting the meaning, because it isnt clear.

In these cases, my first thought is usually that it's not intentional, but simply poor construction. If they simply use deliberately vague language like

"including but not limited to",​

"in the future",​

"when available",​

it would be more obvious that it's just cautious legalism. But when they create confusing constructions like

"...Including Park Assist, alAutopark,Summon. Smart Summon Full Self Driving capability features (if applicable). ..."​

then it becomes very difficult to understand, and leads to unnecessary angst, pessimism and sometimes damaging FUD, none if which are in Tesla's interest.

It's as if you looked at the side panel of a box of crackers, and it said

"May contain wheat or other flour, butter, salt, packaging materials and edible ingredients (if applicable)".​

Those could be perfectly normal crackers, but you might think twice before eating them!

Regarding the SA's answer, in response to Jason's attempt to get clarification, well that's just a classic:

"FSD is still working in progress. It does not have any beneficial to FSD"​

Wow. Provides entertainment but not clarity. Probably not a native speaker, but who knows these days. But hey, it's only a car and a $15,000 FSD purchase. If you don't take it the next guy will. Just hit the Accept and Continue button so I can close this chat.

 

[sleepydoc]

In these cases, my first thought is usually that it's not intentional, but simply poor construction. If they simply use deliberately vague language like
"including but not limited to","in the future","when available",it would be more obvious that it's just cautious legalism. But when they create confusing constructions like
"...Including Park Assist, alAutopark,Summon. Smart Summon Full Self Driving capability features (if applicable). ..."then it becomes very difficult to understand, and leads to unnecessary angst, pessimism and sometimes damaging FUD, none if which are in Tesla's interest.

The problem is in addition to the angst, pessimism and FUD you mention, the ambiguity can also harm them legally, rather than protect them.

In any case, the ambiguity is not good.

 

[DarkForest]

FSDb (Autosteer on City Streets) is a completely different animal. I have never had an intervention-free drive. The thing can’t make a mile or two without requiring and intervention. And while I can contemplate some reasons why other folks may let it go a little further than I do (e.g., they’re not in a hurry so they let it miss a turn or they don’t care if they are pissing off other drivers around them), it’s hard to imagine how my drives in suburban Atlanta can be so different from other parts of the U.S. that folks are taking it on intervention free drives and/or finding driving with it to be relaxing or useful.

I think the location matters a whole lot. The NNs are trained more on certain characteristics than others. In a small town in Texas, it works pretty well. Traffic is really chill, so the acceleration is fine, maybe a bit too fast. It almost never rains here, so weather isn’t a problem. There’s one intersection where it sucks at understanding the lanes, but I can let it mess up because there’s never any traffic there to bother. There are other intersections it sucks at, so I just take over for a few seconds in those spots.

But if I wanted to, I could easily come up with a 30 minute route at a specific time of day where it’d have no problems whatsoever. However, while that may be good shill material, I wouldn’t really benefit from it. Instead, I’m just glad it’s actually useful to me... but also concerned that it doesn’t work well for you. I’m really hoping that in a decade, it works well enough everywhere that I can trust it to drive around my son and that he’ll never have to drive if he doesn’t want to.

I think the solution to this puzzle is: this is not actually happening.

I wish I could take a video of my drives, but I’d rather not doxx myself.

 

[AlanSubie4Life]

I wish I could take a video of my drives, but I’d rather not doxx myself.

I would definitely be shocked by a video of reasonable length with no interventions (or places where a reasonable intervention should have occurred).

30 minutes would be absolutely stunning and I would be blown away. That would indicate a vehicle difference or massive location dependency (which I don’t see other than the obvious dependencies related to complexity of course).

It’s quite incredibly capable now. But it does require constant interventions to ensure correct behavior.

 

[Dewg]

or places where a reasonable intervention should have occurred

This is where you typically get into trouble with the group here.

 

[AlanSubie4Life]

This is where you typically get into trouble with the group here.

Lol, I think we all can basically agree on what is reasonable. Not complicated.

 

[JB47394]

I would definitely be shocked by a video of reasonable length with no interventions (or places where a reasonable intervention should have occurred).

DIrtyTesla is my favorite FSDb YouTuber. He did a roughly 20 minute drive on 10.69.2 a couple weeks ago and had "zero interventions". He's very tolerant of the car's antics, so whether any interventions should have occurred is up to you. There are some glitches in the driving because he's going to a series of destinations and the car wants to stop at each destination until told to move again. There's one outright refusal for what is essentially a U-turn and DirtyTesla says that the car has completed that turn in the past, but it fails it in this video.

 

[Dewg]

Lol, I think we all can basically agree on what is reasonable. Not complicated.

No, we can't. That's the issue.

 

[Supcom]

No, we can't. That's the issue.

Of course not. "Reasonable" is what system engineers would call non-verifiable because it is subjective. "Reasonable" would need defined in objective terms that we would all agree to (which is unlikely).

 

[AlanSubie4Life]

Of course not. "Reasonable" is what system engineers would call non-verifiable because it is subjective. "Reasonable" would need defined in objective terms that we would all agree to (which is unlikely).

I think it is extremely objective and verifiable. Do we really need take the time to define something when we “know it when we see it?”

In any case I hope we can agree that the car failing to take a trivial mapped right turn (8:50) would count as an intervention. The car did great in the first 9+ minutes of this video - overall impressive (minimal traffic of course which does make it easier). But I think most would not have the patience for the stops. It’s really the sort of thing you have to be in the car for to understand how slow they are - it’s weird how it is not more obvious when watching, actually.

Let me know when someone can point to an actual intervention-free example of significant length. It will happen eventually!

 

[JB47394]

In any case I hope we can agree that the car failing to take a trivial mapped right turn (8:50) would count as an intervention.

I'd intervene, but "trivial"?

Let me know when someone can point to an actual intervention-free example of significant length. It will happen eventually!

I think it's obvious that the car can perform zero intervention drives. It's just a matter of getting lucky. But one or two examples of zero intervention drives doesn't really mean anything other than progress is progressing. It won't be the product we were promised until the great majority of drives are zero intervention.

I remember when this thread was about v11.

Oh, and I sure hope that V11 makes zero intervention drives more common. There, now we're back on track.

 

[dckiwi]

I remember when this thread was about v11.

 

[kabin]

I remember when this thread was about v11.

Yes, those were the days. V11, software that time forgot and releases seemingly cannot improve.

 

[AlanSubie4Life]

I'd intervene, but "trivial"?

I think it's obvious that the car can perform zero intervention drives. It's just a matter of getting lucky. But one or two examples of zero intervention drives doesn't really mean anything other than progress is progressing. It won't be the product we were promised until the great majority of drives are zero intervention.

Oh, and I sure hope that V11 makes zero intervention drives more common. There, now we're back on track.

Not talking about freeways (that I definitely believe can be zero intervention).

Real-time videos only with audio.

This is an intervention, you cannot do this (pull out and stop in the middle of the road in front of an oncoming vehicle on the presumption they are turning left). I saw this, knew it looked awful, and went to the real-time video. In the real-time video the oncoming vehicle honked - this is long established as an intervention!

25 minutes in on this video. This is not the only example of incorrect behavior in this video. I saw hints of others but did not investigate.

It is well established that Whole Mars “zero-intervention” videos have occasional interventions because he accomplishes them by allowing the car to do things extraordinarily slowly, or incorrectly, as illustrated above.

 

[sleepydoc]

This is where you typically get into trouble with the group here.

Bingo!

Lol, I think we all can basically agree on what is reasonable. Not complicated.

no - from past discussions you would fail FSD if it didn't perform exactly as a human would (and by human you typically mean exactly how you would.) As I recall, you're unhappy with the braking pattern as well as the acceleration pattern, among other things.

To be clear, a lot of times it depends on the driver's tolerance and surrounding traffic. One of my most common interventions is to goose the accelerator at a right turn on red or yield. From past experience, FSD will eventually proceed but takes a few seconds before it's comfortable doing so. It's far more conservative than most human drivers that way and I nudge it along so as not to inconvenience the driver behind me but what it's doing is not illegal by any stretch.

I'll also say that many of the other interventions I do are fairly predictable and 'known bugs.' That's not saying they're not significant, just that in theory it should be easier to target them.