FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[PACEMD]

Now I have to drive my wife's M3P with only AP and see if I can rack up some strikes. And just how many AP strikes does a godly knight errant get before being led to a folie à deux with the King of X.......or having to steer a damn car..........

"People are not staring at me, they're staring with me".......

 

[Yelobird]

Well, that sucks junk right up the nose. Getting suspended from AP is like staring into the abyss of the Baby Driver........

Scoot down the road, what's my number?
I wonder how your engines feel

Hey Mr law breaker! You take your penance as deserved! Your next suspension the “Shame” soundtrack will play on All of your external speakers continuously with wipers a 140% speed to signal All that you disobeyed the rules and all should fear you.

 

[mgs333]

A new behavior I've started seeing in the last few weeks is the car seems to respond/yield to other cars signalling. Like my car will slow down when a car is signaling so the other car can go in front of me. The car used to completely disregard other cars signaling even on 11.4.9.

My autowipers were back to being lazy in the rain today. It did work but way too late to wipe every time.

 

[Dewg]

A new behavior I've started seeing in the last few weeks is the car seems to respond/yield to other cars signalling. Like my car will slow down when a car is signaling so the other car can go in front of me. The car used to completely disregard other cars signaling even on 11.4.9.

My autowipers were back to being lazy in the rain today. It did work but way too late to wipe every time.

Yes, it's a new feature. You can override with the accelerator. Makes the car more courteous as a human that would let people merge when they indicate.

 

[AlanSubie4Life]

It seems to be. As I’ve said, I’ve gotten several strikes without warning while looking straight ahead.

Yeah something wrong with your car. This is abnormal (though I think you are also including experiences with old software which was allegedly defective in this regard (not a car issue)), but really the question is whether this is happening for the current software.

It now always gives that warning on initial engagement of FSDb. That one you can ignore. Just a well-pay-attention if you're gonna play FSDb reminder.

It has warned for years, but it depends on what warning you are talking about (there is also the “pay attention to the road” warning accompanied by a beep.

When successful, this technique gets me the initial lower screen small warning that you have to pay attention, but avoids the second and third, as to which the second is high in the screen and includes a touch of red, and the third is the blue horizon thing.

Yeah I don’t get this. Unless deliberately eliciting it by staring downwards when engaging I’ve never seen this. It’s actually pretty hard to get.

The idea is to be driving normally then pull down the lever. If you’re doing that there should be no “pay attention to the road.”

I don’t think you need to be actively steering on engagement though I guess I will have to check that out.

 

[Tronguy]

I get the “pay attention to the road” warning every time I engage FSD. Maybe it’s because I’m looking at the nav route briefly?

Yup, I get a very brief (one or two seconds) alert that one is supposed to pay attention to the road. I'm pretty sure that it doesn't depend upon where one is looking, since I see the message pop up in my peripheral vision even when I'm looking straight out front at the time. 2023.44.30.8, 11.4.9.

 

[pilotSteve]

Yup, I get a very brief (one or two seconds) alert that one is supposed to pay attention to the road. I'm pretty sure that it doesn't depend upon where one is looking, since I see the message pop up in my peripheral vision even when I'm looking straight out front at the time. 2023.44.30.8, 11.4.9.

So glad my 2017 S90D does not have these cameras. “Brick on a string”…….

 

[AlanSubie4Life]

So glad my 2017 S90D does not have these cameras. “Brick on a string”…….

I am curious what the difference in wheel torque frequency requirements is between those cars with cameras and those without.

That would be a potential downside of having a car without a camera, though it is possible there is no difference (I’m still not sure whether or not they are using the camera to detect the presence of hands on the wheel (it of course cannot see hands of the wheel but that is not needed to make the determination)). They could also be detecting hands on but not doing anything with that info to adjust torque nag frequency. Hands on/off isn’t in the list of parameters that I recall but it could have changed.

 

[Tronguy]

I am curious what the difference in wheel torque frequency requirements is between cars with cameras and without.

That would be a potential downside of having a car without a camera, though it is possible there is no difference (I’m still not sure whether or not they are using the camera to detect the presence of hands on the wheel (it of course cannot see hands of the wheel but that is not needed to make the determination)). They could also be detecting hands on but not doing anything with that info to adjust torque nag frequency.

Um. So, until September 29th of last year, I was driving a 2018 M3 LR RWD with FSD; after that, it was a 2023 M3 LR AWD with FSD. (Used the transfer-the-FSD offer..)

It took about a week to get to the then-production version of FSD, instead of the factory version. The factory version number I kind of forget, but, man, it was ugly going back a couple of point releases - much jerkier left and right turns and other stuff that I had been glad to forget.

When I got onto the non-factory version, things got better, but I was behind you-all on FSD version numbers. Nonetheless, the torque and eyeball requirements were there. And, frankly, didn't notice much difference on the new car versus the old one.

I had started my routines of:

1. One handed driving, usually the left, at the 9:00 or 8:00 position, where one hangs on, the car drives, but the nominal continual jiggling of the car vs. the mass of the hand/arm provides the needed torque, or
2. Two handed, 9 and 6 o'clock positions, where one torques slightly to the left on one breath and slightly to the right on the next. Make it a habit, and Torque Warnings R Not Us.

And they worked as well on the new car as the old.

 

[AlanSubie4Life]

Nonetheless, the torque and eyeball requirements were there. And, frankly, didn't notice much difference on the new car versus the old one.

And they worked as well on the new car as the old.

Interesting, but that wouldn't be comparing a car with and without cabin camera.

I'd expect probably it's a bit different at night though for your particular comparison. Maybe.

I don't find that manually firmly torquing the wheel works anymore, but maybe I should go back and try it. It's possible they just broke it at some point, I broke my habits, and now if I went back to gentle consistent torque it would start working again.

It's possible that when I'm not thinking about applying zero torque or some constant torque that I apply enough torque with enough variation and that is enough to minimize the nags to near zero. I really haven't actually experimented with all the permutations. And observing and experimenting tends to result in thinking about it which disrupts the experiment.

All I know is with zero torque it currently seems to be requiring torque once every 3-4 minutes or so on average (it can be longer or shorter).

 

[bhakan]

It now always gives that warning on initial engagement of FSDb. That one you can ignore. Just a well-pay-attention if you're gonna play FSDb reminder.

Yeah. This evening when I drove I got red "pay attention to the road" every time I engaged FSDb. It's new as I hadn't seen it in a while. In addition, the alert comes after engaging FSDb also often now. FSD missed an exit as it could not take the exit quick enough. I had disengage and take the next exit. But the car was making all sorts of alerts. I just sang " you need to calm down" and drove quietly .

 

[AlanSubie4Life]

Yeah. This evening when I drove I got red "pay attention to the road" every time I engaged FSDb. It's new as I hadn't seen it in a while. In addition, the alert comes after engaging FSDb also often now. FSD missed an exit as it could not take the exit quick enough. I had disengage and take the next exit. But the car was making all sorts of alerts. I just sang " you need to calm down" and drove quietly .

I don’t see this at all as long as I am driving and looking at the road when engaging. The only way I can get it is to look down while/after engaging.

 

[PVia]

Yes, it's a new feature. You can override with the accelerator. Makes the car more courteous as a human that would let people merge when they indicate.

My M3 has yielded to adjacent cars' turn signals for many months before 11.4.9

 

[hybridbear]

I am curious what the difference in wheel torque frequency requirements is between those cars with cameras and those without.

That would be a potential downside of having a car without a camera, though it is possible there is no difference (I’m still not sure whether or not they are using the camera to detect the presence of hands on the wheel (it of course cannot see hands of the wheel but that is not needed to make the determination)). They could also be detecting hands on but not doing anything with that info to adjust torque nag frequency. Hands on/off isn’t in the list of parameters that I recall but it could have changed.

We have a 2016 S90D without the cabin camera. If I don't actively put torque on the wheel when engaging FSD Beta the red hands on wheel message appears within seconds. Beyond that the time between hands on the wheel nags on the freeway are every 30 seconds. I have not timed it in other situations.

 

[rlsd]

Tesla recalls 2.2 million cars because of small font sizes.

This is the only "recall" that makes sense. This indicates the kids at Tesla only glue their face to the small cell phones and don't drive cars.

 

[MP3Mike]

Tesla recalls 2.2 million cars because of small font sizes.

This is the only "recall" that makes sense. This indicates the kids at Tesla only glue their face to the small cell phones and don't drive cars.

But only the font size on three icons:

The remedy increases the letter font size of the Brake, Park and Antilock brake system (ABS) visual warning indicators to be not less than 3.2 mm (1/8 inch), as prescribed in FMVSS Nos. 105 and 135.

 

[rlsd]

But only the font size on three icons:

Young people can read them fine. But older people have a hard time.
Tesla needs to do better than what NHTSA specified. Sometimes I cannot read speed limit on the screen due to glare. I have a hard time to figure out the green left/right blinker arrows.

 

[TK211X]

Need fisher price mode for old people.

 

[kabin]

Young people can read them fine. But older people have a hard time.
Tesla needs to do better than what NHTSA specified. Sometimes I cannot read speed limit on the screen due to glare. I have a hard time to figure out the green left/right blinker arrows.

Yep. One good thing about legacy auto manufacturers is they frequently seek feedback from small random groups of potential customers to evaluate designs before they reach production.

Good to know about the FMVSS but I bet there's also an SAE standard for dash indicators that includes font, icon size, ... No reason or benefit for re-inventing the wheel. They need to design for customers driving not playing on an ipad.

I'm not a fan of important vehicle information only being contained on ipad-like displays located at center dash but no doubt it's a cheap @$$ solution. It sucks they chose to use the same approach for the CT.

 

[TK211X]

All I can picture is some demented NHTSA official screaming down the highway in a soaked pull up having an aneurysm because he can’t find the microscopic horn button.

“AHHHH Things are different”

This is why it’s in the airbag again.