Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register
This site may earn commission on affiliate links.
The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.


Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.


Here is the AI Day explanation by in 4 parts.


screenshot-teslamotorsclub.com-2022.01.26-21_30_17.png


Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

 
Last edited:
Interesting. For me, I think 11.3.6 has somewhat reduced but not eliminated) the transient false turn signal blinks that mostly occur approaching intersections. But this version displays a strange and very frustrating insistence to change out of the lane it needs to be in, prior to arriving at a real turn laying on the route. I'm guessing it's a mapping issue but I'm not sure.
I had to take my X into service (squeaking upper control arms) yesterday and my wife followed me. The transient false turn signal still happen for me on 11.3.6 as well as the change out of lane. My maps were updated within the past week. This happens when I have the minimum lane change option on. FSDb gets annoying to use since they took away some of the user options. -- visited a friend two days ago and his car is acting the same way and was 'relieved' to here my same experience ... just because then he knows it just not his car ... but it annoyed him as well.
 
  • Informative
  • Like
Reactions: JHCCAZ and jebinc
Best part about 11.3.6 is how there are basically zero nags as long as you keep your eyes on the road and BOTH hands on the wheel (with zero torque, floating them, appears to be fine, or just holding the wheel normally with balanced torque).

(There is the occasional one, perhaps - hard to say, since the ones I have seen have always correlated with taking a hand off the wheel, and/or looking at the center display.)

Pretty good from that perspective. The days of torquing all the time are in the past it seems. As expected, with the cabin camera, they can ensure that your eyes are on the road, and hands are on the wheel (it cannot see your hands or the wheel, but it of course does not matter for this!) - an easy problem for the system. In spite of it clearly not being intended for this purpose originally, seems like they have done most of the work to make it work.

It’ll be interesting in the future in FSD-Beta-assisted accidents when they have to rely on and report what the camera saw - since torque is clearly no longer going to be sufficient as an indicator of driver attention (it is basically never detected).
 
Last edited:
No .... can't remember to do it every drive :(

But the point is .... there is no reason to change. There are no slow vehicles in the front or anything like that. Just that apparently FSD no longer likes right lanes, even when there are only two and its a city street.
I've noticed that during one of my frequent drive. No matter how I set my FSD, it always switches to the left lane, at the exact same spot. This is even with Minimize turned on. I assumed it must be a mapping info error. This is on a 4-way rural road with almost no traffic light.

Given that's there's a rumor that Tesla collects regional data and share them at regional level between cars, I did disengage and reported this behaviour. Probably won't happen, but I'm hoping Tesla will collect info like this, and override whatever the current version of the map says, and ultimately roll-in the new info into the main map.
 
  • Like
Reactions: JHCCAZ and clydeiii
Here is a scary moment I had on Monday during my usual late afternoon drive.
You can see that the car (on 11.3.6) slowed normally behind this big flatbed trailer, but then lurched forward as if to ram it. I slammed the brake and it ended up very close, it felt like 2 ft away or less. Not good.

As always, there was strong sun hitting the windshield camera, but this was about 90 minutes before sunset so it was still reasonably high, not much of a problem if you have a sun visor - but the camera has no glare shade.
(This is the first time I've pulled a dashcam clip, and I'm not at all experienced with web video so apologies if I did anything wrong with it.)
 
The UPL turn that constantly messes with FSD isn't turning onto a cross road. FSD does pretty well now with that. The issue for me is the left hand turn when a car is waiting to pull out and a car is approaching me.
  1. Does FSD slow down and/or stop to let the car pull out in front of me and turn left onto the road I'm traveling?
  2. Does FSD continue and turn left around the car waiting to pull out?
  3. Does FSD wait for the oncoming car to turn?
  4. Does FSD turn left before the oncoming car?
  5. And what happens when the car waiting to pull out waves me on?
  6. And what happens when the car approaching either waves me to start my turn or flashes their lights to start my turn?
5 or 6 happen a lot.
All of the above can happen and every day when I take this type of UPL all bets are off unless there are no cars.
 
  • Like
Reactions: VanFriscia
Here is a scary moment I had on Monday during my usual late afternoon drive.
You can see that the car (on 11.3.6) slowed normally behind this big flatbed trailer, but then lurched forward as if to ram it. I slammed the brake and it ended up very close, it felt like 2 ft away or less. Not good.

As always, there was strong sun hitting the windshield camera, but this was about 90 minutes before sunset so it was still reasonably high, not much of a problem if you have a sun visor - but the camera has no glare shade.
(This is the first time I've pulled a dashcam clip, and I'm not at all experienced with web video so apologies if I did anything wrong with it.)

It looks to me like it recognized the flatbed as a vehicle at first and pulled up behind it. But after a second it forgot that it had recognized the flatbed, and then the features of the flatbed were strongly reminiscent of driveable space (stripe down the lane, and the ladders could be misinterpreted as a grate).

This is a strong argument for improving FSD Beta's object-permanence. I know the visualization doesn't show everything, but the fact that cars/pedestrians still blink out of existence when obstructed leads me to believe it's not remembering them.
 
It looks to me like it recognized the flatbed as a vehicle at first and pulled up behind it. But after a second it forgot that it had recognized the flatbed, and then the features of the flatbed were strongly reminiscent of driveable space (stripe down the lane, and the ladders could be misinterpreted as a grate).

This is a strong argument for improving FSD Beta's object-permanence. I know the visualization doesn't show everything, but the fact that cars/pedestrians still blink out of existence when obstructed leads me to believe it's not remembering them.
I always pay special attention to flat bed trucks. I had one situation where a flatbed pulled out in front of me on a dark rainy night and FSD never slowed down or displayed the flat bed truck on the display. And that's not the only time I've seen poor response by FSD to flatbed trucks near me.
 
I always pay special attention to flat bed trucks. I had one situation where a flatbed pulled out in front of me on a dark rainy night and FSD never slowed down or displayed the flat bed truck on the display. And that's not the only time I've seen poor response by FSD to flatbed trucks near me.
This is one of those scenarios that radar would do wonders for validating what vision thinks or doesn't think it sees.
 
This is one of those scenarios that radar would do wonders for validating what vision thinks or doesn't think it sees.
I was thinking the same thing. There's no substitute for multiple sensor types since they all have well known limitations. Depending on the scenario even an ultrasonic sensor with a min range of 15ft would've had a chance to initiate a warning and brake versus allowing the status quo.

Multi sensors didn't fit into their simplistic view of the problem - cost over everything else.
 
Last edited:
This is one of those scenarios that radar would do wonders for validating what vision thinks or doesn't think it sees.
Remember radar does not do well with stationary objects.

Not saying radar would not help. But having radar has not helped with fatal underruns so far.

Anyway it is easy to tell in this case there is a truck there. For this case, just need to solve machine vision and have persistence, etc. “Easy!”

Lots of problems, not just a lack of sensors.
 
Remember radar does not do well with stationary objects.

Not saying radar would not help. But having radar has not helped with fatal underruns so far.

Anyway it is easy to tell in this case there is a truck there. For this case, just need to solve machine vision and have persistence, etc. “Easy!”

Lots of problems, not just a lack of sensors.
Perhaps the phoenix high definition radar will solve for this if it's a real thing, since it can detect stationary objects. Biggest downside I can see happening if that is true is there will be so many configurations of hardware out there, it will be some kind of nightmare getting FSD to work on all of them.
 
Perhaps the phoenix high definition radar will solve for this if it's a real thing, since it can detect stationary objects. Biggest downside I can see happening if that is true is there will be so many configurations of hardware out there, it will be some kind of nightmare getting FSD to work on all of them.
Not so sure. Cruise is loaded with sensors including multiple LIDAR and high res RADAR, and one just recently slammed into the back of a bus.
 
Just need to train on more flatbeds, problem solved.

Lol. NNs are so brittle, and they routinely hallucinate. It may be many years (or maybe never) before they’re useful without human vetting to keep them on track.

Yet somehow we’re within striking distance of autonomy. Lol.

Front bumper camera would have solved this specific issue too, probably. (Many others remain!) But “the best part is no part!” Worked out well for Starship platform too.
 
  • Like
Reactions: Sharps97
Remember radar does not do well with stationary objects.

Not saying radar would not help. But having radar has not helped with fatal underruns so far.

Anyway it is easy to tell in this case there is a truck there. For this case, just need to solve machine vision and have persistence, etc. “Easy!”

Lots of problems, not just a lack of sensors.
Doppler radar doesn't do well with stationary objects as they all end up in the 0th bin but not an issue with more sophisticated radar modulation schemes.
 
Best part about 11.3.6 is how there are basically zero nags as long as you keep your eyes on the road and BOTH hands on the wheel (with zero torque, floating them, appears to be fine, or just holding the wheel normally with balanced torque).

(There is the occasional one, perhaps - hard to say, since the ones I have seen have always correlated with taking a hand off the wheel, and/or looking at the center display.)

Pretty good from that perspective. The days of torquing all the time are in the past it seems. As expected, with the cabin camera, they can ensure that your eyes are on the road, and hands are on the wheel (it cannot see your hands or the wheel, but it of course does not matter for this!) - an easy problem for the system. In spite of it clearly not being intended for this purpose originally, seems like they have done most of the work to make it work.

It’ll be interesting in the future in FSD-Beta-assisted accidents when they have to rely on and report what the camera saw - since torque is clearly no longer going to be sufficient as an indicator of driver attention (it is basically never detected).
It feels like it’s almost easier to rest one hand on your leg and place it at 7 o’clock on the wheel and give it a-little resistance than keeping both hands on the wheel with no pressure. I also wear sunglasses quite a bit so not sure how that affects the cars ability to accurately evaluate attentiveness.
 
Last edited:
It feels like it’s almost easier to rest one hand on your leg and place it at 7 o’clock on the wheel and give it a-little resistance than keeping both hands on the wheel with no pressure. I also wear sunglasses quite a bit so not sure how that affects the cars ability to accurately evaluate attentiveness.
I often use prescription sunglasses and FSD cannot determine if I'm paying attention or not. My understanding is it's likely unique to my sunglasses.
 
  • Like
Reactions: JohnB007