Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register
This site may earn commission on affiliate links.
The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.


Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.


Here is the AI Day explanation by in 4 parts.


screenshot-teslamotorsclub.com-2022.01.26-21_30_17.png


Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

 
Last edited:
I'm curious, anyone on 11.4.2 see their strikes disappear? I haven't been suspended but have 4 strikes. The release notes just say the suspension will last about a week; so wondering if this means the strikes go away after the same time period.
Haven’t seen them reset but I’ve noticed a bug - there is little to no audible warning when it wants you to turn/torque the wheel.

I got a strike a week or two ago when I didn’t notice the screen flashing blue. It beeped once and I glanced down just in time to see the red wheel of death. Then again yesterday I was driving I didn’t notice the screen flashing blue. I happened to glance down and saw it flashing very rapidly in a manner that would have been accompanied by beeping in past releases but there was absolutely no other warning. (I tugged the wheel quick enough to avoid a strike this time.)
 
I'm curious, anyone on 11.4.2 see their strikes disappear? I haven't been suspended but have 4 strikes. The release notes just say the suspension will last about a week; so wondering if this means the strikes go away after the same time period.
No. It doesn't appear that strikes are going away...you just don't strike out until you get 5 with cameras and 3 without. This is mostly a guess, but I can say for sure that 11.4.X doesn't reset them.
 
  • Like
Reactions: FSDtester#1
No. It doesn't appear that strikes are going away...you just don't strike out until you get 5 with cameras and 3 without. This is mostly a guess, but I can say for sure that 11.4.X doesn't reset them.
It doesn't, I still have a strike from a while ago when I was using my phone on my chest to record the Junk doing junky stuff
 
  • Like
Reactions: FSDtester#1
Haven’t seen them reset but I’ve noticed a bug - there is little to no audible warning when it wants you to turn/torque the wheel.

I got a strike a week or two ago when I didn’t notice the screen flashing blue. It beeped once and I glanced down just in time to see the red wheel of death. Then again yesterday I was driving I didn’t notice the screen flashing blue. I happened to glance down and saw it flashing very rapidly in a manner that would have been accompanied by beeping in past releases but there was absolutely no other warning. (I tugged the wheel quick enough to avoid a strike this time.)
Not beeping during the initial blue flashing is a long-established behavior, not new. Also, if you let it go on long enough to get the red hands and an FSD disengagement, that's not generally a Strike. It happens to me maybe once or twice a week when I'm not thinking about the wheel torque enough. FSD will disengage but will become available again in about 30 to 60 seconds.

This has happened to me many many times, but I still have no Strikes, which is confirmed by visiting the Autopilot FSD menu.

There's a deeper Autopilot "jail" disengagement in which less for the remainder of the current drive. I'm not clear if that is 1:1 the same as a Strike on FSD. I think the only times that happened to me was on the highway on standard Autopilot (I avoided NoA by choice back then), when I drove too far above the speed limit during a passing maneuver. I had to actually get off the road, put it in Park, I get out of the car and start over. But that wasn't FSD beta.

Occasionally I've had FSDb disengage due to bright sun glare, and at least twice it wouldn't come back until I parked and started over. However those weren't driver penalty incidents, and (correctly) did not create FSD Strikes.
 
  • Informative
Reactions: FSDtester#1
True - but then, won't the visualization show it as flickering ?

In Curt's video, it was solid red.

ps : It makes sense that somehow FSD thinks its like a stop sign situation ...
Ah. This is my field, digital signal processing. We may be looking at an aliasing problem.

Thing is, in DSP, when one samples, it's kind of important to sample (in theory) 2X as fast as the highest incoming frequency present. (Official name: The Nyquist Limit.) People who do this kind of thing for real with, say, digital oscilloscopes, typically sample at 3X or 5X the incoming frequency. A 1 GSa/s 'scope is typically rated for 200-250 MHz bandwidth. One can play silly buggers with this (Re: so-called, "Communications Analyzers"), but the general rule is at least 2X sample rate. Another obvious example is basic, POTS telephony: The audio sample rate is 8 kSa/s, but, to prevent aliasing, the audio before sampling is typically analog filtered to 2.5 kHz or so before the digital sample. (This is one reason why men tend to sound better than women on POTS phones; their voices are typically pitched lower.)

So, what happens if one samples too slowly? That's where it gets interesting. This is complex, but bear with me, here.

Suppose one has an analog band-pass filter with resistors, capacitors, and such, and no sampling. Take a sine wave, slowly increase it in frequency, and, once one has passed the low-frequency edge of the bandpass filter, the filter passes the signal on through. Then, as one keeps on increasing the frequency of the signal, one hits the upper limit of the bandpass filter, the signal gets attenuated. And, as one goes to infinity on the frequency, the filter continues to kill the signal.

Now.. what if one is sampling? And there's no analog front end filter? Say one is sampling at 100 kSa/s and the digital bandpass filter is, say, between 1 kHz and 2 kHz. So, run the sine wave into the sampler, get digital numbers. Run the digital numbers through the digital filter, which is running at 100 kHz clock frequency. Take the output of the filter (still binary numbers, mind you), and run it into an digital-to-analog converter, also running at 100 kHz.

Start running the frequency up. Nothing below 1 kHz or so; signal gets through at 1.5 kHz; gets attenuated starting at 2 kHz. So far, so good. Now, keep increasing the frequency of the analog input. Keep an eye on the binary numbers after the sampler. Here's where it gets weird: When the analog signal passes 50 kHz, the binary numbers start looking like they're a sine wave dropping in frequency. This is the aliasing that I was mentioning. When one gets to 98 kHz which, if things were working like the designer intended, the signal wouldn't be present at all, at the output of the digital to analog filter, one will discover a 2 kHz sine wave, coming up in amplitude. At 98.5 kHz, it'll have a 1.5 kHz output at full strength; at 99 kHz, it'll start dropping again, just as if the input signal at that's at 99 kHz was actually at 1 kHz. At 100 kHz, the filter acts as if the incoming signal was at DC (0 Hz) and it doesn't pass.

It gets worse: At 101 kHz, it looks to the digital filter like it's a 1 kHz input and it passes through; at 102 kHz, it looks like a 2 kHz and passes through, and so on.

In theory, with ideal components, samplers, and so on, this repeats every 100 kHz ad infinitum. And this is why EE's building digital filters either put in anti-aliasing analog low-pass filters or sample at $DIETY's own rate where, for whatever reason, the frequency content of the incoming signal is never going to go.

It's not just laypeople that get tripped up on this stuff. I once had the joy and pleasure of explaining to a group of 10-15 or so EE's Who Shoulda Known Better that Aliasing Was A Thing; they were sampling at 1 Sa/s on a signal that had 50 Hz (and up!) signal components and were wondering why their whole highly precise phase locked loop was wandering off into never-never land. Some of them refused to believe me; I had to bring some rather specialized test equipment to their location, show their own people how to use it, then leave, lest I somehow make it fail on purpose. The light did dawn over there. Eventually.

So, back to blinking traffic lights. So, our heroes over at Tesla are trying, say, to detect a blinking red light. I have no idea how fast a blinking red light is supposed to blink; I suppose there's a standard somewhere. Let's say it's a nominal 1 Hz (0.5s on, 0.5s off), and it could be as low at 0.5 Hz (1s on, 1s off) or as high as 2 Hz (0.25s on, 0.25s off).

But.. there is a sample rate. That's how fast the vision is sampling the visual input. Hm. Once every 20 ms or so? That would be 50 Sa/s. Oh! City power is at 60 Hz in the U.S.. Yeah.. if there's no analog filter up front (response time of the cameras to photons) aliasing might truly be real, here. Especially if the LEDs in the red/green/yellow light are either running at 60 Hz (16.666 ms), or some $RANDOM frequency set by the people who built the lights.

And this problem would show up only if the internal sample rate of the Tesla just happened to be on a harmonic of the blink rate of the light. Whee!

Yeah, bet that's it. And it's going to be a bear to fix.

Um. How does one send a message to the development team at Tesla?
 
Is anyone else’s energy app completely broken by 11.4.2? It’s giving me absurdly high readings. This pic was taken mostly driving on 30 mph roads with minimal A/C. A scroll wheel reset brings it back to normal ~250 Wh/m (i did one right before the end of the graph), but then it goes back to this. Anyone having (or ever had) a similar problem?
FBFFECBC-62EE-46D2-95ED-D98166292D5D.jpeg
 
Ah. This is my field, digital signal processing. We may be looking at an aliasing problem.

Thing is, in DSP, when one samples, it's kind of important to sample (in theory) 2X as fast as the highest incoming frequency present. (Official name: The Nyquist Limit.) People who do this kind of thing for real with, say, digital oscilloscopes, typically sample at 3X or 5X the incoming frequency. A 1 GSa/s 'scope is typically rated for 200-250 MHz bandwidth. One can play silly buggers with this (Re: so-called, "Communications Analyzers"), but the general rule is at least 2X sample rate. Another obvious example is basic, POTS telephony: The audio sample rate is 8 kSa/s, but, to prevent aliasing, the audio before sampling is typically analog filtered to 2.5 kHz or so before the digital sample. (This is one reason why men tend to sound better than women on POTS phones; their voices are typically pitched lower.)

So, what happens if one samples too slowly? That's where it gets interesting. This is complex, but bear with me, here.

Suppose one has an analog band-pass filter with resistors, capacitors, and such, and no sampling. Take a sine wave, slowly increase it in frequency, and, once one has passed the low-frequency edge of the bandpass filter, the filter passes the signal on through. Then, as one keeps on increasing the frequency of the signal, one hits the upper limit of the bandpass filter, the signal gets attenuated. And, as one goes to infinity on the frequency, the filter continues to kill the signal.

Now.. what if one is sampling? And there's no analog front end filter? Say one is sampling at 100 kSa/s and the digital bandpass filter is, say, between 1 kHz and 2 kHz. So, run the sine wave into the sampler, get digital numbers. Run the digital numbers through the digital filter, which is running at 100 kHz clock frequency. Take the output of the filter (still binary numbers, mind you), and run it into an digital-to-analog converter, also running at 100 kHz.

Start running the frequency up. Nothing below 1 kHz or so; signal gets through at 1.5 kHz; gets attenuated starting at 2 kHz. So far, so good. Now, keep increasing the frequency of the analog input. Keep an eye on the binary numbers after the sampler. Here's where it gets weird: When the analog signal passes 50 kHz, the binary numbers start looking like they're a sine wave dropping in frequency. This is the aliasing that I was mentioning. When one gets to 98 kHz which, if things were working like the designer intended, the signal wouldn't be present at all, at the output of the digital to analog filter, one will discover a 2 kHz sine wave, coming up in amplitude. At 98.5 kHz, it'll have a 1.5 kHz output at full strength; at 99 kHz, it'll start dropping again, just as if the input signal at that's at 99 kHz was actually at 1 kHz. At 100 kHz, the filter acts as if the incoming signal was at DC (0 Hz) and it doesn't pass.

It gets worse: At 101 kHz, it looks to the digital filter like it's a 1 kHz input and it passes through; at 102 kHz, it looks like a 2 kHz and passes through, and so on.

In theory, with ideal components, samplers, and so on, this repeats every 100 kHz ad infinitum. And this is why EE's building digital filters either put in anti-aliasing analog low-pass filters or sample at $DIETY's own rate where, for whatever reason, the frequency content of the incoming signal is never going to go.

It's not just laypeople that get tripped up on this stuff. I once had the joy and pleasure of explaining to a group of 10-15 or so EE's Who Shoulda Known Better that Aliasing Was A Thing; they were sampling at 1 Sa/s on a signal that had 50 Hz (and up!) signal components and were wondering why their whole highly precise phase locked loop was wandering off into never-never land. Some of them refused to believe me; I had to bring some rather specialized test equipment to their location, show their own people how to use it, then leave, lest I somehow make it fail on purpose. The light did dawn over there. Eventually.

So, back to blinking traffic lights. So, our heroes over at Tesla are trying, say, to detect a blinking red light. I have no idea how fast a blinking red light is supposed to blink; I suppose there's a standard somewhere. Let's say it's a nominal 1 Hz (0.5s on, 0.5s off), and it could be as low at 0.5 Hz (1s on, 1s off) or as high as 2 Hz (0.25s on, 0.25s off).

But.. there is a sample rate. That's how fast the vision is sampling the visual input. Hm. Once every 20 ms or so? That would be 50 Sa/s. Oh! City power is at 60 Hz in the U.S.. Yeah.. if there's no analog filter up front (response time of the cameras to photons) aliasing might truly be real, here. Especially if the LEDs in the red/green/yellow light are either running at 60 Hz (16.666 ms), or some $RANDOM frequency set by the people who built the lights.

And this problem would show up only if the internal sample rate of the Tesla just happened to be on a harmonic of the blink rate of the light. Whee!

Yeah, bet that's it. And it's going to be a bear to fix.

Um. How does one send a message to the development team at Tesla?
The effect I referred to was not the flashing rate that you are intended to see, but the high frequency flicker rate that is above the human persistence. This paper discusses the issue.
 
The effect I referred to was not the flashing rate that you are intended to see, but the high frequency flicker rate that is above the human persistence. This paper discusses the issue.
I think he is saying that the high frequency flicker rate you are talking about is detected as a flashing red light to the Tesla Vision system because of the interaction with the sampling rate. (Which could explain why it happens at some but not all intersections, depending on the design of the LED stop light installed.) So, it stops, and then thinking it is a flashing red light, it proceeds as soon as it thinks it is clear/its turn.

Interesting possibility.
 
The effect I referred to was not the flashing rate that you are intended to see, but the high frequency flicker rate that is above the human persistence. This paper discusses the issue.
But that’s the point: the light is flickering at a rate faster than a human can perceive it: the car is sampling at a rate that violates the Nyquist limit for the system; and the offset in frequency between the two rates makes the aliasing happen, causing the digital filter in the Tesla to see a slowly blinking red light where there isn’t one.

We’re probably saying the same thing in different ways. And I’m not surprised that there would be a paper on the subject.
 
But that’s the point: the light is flickering at a rate faster than a human can perceive it: the car is sampling at a rate that violates the Nyquist limit for the system; and the offset in frequency between the two rates makes the aliasing happen, causing the digital filter in the Tesla to see a slowly blinking red light where there isn’t one.

We’re probably saying the same thing in different ways. And I’m not surprised that there would be a paper on the subject.
Yes, we are in agreement on the issue.
 
  • Like
Reactions: FSDtester#1
I noticed something today with HW4 and 11.4.2. On the exact same road, two lanes, one in each direction:

On a sweeping left turn, my Model S slowed down from 55 mph (the speed limit) to about 35 mph and balked with some intermittent phantom braking.

On the same curve, going the opposite direction (in which case, the turn is a sweeping right turn) my Model S did not slow down at all.

Visibility was sunny and clear, sun high in the sky, both directions.

It seems when FSD 11.4.2 is turning in the direction of oncoming traffic on sweeping turns (i.e., left sweeping turns) it slows down and balks, but when turning away from traffic (right sweeping turns) it runs perfectly.

I have tried this twice with exactly the same results. Now that this characteristic is ahem… on my radar (since my Tesla does not have radar, lol) I will pay more attention to other situations like this to see if this characteristic is repeated.

Joe
 
  • Informative
Reactions: FSDtester#1
I noticed something today with HW4 and 11.4.2. On the exact same road, two lanes, one in each direction:

On a sweeping left turn, my Model S slowed down from 55 mph (the speed limit) to about 35 mph and balked with some intermittent phantom braking.

On the same curve, going the opposite direction (in which case, the turn is a sweeping right turn) my Model S did not slow down at all.

Visibility was sunny and clear, sun high in the sky, both directions.

It seems when FSD 11.4.2 is turning in the direction of oncoming traffic on sweeping turns (i.e., left sweeping turns) it slows down and balks, but when turning away from traffic (right sweeping turns) it runs perfectly.

I have tried this twice with exactly the same results. Now that this characteristic is ahem… on my radar (since my Tesla does not have radar, lol) I will pay more attention to other situations like this to see if this characteristic is repeated.

Joe
I've noticed this as well. It's a regression in 11.4.X, IMO.

Although, I would argue that's obvious about the event it's braking for thus not phantom braking, but I know any braking event is referred to that here.
 
Not beeping during the initial blue flashing is a long-established behavior, not new. Also, if you let it go on long enough to get the red hands and an FSD disengagement, that's not generally a Strike. It happens to me maybe once or twice a week when I'm not thinking about the wheel torque enough. FSD will disengage but will become available again in about 30 to 60 seconds.

This has happened to me many many times, but I still have no Strikes, which is confirmed by visiting the Autopilot FSD menu.

There's a deeper Autopilot "jail" disengagement in which less for the remainder of the current drive. I'm not clear if that is 1:1 the same as a Strike on FSD. I think the only times that happened to me was on the highway on standard Autopilot (I avoided NoA by choice back then), when I drove too far above the speed limit during a passing maneuver. I had to actually get off the road, put it in Park, I get out of the car and start over. But that wasn't FSD beta.

Occasionally I've had FSDb disengage due to bright sun glare, and at least twice it wouldn't come back until I parked and started over. However those weren't driver penalty incidents, and (correctly) did not create FSD Strikes.
Up until recently it would start flashing, then start beeping, then put you in timeout. A timeout from ‘inattention’ in this manner absolutely was counted as a strike and the episode I had a week or two ago was counted as a strike (as evidenced by the notice that it had given me another strike after beeping at me once.

The behavior I noticed recently was different in that the speed/frequency of the flashing was such that it was approaching self disengagement that would have led to a strike, and in the past would have been accompanied by audible, warning beeps but in this case was completely silent.
 
Three observations in the last few days. Something must have changed on the back end NN.
  • FSD is doing much better handling fairly sharp secondary road curves by slowing down before the curve starts. Before FSD wouldn't slow down until it was too late and the car would go over the center yellow line. Especially noticeable on curves with elevation changes. Seeing continued improvement on the same curves over the last few days.
  • v11 broke an exit I take routinely by getting in the 3rd left hand lane as the car entered a large rotary. Just before entering the rotary controlled by a Stop light FSD would attempt to jerk over 2 lanes to the right hand lane. Not a chance with traffic so I always jerked the steering wheel very hard hoping Tesla might be looking at data with hard steering wheel changes. Previously FSD always exited and stayed in the right lane since the exit off the rotary was on the right. Yesterday and again today FSD took the exit properly after 25+ times of screwing the exit up. So Tesla must have done something on the back end.
  • FSD would make a lane change to get out of the right hand lane just before an exit only to have to move right back to the right lane. Just like the 2nd example I would make a really hard steering wheel change to move over and then give a voice note. Yesterday and today no unnecessary lane change.
Wondering if the harshness of the steering wheel "override" is being targeted by Tesla compared to the typical steering wheel adjustment? I have been using this approach for awhile now and several other repeatable FSD errors are now working ok. In other words using "jerky" steering wheel movements to get Tesla's attention. Probably just wishful thinking. You just have to be careful not to use this method with other cars near you.
 
  • Like
Reactions: FSDtester#1
Anyone else experiencing this side-to-side wobble? See video. It's pretty bad! It started with 11.3.6 and no change with recent update to 11.4.2

Only when the car is driving. I tried both FSDb or TACC same thing.
If I disengage I don't feel anything our of the ordinary. Even without touching the wheel the car goes straight as an arrow.

Below 50 mph hardly any wobble at all, but above 65 mph it gets really noticeable.
The faster I go and the more it wobbles. It's making my GF sick so this is serious!!

Only on AP/FSDb. Driving manual ... smooth as butter.

Had an alignment, made no difference.
Tires are in good shape; rear are new and front have lots of thread left... suspension inspected and all is tight ... no idea what to do :(

 
Up until recently it would start flashing, then start beeping, then put you in timeout. A timeout from ‘inattention’ in this manner absolutely was counted as a strike and the episode I had a week or two ago was counted as a strike...
I don't know what else to say, except that what you're describing is different from my experience. I've been put in inattention timeout (-1 minute) quite a few times, yet I have zero strikes. I've heard that two or three such events in one drive (or within some timespan) may generate a strike.

Some people have said that phone use generates more severe penalties, but that's not what you're talking about.

In any case, I certainly agree that the car should beep at you and give you a chance to respond before tarnishing you with a strike. Blue flashing alone, without even a sound, would be an unfair basis for a strike.
 
Anyone else experiencing this side-to-side wobble? See video. It's pretty bad! It started with 11.3.6 and no change with recent update to 11.4.2

Only when the car is driving. I tried both FSDb or TACC same thing.
If I disengage I don't feel anything our of the ordinary. Even without touching the wheel the car goes straight as an arrow.

Below 50 mph hardly any wobble at all, but above 65 mph it gets really noticeable.
The faster I go and the more it wobbles. It's making my GF sick so this is serious!!

Only on AP/FSDb. Driving manual ... smooth as butter.

Had an alignment, made no difference.
Tires are in good shape; rear are new and front have lots of thread left... suspension inspected and all is tight ... no idea what to do :(
Not good. You didn't mention performing a camera calibration, not sure that would help but it's worth a try. I haven't needed to do it at all, but it's an available procedure in the service menu.