FSD Steering Characteristics Discussion

[AlanSubie4Life]

Moving this discussion from the Elon FSD tweets thread...this is a discussion about the behavior of the steering in FSD Beta (at the moment 10.4). My summary is that it jerks around a lot unnecessarily and the preceding discussion in the other thread was about whether or not this was a bug, and what the reasons were for the behavior.

I just tried to engage FSD for one right turn on the way home. It jerked the wheel so hard to the left that it instantly disengaged bc I was holding the wheel. It was an easy 10 mph right turn with no tricks. I have no idea where it was going turning left. That was the 0.25 sec of FSD engagement for me on my way home

Just remember, “it is not a bug.” And Elon says they have expended a lot of effort to minimize jerk, so here we are.

If you just let it twitch, nothing disastrous will happen.

How can you know that? I know for a fact it will violate traffic laws if you let it complete a twitch, in some situations. So I prefer to intervene when it makes incorrect decisions.

I know it seems like I'm saying that somehow my way of driving is best. I'm willing to concede that FSD may always take a slightly different path than I would prefer. But that is not what we're discussing here.

Yes - but there were no cinder blocks - but it was acting like there were. Thats why its a bug.

Agreed. I don't understand why we're discussing cinder blocks when it's clear we're discussing situations which are not challenging for FSD.

 

[diplomat33]

Yeah, I find that FSD Beta jerks the wheel a lot during turns like it is kind of feeling its way through the turn. Also, sometimes, coming to a complete stop a red light but where the car will need to make a turn, it turns the wheel before stopping instead of keeping the wheel straight and only turning when it starts the turn.

 

[AlanSubie4Life]

Arguably If someone who pays 10k for FSD got what they "expected" they wouldn't care at all what the wheel is doing, since they're not responsible for it and don't need to pay attention to it.


Of course that's not what those who paid 10k have gotten yet

Even if people got a system that truly was autonomous and all liability was on Tesla, I think they'd be pretty upset with the quality of the driving if the steering wheel behaved that way (or the steering rack, since the steering wheel would presumably disappear).

It would be extremely disconcerting.

But I suspect if we ever get to a system that is actually truly capable of autonomous driving (L4/L5), driving smoothly and without jerking the steering wheel (except in emergencies of course) will be a problem that is solved on the way to that end state.

 

[TIppy]

Yes - but there were no cinder blocks - but it was acting like there were. Thats why its a bug.

You miss the point. You said if you're turning right the wheel should never turn left. But it should if it's following a brief change in path. The blocks just make you program a brief deflection of the path in your head that you then follow, but you ultimately return to the business of turning right.

It's not a bug unless you consider the algorithm they use to determine the path a bug. It's going to have noise that causes brief deflections of the path. When starting the turn and travelling slowly, large steering angle changes are going to be required. Not a bug, it's working as designed.

You can complain it's a bad approach, but that's just the way it works.

 

[TIppy]

How can you know that? I know for a fact it will violate traffic laws if you let it complete a twitch, in some situations. So I prefer to intervene when it makes incorrect decisions.

The twitching is just a consequence of the combination of low forward speed and noise on the path. It's still going to follow the path. Now if the path is wrong, something disastrous could happen. But that's the case whether it twitches or not.

What traffic laws will it violate?

 

[AlanSubie4Life]

It's not a bug unless you consider the algorithm they use to determine the path a bug. It's going to have noise that causes brief deflections of the path.

If this is the reason for the steering behavior, yes, I consider that to be a bug. In general, it doesn't really help systems to add noise as you've described it (unless you can subtract it later, which can be done). It can "whiten" certain behaviors and patterns which can be helpful for some applications, but it will negatively impact SNR. Anyway, to me it appears the perception is just not good enough to have a low-noise path decision in the immediate proximity to the vehicle. Maybe the issue is that uncertainty in the perception at a distance results in an alternate path which ends up with a better cost function and then propagates back and changes the path close to the vehicle. That seems bad.

I don't remember all the details of what was explained on AI day about their cost functions and Monte Carlo driven tree of decisions or whatever, but the end result doesn't seem that great right now.

 

[AlanSubie4Life]

The twitching is just a consequence of the combination of low forward speed and noise on the path. It's still going to follow the path. Now if the path is wrong, something disastrous could happen. But that's the case whether it twitches or not.

What traffic laws will it violate?

In this case I'm referring to jerking on turns where the vehicle is moving. I have had FSD on multiple occasions jerk painfully into a prohibited area surrounded by double yellow lines (which are treated as solid barriers which should never be entered). Of course, when I override the steering by preventing the jerking behavior (meaning I hold the wheel and do not allow incorrect movements), the moving violation does not occur.

I'll go back and review the video of this occurrence and see what the pathing was doing during this event. If I have time I'll publish it.

 

[EVNow]

It's not a bug unless you consider the algorithm they use to determine the path a bug.

I have a broad definition of bug. I listed the two criteria. A lot of the bugs I've dealt with in the last 30 years have been where the users think its a bug - but its not a code bug but an analysis bug.

How do you know this? The car is designed to follow a path and that's what it's doing.

In this particular case - unless the FSD team is full of idiots - which I don't think is the case - its not definitely a design intent to be jerky. I'm sure the design intent is to be as smooth as an experienced human driver.

If the algorithms are producing jittery behavior they have to change the algorithm - not the requirement.

 

[TIppy]

If this is the reason for the steering behavior, yes, I consider that to be a bug. In general, it doesn't really help systems to add noise as you've described it (unless you can subtract it later). It can "whiten" certain behaviors and patterns which can be helpful for some applications, but it will negatively impact SNR. Anyway, to me it appears the perception is just not good enough to have a low noise path decision in the immediate proximity to the vehicle. Maybe the issue is that uncertainty in the perception at a distance results in an alternate path which ends up with a better cost function and then propagates back and changes the path close to the vehicle. That seems bad.

I don't remember all the details of what was explained on AI day about their cost functions and Monte Carlo driven tree of decisions or whatever, but the end result doesn't seem that great right now.

I'm not saying they are adding noise. I'm saying there is noise because of the algorithm there using and the drivable area that the NNs return. That's what causes the path to move about a little to the left and right of the current path.

 

[EVNow]

I'm not saying they are adding noise. I'm saying there is noise because of the algorithm there using and the drivable area that the NNs return. That's what causes the path to move about a little to the left and right of the current path.

They have to do something to "smooth" the path. Don't move until a path is determined with confidence and the car is about to start accelerating.

Since this is all procedural code - should be easy to fix / patch.

 

[_jal_]

If this is the reason for the steering behavior, yes, I consider that to be a bug. In general, it doesn't really help systems to add noise as you've described it (unless you can subtract it later, which can be done). It can "whiten" certain behaviors and patterns which can be helpful for some applications, but it will negatively impact SNR. Anyway, to me it appears the perception is just not good enough to have a low-noise path decision in the immediate proximity to the vehicle. Maybe the issue is that uncertainty in the perception at a distance results in an alternate path which ends up with a better cost function and then propagates back and changes the path close to the vehicle. That seems bad.

I don't remember all the details of what was explained on AI day about their cost functions and Monte Carlo driven tree of decisions or whatever, but the end result doesn't seem that great right now.

Yes. It's a bug at best; design fail at worst. Tesla wrote that we're responsible for a 5,000# machine doing potentially the worst possible thing at the worst possible time. I don't give a crap why it does what it does - if it can spin the wheel 90 degrees in a second and it can accelerate to 60 mph in 2.5 seconds, I'm hitting the brake when it does something I don't want it to do cuz I'm the one behind the wheel.

The way I explained FSD to my wife was that it's basically like a parrot - sure, it's wild that a parrot can talk, but it's not actually talking. The Tesla kind of drives, but it's not actually driving.

 

[TIppy]

In this case I'm referring to jerking on turns where the vehicle is moving. I have had FSD on multiple occasions jerk painfully into a prohibited area surrounded by double yellow lines (which are treated as solid barriers which should never be entered). Of course, when I override the steering by preventing the jerking behavior (meaning I hold the wheel and do not allow incorrect movements), the moving violation does not occur.

This is when the determined path is wrong. It moves abruptly to the side by a lot, so even if you are going fast, FSD has to input large steering angle changes to quickly reduce the error between the current path and the new erroneous path. This is different than the subtle noise on the correct path when you are going slow.

Your complaint is with the path planner in this case. The stochastic choice wasn't particularly good. Again not a bug, just the consequences of a non-linear minimizer with local minima. Yes, they are going to have to work on that.

 

[AlanSubie4Life]

They have to do something to "smooth" the path. Don't move until a path is determined with confidence and the car is about to start accelerating.

Since this is all procedural code - should be easy to fix / patch.

To me it's really unclear whether it is limited to the procedural code or whether there are issues with the probabilistic nature of the perception at a distance (or nearby for all I know - it seems to do poorly with visualizing crosswalks which are clearly visible which I don't understand) which result in path instability.

I explained FSD to my wife was that it's basically like a parrot - sure, it's wild that a parrot can talk, but it's not actually talking. The Tesla kind of drives, but it's not actually driving.

I had a similar discussion with my wife and she made it clear she wanted nothing to do with the microencephalitic parrot while she was in the car.

Your complaint is with the path planner in this case.

I don't understand how people can so glibly separate path planning from perception. I just don't understand how it's so clear to people that that's the issue, when it's far from clear to me that the perception is stable.

 

[_jal_]

This is when the determined path is wrong. It moves abruptly to the side by a lot, so even if you are going fast, FSD has to input large steering angle changes to quickly reduce the error between the current path and the new erroneous path. This is different than the subtle noise on the correct path when you are going slow.

Your complaint is with the path planner in this case. The stochastic choice wasn't particularly good. Again not a bug, just the consequences of a non-linear minimizer with local minima. Yes, they are going to have to work on that.

But it's so wrong. So. Wrong. Path planner. Planners of paths. Path planning daemon of whatever domain. It's just wrong.

Here I'm stopped. This is the predicted path

Or, you can look at the world and realize the path is ... a straight line forward.

 

[AlanSubie4Life]

But it's so wrong. So. Wrong. Path planner. Planners of paths. Path planning daemon of whatever domain. It's just wrong.

Here I'm stopped. This is the predicted path

View attachment 731262

Or, you can look at the world and realize the path is ... a straight line forward.

View attachment 731263

In this specific situation it seems like an issue with the path planning and not with perception. Did FSD try to shoot the gap in this situation or are you just showing an example of the weird stream of consciousness path planning? It's tough to thread the needle with a Model S.

 

[TIppy]

They have to do something to "smooth" the path. Don't move until a path is determined with confidence and the car is about to start accelerating.

Since this is all procedural code - should be easy to fix / patch.

It's not a static path. It's constantly projecting it's path forward at every instant. And it has to respond to an ever changing environment. Then they fit splines where points that come later can affect the interpolation between earlier point.

Don't worry. It's not an insoluble problem. It's just not a priority at this stage of development. I agree that eventually they will have to address this issue.

 

[nvx1977]

But it's so wrong. So. Wrong. Path planner. Planners of paths. Path planning daemon of whatever domain. It's just wrong.

Here I'm stopped. This is the predicted path

View attachment 731262

Or, you can look at the world and realize the path is ... a straight line forward.

View attachment 731263

FSD route planning:

 

[_jal_]

In this specific situation it seems like an issue with the path planning and not with perception. Did FSD try to shoot the gap in this situation or are you just showing an example of the weird stream of consciousness path planning? It's tough to thread the needle with a Model S.

It's a two lane (each way) road. Addison Street in Chicago. I'm stopped at a light behind a car that is letting a car in from a driveway from the strip mall. We're actually not moving at all. For some reason - I have no idea why - it is trying to move me through a crossover and past the Civic into the Malibu or whatever that is. The lane lines are faint at best and I thought "well, maybe it wants me to merge left?" but then why plan to swerve right again?? It's a video and I don't want to do the whole YouTube thing again, but I'm just sitting there and my car is desperately trying to thread the needle while what it should do is ... just ... roll ... with traffic through the intersection

 

[TIppy]

I don't understand how people can so glibly separate path planning from perception. I just don't understand how it's so clear to people that that's the issue, when it's far from clear to me that the perception is stable.

How have I said path planning and perception are separate. In fact I said the opposite. They clearly are interrelated. You can't plan the path if you don't have a drivable area to navigate. And since the shape of that drivable area has noise on it, that noise will affect the path planner. But the path planner is definitely a different object than what generates the drivable space. The just interact.

 

[AlanSubie4Life]

Your complaint is with the path planner in this case.

This is why I referred to "glibly separating path planning from perception" - meaning, I didn't think you thought they were separate, but I found it odd that you were able to easily separate the two and assign blame to the path planner.

How have I said path planning and perception are separate.

I didn't say you said that. Again, what I meant by "glibly separating" was not separating them from one another in the sense that you think they're independent - I meant separating them in the sense that you decided to assign blame to the path planner - I don't get how you can say that's my complaint so confidently. Yeah, maybe the error in the path is what causes the wheel turn, but that doesn't necessarily mean that the path planning is the problem - why is the path wrong? You could have a perfect path planner and it would provide incorrect results if the perception is wrong.

I'm not saying the path planner is great, BTW.