Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Tracking local maximum to FSD improvements

Mardak

Member
Oct 13, 2018
666
1,319
USA
It seems pretty clear there won't be much improvement to the current pre-rewrite Autopilot, so that gives an opportunity to see the limits of the local maximum. Hopefully people can share examples of where Autopilot requires interventions and ideally when Autopilot is able to consistently pass with the FSD rewrite.

I've got plenty of examples, so I wanted to have a reference of failures as once Autopilot is able to pass some of these, it'll be harder to differentiate from "of course Autopilot could handle this." I've tried to keep these to be "static" cases of just staying in the current lane without dynamic actors.

Curve with clear lines on both sides taken too fast/wide
curve wells kuenzli.jpg


Angled intersection where Autopilot wanted to switch from left to right lane when crossing
intersection angled 4th vine.jpg


Crested intersection where Autopilot panicked because it couldn't infer lines
intersection crested caughlin mccarran.jpg


Large intersection under the interstate where Autopilot starts to follow lines into the oncoming turn lane
intersection large plumb 580.jpg


Shifted lanes across a large intersection where Autopilot wanted to switch from the right lane to left
intersection shifted mccarran pyramid.jpg


Single lane splitting 3 ways where Autopilot wanted to take the onramp instead of staying straight
lane split 4th 580.jpg


A temporary gap in the lines and a dip then rise where Autopilot panics after incorrectly following the bike lane
line gap plumas manzanita.jpg


Roundabouts just aren't handled yet - this one is pretty simple with one lane
roundabout virginia center.jpg
 

Ludalicious

Active Member
Feb 22, 2018
1,105
1,193
Vancouver
Ahhhh have you actually read the release notes. No mention of autopilot working on city streets but looks like most scenarios are in the city.
 

Mardak

Member
Oct 13, 2018
666
1,319
USA
No mention of autopilot working on city streets but looks like most scenarios are in the city.
Yes, I realize these are all unofficial use cases, but Autopilot does actually work pretty well on local roads and staying in the current lane. I would estimate for each of the examples I shared so far, Autopilot went through 20 intersections just fine (and not just the simple 2-road crossing at 90°). Then again, with stop behavior for traffic lights and stop signs, one would expect some city street ability as that's where one would find those traffic controls.
 

Mardak

Member
Oct 13, 2018
666
1,319
USA
Do people expect all or most of the above examples to be addressed by the initial FSD rewrite/beta? I suppose they might be considered some of the easier aspects of FSD as I purposely chose examples that don't involve planning for other vehicles' movements. So in some sense, these examples are trivial for those that rely on HD maps as that information should be correct. Karpathy did show off intersection predictions with lanes and traffic flow directions, but unclear if this is intended for the initial release.
 

thewishmaster

Member
Jun 4, 2018
444
360
California
Most of your examples expose the rather naive algorithms currently used for lane following; they seem to check for some level of confidence in left/right lane markings and then bias the system to pick whichever left lane line is latitudinally closest to the car as the new left lane line when moving across intersections (or something like that). In the onramp case it probably is attracted to the sharply defined left curb on that on-ramp and the slight curve in the pavement markings that nudge it to take the curve instead of going straight.

I expect lane-following over ambiguous lane lines to improve. I'd be surprised if this isn't addressed as part of the rewrite because of how critical it is for city driving, although I half expect it to work "intelligently" only if you set a destination. Wouldn't be surprised if basic AP (or even FSD cars using simple lane-following without a destination) continues behaving the way it does today.

Not sure how much longitudinal control will improve for taking sharp curves, but I'd expect it to get better since the car will need to take 90+ degree turns.
 

Mardak

Member
Oct 13, 2018
666
1,319
USA
Most of your examples expose the rather naive algorithms currently used for lane following
Yeah, most of the examples I've posted so far are of the same family of problems of knowing what lane you're supposed to stay in. I'm not sure if Autopilot has an explicit algorithm or is just relying on neural network outputs, e.g., the thicker orange line in green's videos

I guess if people think lane following will be fully addressed with the initial FSD rewrite, are there other types of interventions I should share? E.g., parked cars with/without lane lines, behaviors around pedestrians/cyclists/animals. Once the rewrite is available, there might be some types of turning through intersection and yielding to other vehicles situations that the initial version fails and would be interesting to track.

Wouldn't be surprised if basic AP (or even FSD cars using simple lane-following without a destination) continues behaving the way it does today.
As in the current software runs for basic AP even after the rewrite? Or the rewrite could be smarter or at least behave differently when there's no destination -- just not guaranteed to "stay straight?"
 

thewishmaster

Member
Jun 4, 2018
444
360
California
I guess if people think lane following will be fully addressed with the initial FSD rewrite, are there other types of interventions I should share? E.g., parked cars with/without lane lines, behaviors around pedestrians/cyclists/animals. Once the rewrite is available, there might be some types of turning through intersection and yielding to other vehicles situations that the initial version fails and would be interesting to track.

Those are probably all would be good to note, although parked cars to me falls under the same "lane finding algorithm failures" category. It would also be interesting to see if there was any specific behavior for navigating around double-parked cars or cars that intrude into your lane; right now with the old algorithms you'd stop behind the obstruction (assuming it was even detected, as in the case of when only a small part of a car is sticking out into your path).

Generally reacting to behavior of other agents on the road like pedestrians, cars that make left turns in front of you, etc are probably good to observe as well. I've noticed some extra-panicky behavior recently if there are pedestrians near your lane, even if they're not even moving towards you.

As in the current software runs for basic AP even after the rewrite? Or the rewrite could be smarter or at least behave differently when there's no destination -- just not guaranteed to "stay straight?"

The former would be my guess, as I'd expect performance/reliability reasons for the rewrite not scaling down to HW2.5 well, which they would need to support for running basic AP in the older cars. Of course there's always the chance that they'd keep the feature set the same but fork the code for HW2.5 cars vs. HW3...
 

Mardak

Member
Oct 13, 2018
666
1,319
USA
It would also be interesting to see if there was any specific behavior for navigating around double-parked cars or cars that intrude into your lane; right now with the old algorithms you'd stop behind the obstruction
From watching FSD beta videos, looks like all of my shared examples and more including intruding parked vehicles are handled just fine. Perhaps we set very low expectations for what the initial early access release would be capable of.

When I get the FSD software probably as part of general wide release, I'll definitely retest my examples and more that I have saved but haven't shared yet. But I somewhat expect Autopilot to handle all of these cases of "stay in lane" now.

I wonder if this indicates Autopilot team has indeed been collecting disengagements and tested that FSD rewrite would pass them. So many of the cases that FSD would have trouble are when Autopilot hasn't been allowed to be enabled, e.g., turning left or right, so there might be issues with predicting destination lanes.
 

About Us

Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.

Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


SUPPORT TMC
Top