Higher speeds and less time to react so we'll see. Hopefully they can squeak out significantly better performance than the current release.
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
FSD v12.x (end to end AI)
- Thread starter Buckminster
- Start date
-
- Tags
- #fixphantombraking 12.4.x is coming autopilot chill is not chill elon is an ass elon rocks fix fsd extreme late deceleration fsd does not commit to go fsd does not maintain proper speed fsd doesn't read signs properly fsd hard and late braking fsd has decision wobble fsd jerky driving fsd late deceleration fsd sometimes may not detect train crossings fsd stops too early too quickly fsd stops too far behind line fsd stops when there is zero visibility fsd tailgating in city streets fsd very slow at stops fsd will never enable l3 with hw4
FSDtester#1
Large Member
I just sleep in my car every night connected to Wi-Fi and tap advanced nonstop then move the slider from standard to advanced 200 times, power off the car, go in and out of service mode, and a few other things...
The ballz of my wife calling me OCD....
Agree, we just need Tesla to do something bold and enable Theatre mode on access controlled highways with Level 3. First phase towards L4/L5. It's refreshing to see more and more owners bringing up L3 for the highway.
A couple of challenged unprotected left turns.
Chuck's recent videos looked promising assuming they were running v12. They could also be dialing in HW4. I wonder if HW4 still qualifies for the beer challenge.
Chuck's recent videos looked promising assuming they were running v12. They could also be dialing in HW4. I wonder if HW4 still qualifies for the beer challenge.
PianoAl
Active Member
If the car is acting solely on the neural net, that is, on the behavior of other cars in similar situations, how are errors such as the "phantom gradual slowing" that AI Driver and I saw corrected?
....and what's so odd is since late v10's FSD has gotten consistently pretty good at these type of lefts.
sleepydoc
Well-Known Member
diplomat33
Average guy who loves autonomous vehicles
I would say it is a definite regression, likely caused by the V12 training data. It just shows that E2E does not automatically improve behavior, it will depend on the training data. Maybe Tesla is using data where the car is aggressive in making unprotected turns and so V12 is learning that behavior. Here's the thing though: that aggressive behavior may be appropriate in certain scenarios but not in others. Training V12 to know the difference between when to be assertive in a specific scenario and when not to be, is tricky.
In unprotected turns, you HAVE to be confident otherwise you will end up waiting forever
I think it should use other factors (slow neighborhood vs fast commuting road, speed limit, traffic density,...) to adjust the behaviors.
If I did not disengage during the unprotected right turn that I posted earlier I would not be able to read and post anything here. I like FSD but I am not willing to risk my life. When in doubt, don't do it.
diplomat33
Average guy who loves autonomous vehicles
Correct. But I think that goes to my point: training the end to end to take all these factors into consideration can be a challenge. It is not self-evident to me.
If it was trained from actual humans driving, maybe the phantom slowing is an actual “feature” of the neural net, end-to-end? How many people out there can drive by hand at an exact speed while also paying attention to other driving tasks at the same time? Perhaps one case where heuristics has an advantage.
stopcrazypp
Well-Known Member
I don't think it is viable to manually adjust weights because they don't necessarily correspond to clear concepts. One thing that can be done is to for example add a new input that takes road type as an input, but then it ceases to be E2E as you now have a separate module for that. But I think the whole point of E2E is to try to avoid doing manual adjustments like that. As long as the issue doesn't affect driving safety (which phantom slowing doesn't) I don't think there would be much effort expended on "fixing" it. There are much more serious deficiencies that need to be addressed.
OxBrew
Active Member
Dumb questions, no snark. If end to end is a black box NN, how does the engineer or PM or whoever has to sign off on it ever really validate behavior?
Re: crossing double yellow and trying to pass a line of cars at a stop sign, almost causing a head on collision (above):
1. Let's say somehow this behavior is "trained out," based on IDK, continued testing? How can anyone be sure it hasn't just been reduced enough that it's OK in the tests run, but there's still a latent condition in the NN that would trigger the behavior again, when IDK, the angle of the sun is different than all other times and some latent set of nodes gets happy and boom, does it again?
2. Why should I not be terrified to think that there could be 1000 latent conditions lurking in the NN, just waiting for that lucky day when the car drives me off the cliff? Really, no one can probe the NN for the exact function of every combination of inputs and outputs? How can anyone validate anything? What fundamental understanding am I missing?
Re: crossing double yellow and trying to pass a line of cars at a stop sign, almost causing a head on collision (above):
1. Let's say somehow this behavior is "trained out," based on IDK, continued testing? How can anyone be sure it hasn't just been reduced enough that it's OK in the tests run, but there's still a latent condition in the NN that would trigger the behavior again, when IDK, the angle of the sun is different than all other times and some latent set of nodes gets happy and boom, does it again?
2. Why should I not be terrified to think that there could be 1000 latent conditions lurking in the NN, just waiting for that lucky day when the car drives me off the cliff? Really, no one can probe the NN for the exact function of every combination of inputs and outputs? How can anyone validate anything? What fundamental understanding am I missing?
I know I can't keep a steady speed and so rely on cruise control to stop me from either speeding (if I'm in a hurry and traffic is light) or slowing down and impeding the traffic behind me (on a sunny day on meandering country roads.)
Unfortunately, TACC is not the cruise control for me because it decides the speed to go and doesn't consistently follow the speed I've set. I love that I don't have to disengage when coming up on traffic but last week, (holiday, sunny day, no traffic, divided highway) on TACC alone, not EAP, it slowed me from the set 114kph to 102 kph (speed limit on the road is 100kph) because it said the camera was obstructed. Best guess it was the sun or the salt spray off the road. Either way, I could see out the window just fine and much better than I can in the rain when the wipers REFUSE to come on but the car confidently runs on FSDb.
I'm one of the ones regretting not having 'dumb' cruise control on this car.
Note I'm running an older update, not the latest AP or FSDb firmware.
stopcrazypp
Well-Known Member
I don't know how it's done for cars or more non-deterministic functions like NNs, but from my experience for software in general, QA will have a set of tests that are run on every release. When there's a specific issue that is found, they add a test to test for that issue. That is how you validate if the issue is fixed and also to catch regressions. For an NN, perhaps the test would have to be run multiple times to get to a satisfactory probability.
I supposed if the issue is easily reproducible (like most of the time it triggers given a set of inputs) you can examine the nodes and figure which ones are responsible (similar to examining a call stack for traditional code debugging?). But given there would be thousands interlinked nodes, I don't think this is really viable task (unlike a call stack which is relatively simple in comparison). You might need another external module to translate those nodes to something that is human understandable (like the LLM experiments mentioned elsewhere), but that won't be 100% reliable either (as it's based on NNs itself).
I don't know how it would be done for E2E, but for traditional approaches there would be "rails" in terms of other functions that set a hard limit on what the car is allowed to do. But with an NN it's unfortunately hard to predict. Even for the L4 companies like Cruise and Waymo, you would think with the car loaded with sensors, it would be "easy" to set a function to "never" crash into something big and slow like a bus or a tow truck, but they did just that because of bugs in the code. When it happened to Cruise I thought it was just because Cruise has a subpar system, but then recently it happened to Waymo also (and two separate cars hit the same truck in the span of a few minutes!) So even in the best systems today, sometimes the stars can align and it does something completely unexpected.
Last edited:
stopcrazypp
Well-Known Member
@rlsd @OxBrew
Wayve (the company most known for E2E right now) has a blog post on debugging using simulators and they point out an issue of the car slowing down at a zebra crossing when there are no pedestrians present. They don't go into details of what they actually adjusted to fix this, but they do describe how they use simulation to test if the issue is fixed.
Ghost Gym: A Neural Simulator for Autonomous Driving
Wayve (the company most known for E2E right now) has a blog post on debugging using simulators and they point out an issue of the car slowing down at a zebra crossing when there are no pedestrians present. They don't go into details of what they actually adjusted to fix this, but they do describe how they use simulation to test if the issue is fixed.
Ghost Gym: A Neural Simulator for Autonomous Driving
