It kind of sounds like the code trying to decide if you're a safe driver is at best the same pile of neural nets that you're supposed to be monitoring, and at worst, even worse at judging correctness. So doesn't that completely defeat the purpose of doing this analysis in the first place?
It seems like what they should really be doing is awarding FSD access to the people with the largest number of legitimate disengagements. And if you can't determine if the disengagements are legitimate (e.g. by running some reasonable portion of the FSD stack in parallel), then it probably isn't good enough to release yet.
For example, consider hard braking:
- To avoid a collision, stopping within a short distance of another vehicle or obstruction: driver error.
- To maximize regen before coasting slowly up to another vehicle or obstruction: desirable driver behavior.
- Because you didn't slow down for the yellow light that just turned red: driver error.
- Because Autopilot thought a flashing yellow light on the freeway was a traffic light: car error; bonus points for driver quickly stomping the gas pedal to avoid getting rear-ended.
- Because Autopilot in highway mode thought another vehicle cut into your lane: car error; bonus points for driver quickly stomping the gas pedal to avoid getting rear-ended.
And unfortunately, the legitimate causes of hard braking greatly exceed the driver error causes, both in frequency and in importance, so if they can't do better at weeding out the noise, they should drop that one entirely.
Then again, we MCU1 users are still swearing under our breath wondering if they'll ever even give us a button to push, so... whatevs.