run-the-joules
Turgid Member
I'm not sure I agree. If you're paying attention then you'll know if the car should be taking any action and you'll either allow it to do so or you'll take over.
Please re-read the last sentence of the multi-sentence paragraph. Paying attention doesn't mean you see everything, and there's plenty of fun to be had demonstrating this in various ways.
On this subject, though, consider the following:
Condition 1: "We repeatedly have to override the car because it does a dumb thing in the same way regularly when there is no real threat" - this is already the case.
Condition 2: "The meat bag behind the wheel, despite being quite alert and attentive, doesn't notice something happening that could be avoided by braking, such as a car swerving from three lanes over, across the freeway, towards a space that will be occupied by the Tesla in a moment" - this is common because humans are highly fallible morons, as a species.
One of these on their own isn't the end of the world.
Just Condition 1 occurring doesn't hurt anything.
Just Condition 2 occurring doesn't hurt anything if the computer notices and reacts appropriately.
Condition 1 AND 2 occurring means that the idiot behind the wheel might literally *accelerate into an accident that the car was trying to avoid* because every other time the car braked suddenly, it was for a dumb reason, and the human is now trained to override sudden braking events.