@diplomat33 gave a goal of about 1 intervention per 150k miles as a target rate for fsd "safety related" interventions, based on accident data. Regardless of what kind of interventions we choose to compare to this number, I think that it is overly strict. Every dangerous maneuver we make doesn't lead to an accident, in large part because other drivers will usually react to prevent the accident. It usually takes errors from both parties involved . So if we assume the combined probability can be equally attributed to both drivers, the probability of 0.000007 accidents per mile is the product of the individual probabilities of 0.00265 when they both make a mistake at the same time. That's an error rate of 1 in every 378 miles per individual. So, shouldn't fsd be held to this rate or better?
I don't think this works. We cannot assume the probability is equally distributed to both drivers. We don't know the percentage of errors that are fatal. We don't know the other driver will prevent the accident. Also, "safety related" interventions mean interventions that prevented a real accident. So it already factors in that the other driver did not prevent the accident. Plus, there are accidents caused by other drivers that you want your AV to avoid. It's called defensive driving.
I think your rate of 1 error over 378 miles is way too low.
Also, you don't want your AV to do any dangerous maneuvers, period. It is not ok for the AV to do a dangerous maneuver just because other drivers might avoid an accident. You want your AV to drive as safely as possible all the time.
I admit interventions might be a poor metric. There are too many unknowns.
So let's look at Mobileye's CES 2020 presentation that gives us some hard numbers:
The probability of human injury is 10^-4 injuries per hour of driving or 10,000 hours of driving per injury.
The probability of human fatality is 10^-6 fatalities per hour of driving or 1,000,000 hours of driving per fatality
So to match humans, FSD needs to do 10,000 hours of driving per safety error. Factoring in some margins, Mobileye multiplies it by 10 and concludes that you need 10M hours of autonomous driving without a safety critical error to be "good enough".
Source: Mobileye CES 2020