What does your timeline look like for driverless vehicles?

[Daniel in SD]

Stop with the "cultural" BS. Lot of other standards have safety built in - including tolerances allowed, MTBF etc.

The fact is the standards are setup in such a way that the "test" can be done by a 3rd party in a day. That is what thinking "inside the box" looks like.

What is stupid is not just the levels - but defending them as well.

Most things it possible to define safety specs that are achievable at low enough cost and with so much margin to make cultural differences irrelevant.
Road safety is different in every country as are crash safety standards.
I guess I just have no idea how to define a universal standard much less a way to test it.

 

[EVNow]

Thanks for the detailed take on my post which never considered that!

Reconsider - because that is exactly what Waymo & Cruise are doing. I'm not saying anything "outside the box".

 

[EVNow]

Most things it possible to define safety specs that are achievable at low enough cost and with so much margin to make cultural differences irrelevant.

MTBF is well established. A number of standards call for minimum MTBF levels or MTBF to be displayed on the product. Like simple light bulbs or switches. SAE should be well aware of it since the industry tracks MTBF of literally every part they use.

 

[gearchruncher]

because that is exactly what Waymo & Cruise are doing. I'm not saying anything "outside the box".

It's exactly what Tesla is doing too. The issue is that it's very far away from answering the question if the system is "5X to 10X better than humans in the ODD with real world testing" as I detailed in my post and that you suggest is the correct baseline.

 

[diplomat33]

So we put safety drivers in. Now they take over before the collision, and we blame them if a collision happens. So what's our rate without the human? We have no idea.

By not letting the drivers allow a collision to happen, we also don't get any severity of collision data. What if autonomy is good at saving lives, but actually causes WAY more property damage? Like Fatalities move from 1:90M miles to 1:500M miles, but in the process, we have $1T a year in "fender benders" and non-fatal injuries? What do we even mean by "better than humans at the ODD" when their is not just one singular outcome to a collision?

This is why we keep coming back to intervention rate. I'm at the "show me that the human backup didn't decide to intervene more than 1:1M miles across a statistically valid sample of at least 100M miles and all 4 seasons covering the area you will release in, and then let's talk about real world rate testing"

That's why companies like Waymo run simulations to see what would have happened if the safety driver had not disengaged. That way they can see if the disengagement was needed to prevent a collision or not and what the severity of the collision would have been. You must do this analysis to actually measure the safety of the AV without a safety driver.

 

[gearchruncher]

MTBF is well established. A number of standards call for minimum MTBF levels or MTBF to be displayed on the product. Like simple light bulbs or switches. SAE should be well aware of it since the industry tracks MTBF of literally every part they use.

Nothing you reference is tied to safety criticality. MTBF of an individual item in a system tells you nothing about the system's performance or expected outcome when that item fails, much less against specific failures which could have very different outcomes (think airbag not going off vs going off when it wasn't supposed to).

Still looking for one of these standards that sets out a minimum "MTBF" when "failure" is the same as death. Never once ran into a safety standard that uses the term MTBF to evaluate the rate of human death.

 

[gearchruncher]

That's why companies like Waymo run simulations to see what would have happened if the safety driver had not disengaged.

Agree, and this is an awesome next step in evaluating the system. It doesn't meet @EVNow's goal of "real world testing" though.

 

[EVNow]

Do you have an example of an international standard, widely used across areas beyond the US, Canada, and EU that includes absolute injury rates as part of the standard?

Checkout all the standards related to airbags. Compare the details to what is available with "levels".

Inflatable Restraints Committee - Profile - SAE International

profiles.sae.org

 

[EVNow]

Agree, and this is an awesome next step in evaluating the system. It doesn't meet @EVNow's goal of "real world testing" though.

The testing is RealWorld. To figure out what would have happened if not disengaged would have to be simulation or manual. Duh.

ps : Your answer to "real world testing but possibly imprecise analysis of disengagements" seems to be "don't test at all". Crazy.

 

[diplomat33]

Checkout all the standards related to airbags. Compare the details to what is available with "levels".

Inflatable Restraints Committee - Profile - SAE International

profiles.sae.org

The "levels" are not the only standard we have. They are simply a taxonomy that defines types of autonomous driving. There are other standards and best practices for AV safety.

For example: http://go.sae.org/rs/525-RCG-129/im...5G66TS0BzITZUR8UPdf5KUeXbGL5mqxbKwtI8kBYx5T4Q.

 

[gearchruncher]

ps : Your answer to "real world testing but possibly imprecise analysis of disengagements" seems to be "don't test at all". Crazy.

Only for someone not reading.

This is why we keep coming back to intervention rate. I'm at the "show me that the human backup didn't decide to intervene more than 1:1M miles across a statistically valid sample of at least 100M miles and all 4 seasons covering the area you will release in, and then let's talk about real world rate testing"

 

[gearchruncher]

Checkout all the standards related to airbags. Compare the details to what is available with "levels".

You sent us to a page of 26 standards, without any guidance to which ones specifically indicate a rate of allowed failure tied to human safety. Asking the reader to pay $87 each to read them isn't really helping your point that you know of standards that define acceptable MTBF when failure=human harm.

Also, SAE is an industry standard than can be used, it does not mean it IS used everywhere. Is adherence to these SAE standards required in India, China, Russia, etc where different risk profiles may be used?

This all started because you are arguing different cultures don't have different safety thresholds and that international standards are common. Yet Mexico didn't even require airbags at all until 2019, so the existence of an SAE standard for airbags was totally irrelevant for the 20 years between the USA making them mandatory and Mexico making them mandatory...