FSD rewrite will go out on Oct 20 to limited beta

[Lindamon]

Entirely irrelevant statement to your following paragraph, just saying.



The point is L4 cars will never be 100% but they will be 100x-1000x greater reliability/safety than human driven cars. (They already are) And by allowing humans to monitor and take over them will decrease safety.

You are right. My wife just showed me this video, and I had no idea automation was so advanced. I take it all back.

 

[EVNow]

I get that people want "levels" that are more descriptive of how good the FSD is. They want to know if the FSD can work where they live. But as I mentioned before, the problem is that there are too many parameters for ODD.

SAE, p14 defines ODD as "Operating conditions under which a given driving automation system or feature thereof is specifically designed to function, including, but not limited to, environmental, geographical, and time-of-day restrictions, and/or the requisite presence or absence of certain traffic or roadway characteristics. "

IMO, it would not be practical to try to create standards that cover all possible parameters.

That is an interesting idea. But I think coming up with 10 steps that cover everything and does not miss anything would be tricky.

In engineering you have to do it - saying something would be difficult or you might miss something is not an excuse.

The idea is to cover 90%/99% of conditions. "Practical" definitions - not theoretical. Afterall "E" in SAE stands for Engineers ?

 

[EVNow]

Wrong. This is not a problem with L4 def, not a problem with SAE levels at all.

This is a problem with you (and many many others) using SAE levels for purposes they are not meant to be used for.

Yes L4 could mean a single closed off 1km stretch road at 5mph on Fridays at 2-3pm.... And there is nothing wrong with that.

You are trying to use SAE levels to define/measure the capability of a system or how technically advanced a system is, but that is inappropriate and not the purpose of SAE levels.

So, what is the purpose of SAE definitions - and what practical questions are they answering ?

Its not a society of philosophers, afterall.

 

[diplomat33]

In engineering you have to do it - saying something would be difficult or you might miss something is not an excuse.

The idea is to cover 90%/99% of conditions. "Practical" definitions - not theoretical. Afterall "E" in SAE stands for Engineers ?

So, what is the purpose of SAE definitions - and what practical questions are they answering ?

Its not a society of philosophers, afterall.

The SAE levels are not theoretical. They answer practical engineering questions. They answer the question of what autonomous driving is from an engineering point of view. They answer the question of what driving tasks an automated driving system must be able to perform. They answer the question of who is responsible for each aspect of driving. They answer the question of what an ODD is. They answer the question of the difference between a driver assist and an autonomous driving system. And other questions as well.

 

[ItsNotAboutTheMoney]

I get that people want "levels" that are more descriptive of how good the FSD is. They want to know if the FSD can work where they live. But as I mentioned before, the problem is that there are too many parameters for ODD.

SAE, p14 defines ODD as "Operating conditions under which a given driving automation system or feature thereof is specifically designed to function, including, but not limited to, environmental, geographical, and time-of-day restrictions, and/or the requisite presence or absence of certain traffic or roadway characteristics. "

IMO, it would not be practical to try to create standards that cover all possible parameters.



That is an interesting idea. But I think coming up with 10 steps that cover everything and does not miss anything would be tricky.

We don't need any more.

The levels are useful, we know what they mean and we'd judge any particular instance on its merits.

 

[EVNow]

The SAE levels are not theoretical. They answer practical engineering questions. They answer the question of what autonomous driving is from an engineering point of view. They answer the question of what driving tasks an automated driving system must be able to perform. They answer the question of who is responsible for each aspect of driving. They answer the question of what an ODD is. They answer the question of the difference between a driver assist and an autonomous driving system. And other questions as well.

It’s quite clear they define some of these terms but don’t set any particular standards. They should revisit the levels. Currently every FSD company is ignoring the levels.

 

[diplomat33]

It’s quite clear they define some of these terms but don’t set any particular standards. They should revisit the levels. Currently every FSD company is ignoring the levels.

The last revision was in June 2018 so maybe they will publish a revision soon.

I would like to see better standards for AV safety. Maybe I missed the memo but it seems like everybody has different metrics on how safe an AV needs to be. We hear that AVs need to be safer than human driving but I have not seen anyone really put a solid number with that. So, I think it would be helpful if the SAE or regulators defined what exactly is "safe enough" for AVs. How many autonomous miles are needed to validate based on the ODD of the AV? How many accidents per miles is acceptable, broken down by types of accidents, at-fault accidents? Things like that. I think that could really help because AV companies would be on the same page in terms of what to aim for and when they can deploy AVs on public roads.

By the way, a consortium of companies published a non-binding document in 2019 called "safety first for automated driving" that does try to set practical standards for what AVs need to be safe on the road. It details what software and hardware an AV should have, redundancy that the AV should have, cyber security, testing and validation tests, etc... It's long and technical but it does spell out standards for how to engineer a safe AV.

 

[ItsNotAboutTheMoney]

The last revision was in June 2018 so maybe they will publish a revision soon.

I would like to see better standards for AV safety. Maybe I missed the memo but it seems like everybody has different metrics on how safe an AV needs to be. We hear that AVs need to be safer than human driving but I have not seen anyone really put a solid number with that. So, I think it would be helpful if the SAE or regulators defined what exactly is "safe enough" for AVs. How many autonomous miles are needed to validate based on the ODD of the AV? How many accidents per miles is acceptable, broken down by types of accidents, at-fault accidents? Things like that. I think that could really help because AV companies would be on the same page in terms of what to aim for and when they can deploy AVs on public roads.

By the way, a consortium of companies published a non-binding document in 2019 called "safety first for automated driving" that does try to set practical standards for what AVs need to be safe on the road. It details what software and hardware an AV should have, redundancy that the AV should have, cyber security, testing and validation tests, etc... It's long and technical but it does spell out standards for how to engineer a safe AV.

Safety standards are a policy matter, and a moving target, so there's little point trying to set a hard limit on miles and accidents per mile.

 

[diplomat33]

It’s quite clear they define some of these terms but don’t set any particular standards. They should revisit the levels. Currently every FSD company is ignoring the levels.

I would add that the purpose of the SAE document is not to tell companies how to do FSD but simply to define terms to make it easier to understand what FSD is. AV companies are not really ignoring the levels per se. They are simply focused on perfecting their driving product. The SAE levels help the consumer understand what that product is. For example, we can look at Tesla's AP and the SAE document tells us that it is L2. We can look at Waymo and the SAE document tells us that they have L4.

 

[diplomat33]

Safety standards are a policy matter, and a moving target, so there's little point trying to set a hard limit on miles and accidents per mile.

Society sets standards all the time for what is acceptable in the moment. Those standards can change over time but that does not mean that we don't bother setting any standards at all. For example, governments might pass a law setting the legal drinking age to X and then change it later. They still set a legal drinking age even though it might change in the future. So I don't see why we could not set a safety standard for AVs for what we consider acceptable now. It can always be changed later if we think it needs to be changed.

 

[mikes_fsd]

I would add that the purpose of the SAE document is not to tell companies how to do FSD but simply to define terms to make it easier to understand what FSD is. AV companies are not really ignoring the levels per se. They are simply focused on perfecting their driving product. The SAE levels help the consumer understand what that product is. For example, we can look at Tesla's AP and the SAE document tells us that it is L2. We can look at Waymo and the SAE document tells us that they have L4.

SAE International is a standards development organization geared toward the engineering industry. Focusing mainly on the transportation sectors, SAE creates standards for automotive, aerospace, mass transit, and commercial vehicles. Originally known as the Society of Automotive Engineers, SAE serves approximately 140,000 members from all around the globe who share the main goal of standardization. Standards published by SAE are not legally binding but are generally accepted by most industries and government agencies.

SAE while has a good goal of standardization, does not always create the best standards.
The problem is if govt's start using the SAE standard as a reference for regulation, and if the standard is crappy, you now have regulation built on a crap foundation.

 

[diplomat33]

SAE while has a good goal of standardization, does not always create the best standards. The problem is if govt's start using the SAE standard as a reference for regulation, and if the standard is crappy, you now have regulation built on a crap foundation.

Sure, regulation built on bad standards will probably be bad regulation. But I am not aware of the SAE creating bad standards. Do you have any specific examples other than you just don't personally like the SAE levels?

 

[john5520]

I would add that the purpose of the SAE document is not to tell companies how to do FSD but simply to define terms to make it easier to understand what FSD is. AV companies are not really ignoring the levels per se. They are simply focused on perfecting their driving product. The SAE levels help the consumer understand what that product is. For example, we can look at Tesla's AP and the SAE document tells us that it is L2. We can look at Waymo and the SAE document tells us that they have L4.

Absolutely. In the simplest terms, SAE levels tell us how automated the system is. That's information I'd like to know as a consumer. Combining the higher levels into one definition is too broad and opens the door for confusion and dishonesty.

 

[EVNow]

but simply to define terms to make it easier to understand what FSD is.

Its not DAE, its SAE. They are supposed to set standards, not just define them. BluRay Disk Association doesn't just define what a blue ray disk is or what a bit rate is - it defines the standards.

Complexity and difficult to define is not unique to FSD. Laws deal with it all the time - yet, laws just don't define various terms and leave everything else to people's discretion.

SAE (and in turn auto manufacturers) run the risk of individual countries starting to lay down conflicting regulations, if they don't follow up with proper standards. Remember all the "self regulation" industry wants is to ward off strict government regulations. But if the industry refuses to properly regulate itself, invariably the agencies have to step in.

 

[ItsNotAboutTheMoney]

Society sets standards all the time for what is acceptable in the moment. Those standards can change over time but that does not mean that we don't bother setting any standards at all. For example, governments might pass a law setting the legal drinking age to X and then change it later. They still set a legal drinking age even though it might change in the future. So I don't see why we could not set a safety standard for AVs for what we consider acceptable now. It can always be changed later if we think it needs to be changed.

You're talking about setting rules for businesses. Once set, the money focuses on it and becomes very hard to change it.

No problem sticking with the "show me" approach for now, that has allowed companies progressively to advance to remote Level 4.

 

[diplomat33]

Its not DAE, its SAE. They are supposed to set standards, not just define them. BluRay Disk Association doesn't just define what a blue ray disk is - it defines the standards.

Complexity and difficult to define is not unique to FSD. Laws deal with it all the time - yet, laws just don't define various terms and leave everything else to people's discretion.

SAE (and in turn auto manufacturers) run the risk of individual countries starting to lay down conflicting regulations, if they don't follow up with proper standards. Remember all the "self regulation" industry wants is to ward off strict government regulations. But if the industry refuses to properly regulate itself, invariably the agencies have to step in.

What "proper standards" do you want?

I know you complained about the ODD of L4. But I explained that it is not realistic to set standards for every ODD.

Do you want standards for safety like I suggested? Do you want standards about what hardware they can use?

I am just curious. Can you give a specific example of what standard you want to see?

Thanks

 

[EVNow]

SAE while has a good goal of standardization, does not always create the best standards.
The problem is if govt's start using the SAE standard as a reference for regulation, and if the standard is crappy, you now have regulation built on a crap foundation.

SAE being a consortium of manufacturers is supposed to come up with practical standards that can be implemented efficiently and inexpensively by its members. As such their standards, even if crappy, are what the manufacturers have agreed to comply with. So, regulations based on those are likely the lowest denominator types - is that what you mean by "crappy" ?

 

[EVNow]

What "proper standards" do you want?

I know you complained about the ODD of L4. But I explained that it is not realistic to set standards for every ODD.

Do you want standards for safety like I suggested? Do you want standards about what hardware they can use?

I am just curious. Can you give a specific example of what standard you want to see?

Thanks

I already gave examples. Its not rocket science ...

For eg., SAE (or its US affiliate) can say, FSD Level X is when
- It works in top 100 counties, by population
- In all weather conditions, except when certain enumerated emergencies occur​

And as you have noted, standards on testing would be useful too - but without ODD standards, you can't test.

To put this in old software terms, what SAE has now done is given a "conceptual framework". The next step is to give a "functional specification". For those saying SAE doesn't tell "how to implement" - absolutely, functional specs never tell "how" - but only "what".

 

[FloridaJohn]

 

[Thp3]

European regulation standards for self driving are too detailed and IMO unhelpful. For example, The requirements for changing lanes are time limited and turn signal limited. I think they set them too soon and with severe technology limits.
That’s one problem with setting standards before the technology is testable.
I worry that the SAE levels are equally unhelpful.

We have many implementations of SAE level 1 and SAE level 2. Have those helped any manufacturer or driver to date? If not, then what’s the value in SAE3 or SAE 4?

SAE 5 seems to be “Everything works, all the time, everywhere in any weather.” So I’m not seeing value there either since that’s likely physically impossible. Not even professional car drivers do that...