What is SAE Level 5 and can Tesla actually achieve it?

[S4WRXTTCS]

I don't think this is the intent of level 5. If the level 5 feature detects a blocked or possibly malfunctioned sensor, it can pull over or not allow activation of the level 5 feature.

Again, the SAE definition doesn't outline any performance or safety criteria for level 5. I think this is the most common misunderstanding. Since you are using "human" as a performance criteria, this is outside of the SAE definition. The SAE definition uses "driver-manageable" (your wording "human-manageable" is not used) to simply refer to roads that are publicly accessible by a human driver. See the wording below (emphasis always mine). Level 5 is about operation, not performance:

Source: “Unconditional/not ODD-specific” means that the ADS can operate the vehicle under all driver-manageable road conditions within its region of the world. This means, for example, that there are no design-based weather, time-of-day, or geographical restrictions on where and when the ADS can operate the vehicle.

I just want to make it clear that I don't like the SAE definitions. I'm just using this thread to sort out the level 5 misconceptions.

I think you're doing mental gymnastics to force a unique interpretation onto the Level 5 definition.

Obviously I meant driver manageable when I said human manageable. That the driver is a human being, and this is the measuring stick we should use.

All driver-manageable road conditions IS a performance criteria. That human drivers are the bench mark.

The primary difference between L4, and L5 is ODD.

That L4 is allowed to have restrictions based on weather, road type, etc.

L5 is supposed to allow engagement in any ODD that is driver-manageable.

In your interpretation of the SAE Levels what is the purpose of L5?

 

[S4WRXTTCS]

L5 is not expected to drive in conditions that are considered unsafe for human drivers like a flood or severe storm. I don't think anyone is saying that L5 does not need to handle a drizzle.

Under the SAE Level 5 a serious storm would be considered outside driver-manageable, and not expected from an L5 system.

The OP is saying that SAE Level 5 can be significantly more weather restricted than a human driver, and I disagree with that.

The simple fact is that a Tesla HW3 cannot deal with weather that humans can easily deal with, and thereby will never be an L5 vehicle by the SAE definitions.

 

[powertoold]

The OP is saying that SAE Level 5 can be significantly more weather restricted than a human driver, and I disagree with that.

Please show me where I said that. I was referring specifically to snow or mud on one or more of the sensors.

Please do try to show the sae wording to support your views. Level 5 means that no ODD is designed into the level 5 feature, once it's activated. However, Tesla software can prevent the level 5 feature from being activated if it senses that a sensor is obscured.

I'll give you an example. Say you parked your level 5 capable car in the snow. The snow covers a sensor. When you get in the car, the level 5 feature can't be enabled. This is still level 5.

Also, even while it's driving in level 5 mode, if a sensor becomes covered in mud. The car can pull over. This is still level 5.

 

[powertoold]

That the driver is a human being, and this is the measuring stick we should use.

This is totally wrong. Show us where the SAE definition uses human capability or performance to assess the performance of level 5. This is specifically why the SAE uses driver manageable instead of human manageable. There's a semantic difference here.

 

[S4WRXTTCS]

Please show me where I said that. I was referring specifically to snow or mud on one or more of the sensors.

Please do try to show the sae wording to support your views. Level 5 means that no ODD is designed into the level 5 feature, once it's activated. However, Tesla software can prevent the level 5 feature from being activated if it senses that a sensor is obscured.

I'll give you an example. Say you parked your level 5 capable car in the snow. The snow covers a sensor. When you get in the car, the level 5 feature can't be enabled. This is still level 5.

Also, even while it's driving in level 5 mode, if a sensor becomes covered in mud. The car can pull over. This is still level 5.

I don't disagree that Tesla can prevent the Level 5 feature from being activated if a sensor is obscured by snow or isn't working correctly. There isn't any disagreement there. Obviously a human can't drive if the windshield is covered in snow.

But, if I get into my car while its raining in Seattle and the car says "Nope, no can do" then by SAE definitions its not L5 as obviously rain (to a point) is a driver manageable situation. Or if it pulls over during a drive due to obscured sensors because of a little bit of rain on a consistent basis then I would say it's not L5 as I would see those failures as design limitations.

 

[S4WRXTTCS]

This is totally wrong. Show us where the SAE definition uses human capability or performance to assess the performance of level 5. This is specifically why the SAE uses driver manageable instead of human manageable. There's a semantic difference here.

This is the specific block of text were interpreting.

"“Unconditional/not ODD-specific” means that the ADS can operate the vehicle under all driver-manageable road conditions within its region of the world. This means, for example, that there are no design-based weather, time-of-day, or geographical restrictions on where and when the ADS can operate the vehicle. However, there may be conditions not manageable by a driver in which the ADS would also be unable to complete a given trip (e.g., white-out snow storm, flooded roads, glare ice, etc.) until or unless the adverse conditions clear. At the onset of such unmanageable conditions the ADS would perform the DDT fallback to achieve a minimal risk condition (e.g., by pulling over to the side of the road and waiting for the conditions to change"

From this text my definition for driver manageable is that it can be reasonably assumed that a licensed human driver can handle the road or situation.

The SAE Primer on posted on TMC has this example:

"Example of L5 ODD: a robotaxi that can operate on all driver manageable roads in the US, day or night, all weather that human can handle, all legal speeds."

So right there it has "weather that human can handle" as a benchmark.

What's your definition for driver-manageable?

 

[S4WRXTTCS]

or if a sensor is blocked, the car can simply pull over. A blocked/broken sensor isn't considered an ODD limitation. It could be defined as a system failure.

I might disagree with you, but I'm looking forwards to late 2023 when thousands of make believe L5 Tesla's are sitting on the side of I5 waiting for small rain shower to clear up before continuing on.

If I had to pull over every time I got the "sensor obscured" message I would never make it anywhere.

 

[diplomat33]

The SAE Primer on posted on TMC has this example:

"Example of L5 ODD: a robotaxi that can operate on all driver manageable roads in the US, day or night, all weather that human can handle, all legal speeds."

So right there it has "weather that human can handle" as a benchmark.

What's your definition for driver-manageable?

By the way, I created the primer awhile ago. "weather that human can handle" is my phrase, not a direct quote from the SAE document. It is based on this sentence in the SAE document:

"there may be conditions not manageable by a driver in which the ADS would also be unable to complete a given trip (e.g., white-out snow storm, flooded roads, glare ice, etc.) until or unless the adverse conditions clear."

So "driver-manageable" would basically be the inverse of this, ie conditions that a driver could be expected to manage.

Also, here is the example of L5 that the SAE document provides:

"A vehicle with an ADS that, once programmed with a destination, is capable of operating the vehicle throughout complete trips on public roadways, regardless of the starting and end points or intervening road, traffic, and weather conditions."

I think if we combine the two sentences from the SAE document (just adding the word "however" to connect the two), we get a more complete description for L5:

"A vehicle with an ADS that, once programmed with a destination, is capable of operating the vehicle throughout complete trips on public roadways, regardless of the starting and end points or intervening road, traffic, and weather conditions. However, there may be conditions not manageable by a driver in which the ADS would also be unable to complete a given trip (e.g., white-out snow storm, flooded roads, glare ice, etc.) until or unless the adverse conditions clear."

 

[diplomat33]

FSD beta is a level 5 capable feature in testing.

Do you mean FSD Beta has the potential to be L5? Maybe. But FSD Beta is not L5 in testing.

From Section 5.6 page 25 of SAE document:

"NOTE 2: In the event of a DDT performance-relevant system failure(of an ADS or the vehicle), a level 5 ADS automatically performs the DDT fallback and achieves a minimal risk condition.

NOTE 3: The user does not need to supervise a level 5 ADS, nor be receptive to a request to intervene while it is engaged."

FSD Beta does not meet either of these big L5 requirements yet.

 

[mikes_fsd]

Do you mean FSD Beta has the potential to be L5? Maybe. But FSD Beta is not L5 in testing.

SAE FSD levels are design intent - you are the one that pointed that out!
We know Tesla's design intent for FSD is L5.
Therefore, Tesla is testing L5 designed FSD features.

 

[diplomat33]

SAE FSD levels are design intent - you are the one that pointed that out!
We know Tesla's design intent for FSD is L5.
Therefore, Tesla is testing L5 designed FSD features.

Tesla may intend FSD Beta to be L5 at some point. But FSD Beta is NOT L5 now.

Design intent still needs to meet the SAE definitions.

 

[mikes_fsd]

Tesla may intend FSD Beta to be L5 at some point. But FSD Beta is NOT L5 now.

Design intent still needs to meet the SAE definitions.

LMAO - this is why discussing SAE or really any standard is useless.
moving on.

 

[mikes_fsd]

Contrast your response above with this response you wrote just earlier in this thread.

I think the SAE document would disagree with you.

Section 8.2, page 30 says:

"The manifestation of one or more performance deficiencies in either the driving automation system or in the user’s use of it does not automatically change the level assignment. For example: An ADS feature designed by its manufacturer to be level 5 would not automatically be demoted to level 4 simply by virtue of encountering a particular road on which it is unable to operate the vehicle. "

So the SAE says that if a L5 system encounters a road or situation it can't handle, it is still L5.

Design intent is the same for Tesla FSD!

 

[powertoold]

Do you mean FSD Beta has the potential to be L5? Maybe. But FSD Beta is not L5 in testing.

You're right. But I think you know what I mean, lol.

As for the design intent, I think most people here misunderstand that part as well.

 

[powertoold]

From this text my definition for driver manageable is that it can be reasonably assumed that a licensed human driver can handle the road or situation.

We would interpret it that way if the SAE didn't say the definitions don't include any performance or safety criteria. They even say that level 5 is about design intent, not actual performance. So it's very clear we can't compare level 5 to a human.

A dumb analogy is the definition of a "box." As long as it has 6 rectangular sides, then it's a box, no matter the material, design on it, or craftsmanship. This is what the SAE definitions are trying to do: provide a way to describe automation features without inferring their performance.

 

[diplomat33]

Contrast your response above with this response you wrote just earlier in this thread.

Design intent is the same for Tesla FSD!

That quote is talking about how an ADS does not change levels after the fact just because it can't handle something. So if Tesla does deliver L5, it would not suddenly get downgraded to L4 just because it can't handle something.

But AFAIK, "design intent" is about current design, not some future goal. Just because the manufacturer wants to achieve Level X in the future, does not make it level X.

Tesla certainly intends FSD Beta to be L5 some day, sure. But the fact that Tesla tells FSD Beta testers that they must keep their hands on the wheel, supervise at all times and be prepared to take over at any time, indicates that the current design intent of FSD Beta is L2.

Similarly, Waymo say that they are working to make Driver L5 some day. But the current design intent of the Waymo Driver is L4 because the Waymo Driver is currently designed to operate in a geofenced area.

@Needsdecaf @Bladerskb I trust you guys. You are well informed on these matters. If I am wrong, please let me know.

 

[powertoold]

But AFAIK, "design intent" is about current design, not some future goal. Just because the manufacturer wants to achieve Level X in the future, does not make it level X.

That's my understanding as well.

If the feature is currently designed to have no ODD and doesn't expect the user to intervene, then it's level 5, regardless of its performance.

It's a bit of a gray area with FSD beta. I don't think the fsd beta is designed (in the software) to expect intervention. However, Tesla expects testers to intervene and disengage (in agreements or coaching). But yes, I still think this is level 2.

 

[Needsdecaf]

Contrast your response above with this response you wrote just earlier in this thread.

Design intent is the same for Tesla FSD!

I am really not understanding this point or this whole argument, on either side.

Please, help me understand, what "Design Intent" has to do with anything?

 

[S4WRXTTCS]

is my phrase, not a direct quote from the SAE document

That's a good clarification.

My intent was to show how the block of text was being interpreted. I felt like your interpretation of it was correct at the time, and I still feel the same way.

If you feel like your interpretation was wrong on the primer then please explain why it was wrong.

 

[diplomat33]

That's a good clarification.

My intent was to show how the block of text was being interpreted. I felt like your interpretation of it was correct at the time, and I still feel the same way.

If you feel like your interpretation was wrong on the primer then please explain why it was wrong.

No, I feel like my interpretation on the primer is still good. It might be a bit simplistic but I think it captures the meaning of the SAE document.