What is SAE Level 5 and can Tesla actually achieve it?

[powertoold]

Many people, including myself, have strong opinions on level 5, but few have attempted to read and understand the SAE definitions.

Here, we discuss whether or not Tesla can achieve level 5 in the near future, or is it impossible (given the sensor suite and/or feature limitations)?

To create a stronger argument, please source your views or disagreements from actual SAE wording, as I will below.

Link to SAE document: surface vehicle recommended practice - SAE Mobilussaemobilus.sae.org › download

My views on level 5 and Tesla:

1) FSD beta is a level 5 capable feature in testing.

2) Tesla will enable a level 5 feature in at least one of their cars by end of 2023, pending regulatory approval.

3) I say pending regulatory approval because level 5 has no geographical limitations, so the level 5 capable car has to be approved USA-wide.
Source: 5.6 LEVEL or CATEGORY 5 - FULL DRIVING AUTOMATION: “Unconditional/not ODD-specific” means that the ADS can operate the vehicle under all driver-manageable road conditions within its region of the world. This means, for example, that there are no design-based weather, time-of-day, or geographical restrictions on where and when the ADS can operate the vehicle.

Important aspects of level 5 pertaining to Tesla's ability to achieve it:

1) A level 5 feature only pertains to public roadways and doesn't involve drive into or backing out of your driveway.
Source: 1. SCOPE ... “On-road” refers to publicly accessible roadways (including parking areas and private campuses that permit public access)

2) When Tesla has a driverless robotaxi service, a dispatcher (i.e., Tesla employee) can decide when to allow service operation. This means that if Tesla forecasts a snow storm, forest fire, hurricaine coming, they can prevent the dispatch of or recall their robotaxis. This still fulfills level 5.
Source: Table 2 - Roles of human driver and driving automation system by level of driving automation
Level 5 - Full Driving Automation
Driver/dispatcher (while the ADS is not engaged):
• Determines whether to engage the ADS

3) Level 5 doesn't define safety or performance parameters. Whether or not a car is level 5 is based on its unlimited ODD and lack of expectation to intervene. However, a feature may still be level 5 if the user intervenes by choice, if steering wheel and controls are still available.
Source: Table 1 - Summary of levels of driving automation
Level 5 Full Driving Automation
The sustained and unconditional (i.e., not ODD- specific) performance by an ADS of the entire DDT and DDT fallback without any expectation that a user will respond to a request to intervene.
Source: 8.1 J3016 is not a specification and imposes no requirements.
By itself, J3016 imposes no requirements, nor confers or implies any judgment in terms of system performance.

4) The level 5 definition doesn't require a vehicle to fuel itself, so the user can only select a destination within the available battery range. This isn't an ODD. This also means a cross-country trip isn't a requirement to achieve level 5.

5) Level 5 has nothing to do with human capability or not. It's simply a car with a feature like FSD beta that:
a. Has no designed restrictions
b. Does not expect the user to intervene
c. Has fallback mechanisms to pull over

 

[ThomasD]

The question is Will Tesla be willing to take on the full liability risk of levels 4 and 5

 

[Bladerskb]

Many people, including myself, have strong opinions on level 5, but few have attempted to read and understand the SAE definitions.

Just dropping this here aswell.
Primer on SAE Levels of Autonomy

 

[S4WRXTTCS]

This means, for example, that there are no design-based weather, time-of-day, or geographical restrictions on where and when the ADS can operate the vehicle.

I find it pretty easy to rule out L5 with the existing HW3 simply due to obstruction of the sensors due to weather conditions. Whether its rain, light snow, light fog, or even sun at certain angles.

The SAE Level 5 is really clear that the ODD can't be anymore limited than driver-manageable road condition.

Here is the exact line:
“Unconditional/not ODD-specific” means that the ADS can operate the vehicle under all driver-manageable road conditions within its region of the world.

Being limited to L4 is no big deal to me as I'm not a fan of L5.

Everyone has a different ODD so what defines driver-manageable? Some median of the people? If your car is from Cali does that mean it doesn't have to drive in the rain since Californians can't drive in the rain?

Lots of people have different ODD's because we have different safety thresholds. Like I don't like being in situations which raise the risk of an accident. Some people don't like driving at night.

Ultimately I think what's going to happen in the industry is L4 will get better, and better through iterative designs. Then as autonomous driving grows human driving will continue to get worse. At some point autonomous drivers will simply be better. At that point I think its safe to conclude that it's L5.

Where it has ODD's similar to good taxi drivers.

 

[powertoold]

Here is the exact line:
“Unconditional/not ODD-specific” means that the ADS can operate the vehicle under all driver-manageable road conditions within its region of the world.

That part is just a general statement to say:
1) The road is public
2) The road *can* be driven by some driver (for example, if it's under construction and/or blocked, then it's not driver-manageable)

Everyone has a different ODD so what defines driver-manageable? Some median of the people? If your car is from Cali does that mean it doesn't have to drive in the rain since Californians can't drive in the rain?

The SAE document doesn't define level 5 with any performance or safety requirements (see #3 on my original post), so we should interpret driver-manageable as "a human driver can drive on this road" (not related to his performance, just the fact that it can be driven on by a human driver).

I find it pretty easy to rule out L5 with the existing HW3 simply due to obstruction of the sensors due to weather conditions. Whether its rain, light snow, light fog, or even sun at certain angles.

That's the thing. Level 5 doesn't have to mean that it's on every Tesla. The dispatcher can choose when to enable it if it's driverless. That means if the dispatcher sees that car has a blocked sensor, he can decide not to dispatch the car. Also, the level 5 definition seems to allow Tesla to disable the level 5 feature if a sensor is currently blocked, or if a sensor is blocked, the car can simply pull over. A blocked/broken sensor isn't considered an ODD limitation. It could be defined as a system failure.

Source: 3.18 [DDT PERFORMANCE-RELEVANT] SYSTEM FAILURE
A malfunction in a driving automation system and/or other vehicle system that prevents the driving automation system from reliably performing the portion of the DDT on a sustained basis, including the complete DDT, that it would otherwise perform.

 

[Bladerskb]

My views on level 5 and Tesla:

1) FSD beta is a level 5 capable feature in testing.

If that is the case then Mobileye's test fleet and SuperVision is also L5 in testing based on any actual reasonable logic.

2) Tesla will enable a level 5 feature in at least one of their cars by end of 2023, pending regulatory approval.

With these in mind, I'd say Tesla will achieve level 5 with average human level reliability by the end of this year, for sure.

First it was 6 months, then it was end of 2021 now its end of 2023. Which is it?
Secondly Level 5 has no features in the sense that Level 1-4 does. But rather the feature is the use-case.
While there will be Level 4 parking valet system. There is no Level 5 parking valet system.
The is however a level 5 car that was purchased and being used for parking valet.

3) I say pending regulatory approval because level 5 has no geographical limitations, so the level 5 capable car has to be approved USA-wide.
Source: 5.6 LEVEL or CATEGORY 5 - FULL DRIVING AUTOMATION: “Unconditional/not ODD-specific” means that the ADS can operate the vehicle under all driver-manageable road conditions within its region of the world. This means, for example, that there are no design-based weather, time-of-day, or geographical restrictions on where and when the ADS can operate the vehicle.

Its quite easy to know weather a car is capable or not. As Geohotz said, you can't hide behind regulation.

Important aspects of level 5 pertaining to Tesla's ability to achieve it:

These are aspects that you have concluded by walking from your conclusion which is Tesla and the already FSD Beta.
These aspects have nothing to do with SAE level 5.

A level 5 feature only pertains to public roadways and doesn't involve drive into or backing out of your driveway.
Source: 1. SCOPE ... “On-road” refers to publicly accessible roadways (including parking areas and private campuses that permit public access)

The statement "parking areas" includes driveway.
Parking lots, spaces, garages, driveways are not roadways even if they are publicly accessible or not.
The statement made in the parenthesess provides additional info.
This info is "parking areas". Parking area is an umbrella term that includes all form of parking, including driveways.
Lastly the wording "that permit public access" is only in association with private campuses.

2) When Tesla has a driverless robotaxi service, a dispatcher (i.e., Tesla employee) can decide when to allow service operation. This means that if Tesla forecasts a snow storm, forest fire, hurricaine coming, they can prevent the dispatch of or recall their robotaxis. This still fulfills level 5.
Source: Table 2 - Roles of human driver and driving automation system by level of driving automation
Level 5 - Full Driving Automation
Driver/dispatcher (while the ADS is not engaged):
• Determines whether to engage the ADS

This is incorrect.
The decision to engage the ADS cannot be because of weather conditions. You can't say because its raining we won't send our cars but we promise our cars can handle it.
That is a design and a restriction. Unless they are recalling the car to prevent it from being damaged. A Level 5 MUST work in those conditions.

For example a level 5 must be capable of getting your family away from a wildfire aslong as a human is capable of doing it.
If it can't then its not level 5.

3) Level 5 doesn't define safety or performance parameters. Whether or not a car is level 5 is based on its unlimited ODD and lack of expectation to intervene. However, a feature may still be level 5 if the user intervenes by choice, if steering wheel and controls are still available.

Not just unlimited ODD and lack of expectation to intervene.
But also lack of requirement to pay attention to the road or (supervise) the system's performance.

5) Level 5 has nothing to do with human capability or not. It's simply a car with a feature like FSD beta that:

It does. The ODD is based on what the average human capability is.

An Level 5 has to be able to drive in say India.
Paved roads and unpaved roads.

or Africa

or Paris

That part is just a general statement to say:
1) The road is public
2) The road *can* be driven by some driver (for example, if it's under construction and/or blocked, then it's not driver-manageable)

Incorrect. Under construction roads is not the definition of driver managable. Level 5 needs to handle all under constructions roads and blocked roads and take all neccesary detours just as a human would.

That's the thing. Level 5 doesn't have to mean that it's on every Tesla. The dispatcher can choose when to enable it if it's driverless. That means if the dispatcher sees that car has a blocked sensor, he can decide not to dispatch the car. Also, the level 5 definition seems to allow Tesla to disable the level 5 feature if a sensor is currently blocked, or if a sensor is blocked, the car can simply pull over. A blocked/broken sensor isn't considered an ODD limitation. It could be defined as a system failure.

Source: 3.18 [DDT PERFORMANCE-RELEVANT] SYSTEM FAILURE
A malfunction in a driving automation system and/or other vehicle system that prevents the driving automation system from reliably performing the portion of the DDT on a sustained basis, including the complete DDT, that it would otherwise perform.

If the system is unable to drive whenever it snows/snowing then clearly it CAN'T handle that ODD.
IF the system can't reach DDT fallback when there are system/failures, it can't be Level 5. This includes if its driving and its sensors are blocked.
If the system can't complete a trip regardless of the starting and end points or intervening
road, traffic, and weather conditions. Then it can't be Level 5

 

[powertoold]

The statement "parking areas" includes driveway.

No, that statement is clearly saying that parking areas is a subset of publicly accessible roadways.
Your driveway is a private not-even-a-road and definitely not for public access.

For example a level 5 must be capable of getting your family away from a wildfire aslong as a human is capable of doing it.
If it can't then its not level 5.

No, it doesn't. Please show your source from the SAE definitions. A level 5 robotaxi dispatcher can choose not to enable the service in these cases. This is still level 5, see my part about dispatchers and performance/safety requirement in original post.

If the system is unable to drive whenever it snows/snowing then clearly it CAN'T handle that ODD.
IF the system can't reach DDT fallback when there are system/failures, it can't be Level 5. This includes if its driving and its sensors are blocked.
If the system can't complete a trip regardless of the starting and end points or intervening
road, traffic, and weather conditions. Then it can't be Level 5

If the sensors are blocked (by thick snow or mud, etc.), the car can pull over. This is still level 5.

An Level 5 has to be able to drive in say India.
Paved roads and unpaved roads.

Source: For example, an ADS-equipped vehicle that is capable of operating a vehicle on all roads throughout the US, but, for legal or business reasons, cannot operate the vehicle across the border in Canada or Mexico can still be considered level 5, even if geo-fenced to operate only within the US. The rationale for this exception is that the geo-fenced limitation (i.e., US, only) is not due to limitations on the technological capability of the ADS, but rather is due to legal or business constraints, such as legal restrictions in Canada and Mexico/Central America that prohibit level 5 deployment, or the inability to make a business case for expansion to those markets.

 

[Needsdecaf]

If that is the case then Mobileye's test fleet and SuperVision is also L5 in testing based on any actual reasonable logic.





First it was 6 months, then it was end of 2021 now its end of 2023. Which is it?
Secondly Level 5 has no features in the sense that Level 1-4 does. But rather the feature is the use-case.
While there will be Level 4 parking valet system. There is no Level 5 parking valet system.
The is however a level 5 car that was purchased and being used for parking valet.



Its quite easy to know weather a car is capable or not. As Geohotz said, you can't hide behind regulation.



These are aspects that you have concluded by walking from your conclusion which is Tesla and the already FSD Beta.
These aspects have nothing to do with SAE level 5.



The statement "parking areas" includes driveway.
Parking lots, spaces, garages, driveways are not roadways even if they are publicly accessible or not.
The statement made in the parenthesess provides additional info.
This info is "parking areas". Parking area is an umbrella term that includes all form of parking, including driveways.
Lastly the wording "that permit public access" is only in association with private campuses.



This is incorrect.
The decision to engage the ADS cannot be because of weather conditions. You can't say because its raining we won't send our cars but we promise our cars can handle it.
That is a design and a restriction. Unless they are recalling the car to prevent it from being damaged. A Level 5 MUST work in those conditions.

For example a level 5 must be capable of getting your family away from a wildfire aslong as a human is capable of doing it.
If it can't then its not level 5.

Not just unlimited ODD and lack of expectation to intervene.
But also lack of requirement to pay attention to the road or (supervise) the system's performance.



It does. The ODD is based on what the average human capability is.

An Level 5 has to be able to drive in say India.
Paved roads and unpaved roads.

or Africa

or Paris

Incorrect. Under construction roads is not the definition of driver managable. Level 5 needs to handle all under constructions roads and blocked roads and take all neccesary detours just as a human would.



If the system is unable to drive whenever it snows/snowing then clearly it CAN'T handle that ODD.
IF the system can't reach DDT fallback when there are system/failures, it can't be Level 5. This includes if its driving and its sensors are blocked.
If the system can't complete a trip regardless of the starting and end points or intervening
road, traffic, and weather conditions. Then it can't be Level 5

Have you read the entire J3016 paper? I'm curious because you seem to have a pretty high level grasp of the terminology, but I disagree with some of your interpretations of the different levels.

For instance, your wildfire example. I disagree. Here's what the paper says:

" Level or Category 5 - Full Driving Automation

The sustained and unconditional (i.e. not ODD-specific) performance by an ADS of the entire DDT and DDT fallback without any expectation that a user will respond to a request to intervene

Note 1: "Unconditional/not ODD-specific" means that the ADS can operate the vehicle under all driver-managable road conditions within it's region of the world. This means, for example, that there are no design-based weather, time-of-day, or geographical restrictions on where and when the ADS can operate the vehicle. However, there may be conditions not manageable by a driver in which the ADS would also be unable to complete a given trip (e.g., white-out snow storm, flooded roads, glare ice, etc.) until or unless the adverse conditions clear. At the onset of such unmanageable conditions the ADS would perform the DDT fallback to achieve a minimal risk condition (e.g. by pulling over to the side of the road and waiting for the conditions to change)."

I think a driver getting away from a wildfire would be pure luck. I would not consider driving through a wildfire part of the DDT.

 

[Needsdecaf]

Source: For example, an ADS-equipped vehicle that is capable of operating a vehicle on all roads throughout the US, but, for legal or business reasons, cannot operate the vehicle across the border in Canada or Mexico can still be considered level 5, even if geo-fenced to operate only within the US. The rationale for this exception is that the geo-fenced limitation (i.e., US, only) is not due to limitations on the technological capability of the ADS, but rather is due to legal or business constraints, such as legal restrictions in Canada and Mexico/Central America that prohibit level 5 deployment, or the inability to make a business case for expansion to those markets.

You made his point for him.

Look at the line that I bolded. It's for legal or business reasons. Not the technological capability.

 

[powertoold]

Correct me if I'm wrong, but I think most people misunderstand level 5 because it's a feature that can be enabled or disabled.

Once level 5 is enabled, it simply needs to have no ODD restrictions and no expectation that the user intervene.

Level 5 doesn't have to be enabled all the time, but again, IF it IS enabled, then it needs to fulfill the requirements in the SAE definitions.

 

[powertoold]

You made his point for him.

Look at the line that I bolded. It's for legal or business reasons. Not the technological capability.

I know, but Tesla can simply say it's not viable from a business standpoint to deploy their level 5 feature in India. We can't argue that since the SAE doesn't define business viability or performance criteria.

 

[Needsdecaf]

I personally do not believe that Tesla will release FSD as a Level 4 or Level 5 ADS for liability reasons. They are VERY protective of the fact that they legally hide behind the phrasing "driver is required to monitor the system and take over performance at any time". They aren't lightly going to give that up.

I don't believe this is specific to Tesla. I believe that Volvo has said when they have a L4 or L5 ADS, they will accept legal liability during crashes.

I believe that it will take some time and some court precedents will need to be set before manufacturers willingly give up this legal security blanket.

 

[Bladerskb]

No, that statement is clearly saying that parking areas is a subset of publicly accessible roadways.
Your driveway is a private not-even-a-road and definitely not for public access.

Parking lots is private property and not-even-a-road.
Public access is talking about private campus.

If the sensors are blocked (by thick snow or mud, etc.), the car can pull over. This is still level 5.

If Tesla AP2/3 hardware can't achieve this then it can't be Level 5.

Source: For example, an ADS-equipped vehicle that is capable of operating a vehicle on all roads throughout the US, but, for legal or business reasons, cannot operate the vehicle across the border in Canada or Mexico can still be considered level 5, even if geo-fenced to operate only within the US. The rationale for this exception is that the geo-fenced limitation (i.e., US, only) is not due to limitations on the technological capability of the ADS, but rather is due to legal or business constraints, such as legal restrictions in Canada and Mexico/Central America that prohibit level 5 deployment, or the inability to make a business case for expansion to those markets.

I'm talking about technical capability and based on how FSD Beta works using OPENSTREETSMAP (which only supports as their AV Map. Its technologically incapable.

No, it doesn't. Please show your source from the SAE definitions. A level 5 robotaxi dispatcher can choose not to enable the service in these cases. This is still level 5, see my part about dispatchers and performance/safety requirement in original post.

That's not what the SAE document says, you are twisting it to mean whatever you want it to mean.
There are no "dispatchers and performance/safety requirement" in the SAE. Another thing you are making up.
The "Determines whether to engage the ADS" has nothing to do with weather conditions or any specific cases at all.
All the table is explaining and the purpose of it is the functional and liability requirement of the user and the system at each level.

The "Determines whether to engage the ADS" is stating that the user can push a button to start the system/drive. Not to do some performance or safety requirement evaluation before pushing the button. That happens only in level 3. In Level 3, the user actually "Determines when engagement of ADS is appropriate". In Level 5, the system itself determines if a human can drive in this condition or not.

That's why the chart is: Role of User and Role of DAS.

 

[powertoold]

Parking lots is private property and not-even-a-road.
Public access is talking about private campus.

This is wrong. If we get stuck on this, then there's no point debating any further, since our reading comprehension is disjointed.

Source: “On-road” refers to publicly accessible roadways (including parking areas and private campuses that permit public access)

This clearly means that many private campuses (like Target / Walmart / businesses / Universities / etc.) have roads / parking lots / garages that are PRIVATELY owned but are publicly accessible for purposes of business / etc. Anyone is allowed to drive on these roads, despite them being privately owned.

Publicly accessible roadways:
1) Parking areas (parking lots)
2) Private campuses with publicly accessible roads / parking lots

 

[Stavanger]

Im sure they will reach it eventually, like in 20-40 years from now. It will be a totally different set of hardware though.
Current hardware wont really get us much further, all it takes is some weather and the car is crippled.
I can literally clean my car to perfection then drive 5 minutes and bam some camera is now covered in salt/snow/much-whatever.
Low sun and snow and the autopilot barely works at all due to blinded cameras so yeah no...

 

[S4WRXTTCS]

It could be defined as a system failure.

Failing to drive in human-manageable road conditions can't be defined as a system failure. It's a design failure, and is thereby not L5.

 

[powertoold]

Failing to drive in human-manageable road conditions can't be defined as a system failure. It's a design failure, and is thereby not L5.

I don't think this is the intent of level 5. If the level 5 feature detects a blocked or possibly malfunctioned sensor, it can pull over or not allow activation of the level 5 feature.

Again, the SAE definition doesn't outline any performance or safety criteria for level 5. I think this is the most common misunderstanding. Since you are using "human" as a performance criteria, this is outside of the SAE definition. The SAE definition uses "driver-manageable" (your wording "human-manageable" is not used) to simply refer to roads that are publicly accessible by a human driver. See the wording below (emphasis always mine). Level 5 is about operation, not performance:

Source: “Unconditional/not ODD-specific” means that the ADS can operate the vehicle under all driver-manageable road conditions within its region of the world. This means, for example, that there are no design-based weather, time-of-day, or geographical restrictions on where and when the ADS can operate the vehicle.

I just want to make it clear that I don't like the SAE definitions. I'm just using this thread to sort out the level 5 misconceptions.

 

[diplomat33]

Failing to drive in human-manageable road conditions can't be defined as a system failure. It's a design failure, and is thereby not L5.

I think the SAE document would disagree with you.

Section 8.2, page 30 says:

"The manifestation of one or more performance deficiencies in either the driving automation system or in the user’s use of it does not automatically change the level assignment. For example: An ADS feature designed by its manufacturer to be level 5 would not automatically be demoted to level 4 simply by virtue of encountering a particular road on which it is unable to operate the vehicle. "

So the SAE says that if a L5 system encounters a road or situation it can't handle, it is still L5.

 

[S4WRXTTCS]

I think the SAE document would disagree with you.

Section 8.2, page 30 says:

"The manifestation of one or more performance deficiencies in either the driving automation system or in the user’s use of it does not automatically change the level assignment. For example: An ADS feature designed by its manufacturer to be level 5 would not automatically be demoted to level 4 simply by virtue of encountering a particular road on which it is unable to operate the vehicle. "

So the SAE says that if a L5 system encounters a road or situation it can't handle, it is still L5.

L4 and L5 are differentiated primarily by ODD.

That L5 is not supposed to have ODD limitations beyond that of human driver.

Human beings don't cease to be license drivers just because some unique road or situation is beyond their capacity. I would have lost my license a long time ago if that was the case.

But, if you can't drive in Seattle drizzle then you should not be a licensed driver.

Just like Tesla shouldn't be considered L5 if it can't handle Seattle weather.

 

[diplomat33]

L4 and L5 are differentiated primarily by ODD.

That L5 is not supposed to have ODD limitations beyond that of human driver.

Human beings don't cease to be license drivers just because some unique road or situation is beyond their capacity. I would have lost my license a long time ago if that was the case.

But, if you can't drive in Seattle drizzle then you should not be a licensed driver.

Just like Tesla shouldn't be considered L5 if it can't handle Seattle weather.

L5 is not expected to drive in conditions that are considered unsafe for human drivers like a flood or severe storm. I don't think anyone is saying that L5 does not need to handle a drizzle.