TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC
  1. TMC is currently READ ONLY.
    Click here for more info.

What is SAE Level 5 and can Tesla actually achieve it?

Discussion in 'Autopilot & Autonomous/FSD' started by powertoold, Jan 19, 2021.

  1. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    I think you're doing mental gymnastics to force a unique interpretation onto the Level 5 definition.

    Obviously I meant driver manageable when I said human manageable. That the driver is a human being, and this is the measuring stick we should use.

    All driver-manageable road conditions IS a performance criteria. That human drivers are the bench mark.

    The primary difference between L4, and L5 is ODD.

    That L4 is allowed to have restrictions based on weather, road type, etc.

    L5 is supposed to allow engagement in any ODD that is driver-manageable.

    In your interpretation of the SAE Levels what is the purpose of L5?
     
  2. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    Under the SAE Level 5 a serious storm would be considered outside driver-manageable, and not expected from an L5 system.

    The OP is saying that SAE Level 5 can be significantly more weather restricted than a human driver, and I disagree with that.

    The simple fact is that a Tesla HW3 cannot deal with weather that humans can easily deal with, and thereby will never be an L5 vehicle by the SAE definitions.
     
    • Helpful x 1
    • Disagree x 1
  3. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    #23 powertoold, Jan 21, 2021
    Last edited: Jan 21, 2021
    Please show me where I said that. I was referring specifically to snow or mud on one or more of the sensors.

    Please do try to show the sae wording to support your views. Level 5 means that no ODD is designed into the level 5 feature, once it's activated. However, Tesla software can prevent the level 5 feature from being activated if it senses that a sensor is obscured.

    I'll give you an example. Say you parked your level 5 capable car in the snow. The snow covers a sensor. When you get in the car, the level 5 feature can't be enabled. This is still level 5.

    Also, even while it's driving in level 5 mode, if a sensor becomes covered in mud. The car can pull over. This is still level 5.
     
  4. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    This is totally wrong. Show us where the SAE definition uses human capability or performance to assess the performance of level 5. This is specifically why the SAE uses driver manageable instead of human manageable. There's a semantic difference here.
     
    • Like x 1
  5. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    #25 S4WRXTTCS, Jan 21, 2021
    Last edited: Jan 21, 2021
    I don't disagree that Tesla can prevent the Level 5 feature from being activated if a sensor is obscured by snow or isn't working correctly. There isn't any disagreement there. Obviously a human can't drive if the windshield is covered in snow.

    But, if I get into my car while its raining in Seattle and the car says "Nope, no can do" then by SAE definitions its not L5 as obviously rain (to a point) is a driver manageable situation. Or if it pulls over during a drive due to obscured sensors because of a little bit of rain on a consistent basis then I would say it's not L5 as I would see those failures as design limitations.
     
    • Like x 1
  6. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    #26 S4WRXTTCS, Jan 21, 2021
    Last edited: Jan 21, 2021
    This is the specific block of text were interpreting.

    "“Unconditional/not ODD-specific” means that the ADS can operate the vehicle under all driver-manageable road conditions within its region of the world. This means, for example, that there are no design-based weather, time-of-day, or geographical restrictions on where and when the ADS can operate the vehicle. However, there may be conditions not manageable by a driver in which the ADS would also be unable to complete a given trip (e.g., white-out snow storm, flooded roads, glare ice, etc.) until or unless the adverse conditions clear. At the onset of such unmanageable conditions the ADS would perform the DDT fallback to achieve a minimal risk condition (e.g., by pulling over to the side of the road and waiting for the conditions to change"

    From this text my definition for driver manageable is that it can be reasonably assumed that a licensed human driver can handle the road or situation.

    The SAE Primer on posted on TMC has this example:

    "Example of L5 ODD: a robotaxi that can operate on all driver manageable roads in the US, day or night, all weather that human can handle, all legal speeds."

    So right there it has "weather that human can handle" as a benchmark.

    What's your definition for driver-manageable?
     
  7. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    I might disagree with you, but I'm looking forwards to late 2023 when thousands of make believe L5 Tesla's are sitting on the side of I5 waiting for small rain shower to clear up before continuing on.

    If I had to pull over every time I got the "sensor obscured" message I would never make it anywhere.
     
  8. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,832
    Location:
    Terre Haute, IN USA
    By the way, I created the primer awhile ago. "weather that human can handle" is my phrase, not a direct quote from the SAE document. It is based on this sentence in the SAE document:

    "there may be conditions not manageable by a driver in which the ADS would also be unable to complete a given trip (e.g., white-out snow storm, flooded roads, glare ice, etc.) until or unless the adverse conditions clear."

    So "driver-manageable" would basically be the inverse of this, ie conditions that a driver could be expected to manage.

    Also, here is the example of L5 that the SAE document provides:

    "A vehicle with an ADS that, once programmed with a destination, is capable of operating the vehicle throughout complete trips on public roadways, regardless of the starting and end points or intervening road, traffic, and weather conditions."

    I think if we combine the two sentences from the SAE document (just adding the word "however" to connect the two), we get a more complete description for L5:

    "A vehicle with an ADS that, once programmed with a destination, is capable of operating the vehicle throughout complete trips on public roadways, regardless of the starting and end points or intervening road, traffic, and weather conditions. However, there may be conditions not manageable by a driver in which the ADS would also be unable to complete a given trip (e.g., white-out snow storm, flooded roads, glare ice, etc.) until or unless the adverse conditions clear."
     
    • Like x 2
  9. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,832
    Location:
    Terre Haute, IN USA
    Do you mean FSD Beta has the potential to be L5? Maybe. But FSD Beta is not L5 in testing.

    From Section 5.6 page 25 of SAE document:

    "NOTE 2: In the event of a DDT performance-relevant system failure(of an ADS or the vehicle), a level 5 ADS automatically performs the DDT fallback and achieves a minimal risk condition.

    NOTE 3: The user does not need to supervise a level 5 ADS, nor be receptive to a request to intervene while it is engaged."

    FSD Beta does not meet either of these big L5 requirements yet.
     
    • Like x 1
    • Disagree x 1
  10. mspisars

    mspisars Active Member

    Joined:
    May 23, 2014
    Messages:
    2,030
    Location:
    Charlotte, NC
    SAE FSD levels are design intent - you are the one that pointed that out!
    We know Tesla's design intent for FSD is L5.
    Therefore, Tesla is testing L5 designed FSD features.
     
    • Disagree x 2
  11. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,832
    Location:
    Terre Haute, IN USA
    Tesla may intend FSD Beta to be L5 at some point. But FSD Beta is NOT L5 now.

    Design intent still needs to meet the SAE definitions.
     
    • Like x 1
    • Disagree x 1
  12. mspisars

    mspisars Active Member

    Joined:
    May 23, 2014
    Messages:
    2,030
    Location:
    Charlotte, NC
    LMAO - this is why discussing SAE or really any standard is useless.
    moving on.
     
  13. mspisars

    mspisars Active Member

    Joined:
    May 23, 2014
    Messages:
    2,030
    Location:
    Charlotte, NC
    Contrast your response above with this response you wrote just earlier in this thread.
    Design intent is the same for Tesla FSD!
     
    • Disagree x 1
  14. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    You're right. But I think you know what I mean, lol.

    As for the design intent, I think most people here misunderstand that part as well.
     
    • Like x 1
  15. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    We would interpret it that way if the SAE didn't say the definitions don't include any performance or safety criteria. They even say that level 5 is about design intent, not actual performance. So it's very clear we can't compare level 5 to a human.

    A dumb analogy is the definition of a "box." As long as it has 6 rectangular sides, then it's a box, no matter the material, design on it, or craftsmanship. This is what the SAE definitions are trying to do: provide a way to describe automation features without inferring their performance.
     
    • Like x 1
  16. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,832
    Location:
    Terre Haute, IN USA
    That quote is talking about how an ADS does not change levels after the fact just because it can't handle something. So if Tesla does deliver L5, it would not suddenly get downgraded to L4 just because it can't handle something.

    But AFAIK, "design intent" is about current design, not some future goal. Just because the manufacturer wants to achieve Level X in the future, does not make it level X.

    Tesla certainly intends FSD Beta to be L5 some day, sure. But the fact that Tesla tells FSD Beta testers that they must keep their hands on the wheel, supervise at all times and be prepared to take over at any time, indicates that the current design intent of FSD Beta is L2.

    Similarly, Waymo say that they are working to make Driver L5 some day. But the current design intent of the Waymo Driver is L4 because the Waymo Driver is currently designed to operate in a geofenced area.

    @Needsdecaf @Bladerskb I trust you guys. You are well informed on these matters. If I am wrong, please let me know.
     
  17. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    That's my understanding as well.

    If the feature is currently designed to have no ODD and doesn't expect the user to intervene, then it's level 5, regardless of its performance.

    It's a bit of a gray area with FSD beta. I don't think the fsd beta is designed (in the software) to expect intervention. However, Tesla expects testers to intervene and disengage (in agreements or coaching). But yes, I still think this is level 2.
     
    • Like x 2
  18. Needsdecaf

    Needsdecaf Member

    Joined:
    Dec 16, 2018
    Messages:
    829
    Location:
    The Woodlands, TX
    I am really not understanding this point or this whole argument, on either side.

    Please, help me understand, what "Design Intent" has to do with anything?
     
  19. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    That's a good clarification.

    My intent was to show how the block of text was being interpreted. I felt like your interpretation of it was correct at the time, and I still feel the same way.

    If you feel like your interpretation was wrong on the primer then please explain why it was wrong.
     
  20. diplomat33

    diplomat33 Well-Known Member

    Joined:
    Aug 3, 2017
    Messages:
    6,832
    Location:
    Terre Haute, IN USA
    No, I feel like my interpretation on the primer is still good. It might be a bit simplistic but I think it captures the meaning of the SAE document.
     

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC