TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC
  1. TMC is currently READ ONLY.
    Click here for more info.

What is SAE Level 5 and can Tesla actually achieve it?

Discussion in 'Autopilot & Autonomous/FSD' started by powertoold, Jan 19, 2021.

  1. mspisars

    mspisars Active Member

    Joined:
    May 23, 2014
    Messages:
    2,030
    Location:
    Charlotte, NC
    And those L4 were with real drivers for a very long time - yet it was still L4 system being tested and Google/Waymo was expecting the safety driver to intervene at the right moments.

    a la Tesla FSD, the stated intent of the system is L5, and Tesla demands that anyone in the beta agrees to supervise the car at all times, until the software matches the desired performance of the intended L5 target.

    This is all pointless, as discussing intent does not get us closer to L5.
    Executing and seeing forward/positive progress does.
     
  2. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    #42 S4WRXTTCS, Jan 22, 2021
    Last edited: Jan 22, 2021
    If your design intent is to operate on roads, and conditions that can be driven by human drivers then you've set goal post for it.

    With L4 the ODD is allowed to be limited, and there is no need to set a goal post.

    With L5 the ODD needs to have some kind of goal post. It's meaningless without a goal post. The unrestricted ODD is what separates it from L4. But, they can't just say unrestricted as that would be really unrealistic. So they use driver-manageable as the goal post.

    Tesla won't be able to drive on driver-manageable roads with the HW3 Sensor suite. Therefor it does not meet the design intent criteria. It will fall far short of being able to drive on driver-manageable roads, and conditions.

    I agree that the SAE Levels are not intentionally setting performance criteria nor do they set safety criteria. That they avoid that as much as possible, but for L5 they had to have some reference point for design intent.

    If Tesla had a design that could even get close to driving on driver-manageable roads, weather, and situations then this discussion wouldn't be happening.

    What people don't seem to grasp is HW3/FSD is meant to fail. The entire purpose of it is to fail over, and over again as it gets better each time. That Elon knew before they ever released it that they were not going to achieve L5 with the sensor suite.

    Elon accepts embraces failure, and achieves success through it.
     
  3. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    The SAE definition doesn't have any performance reference point. They make that very clear. All other interpretation here is speculation, so we should agree on the clearest SAE statement on performance or safety. In this context, it actually makes the sae levels more meaningful, since the levels simply refer to a level of automation, not performance. See my dumb "box" analogy.
     
  4. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    In your original post you said "Here, we discuss whether or not Tesla can achieve level 5 in the near future, or is it impossible (given the sensor suite and/or feature limitations)?"

    I've explained why they can't achieve it.

    You, and I both know the SAE Levels are about design intent. We also agree that the design intent of FSD is Level 5 as Elon has been very clear about that.

    So we know the goal is L5.

    But, the disagreement comes down to what achieving L5 really is.

    To me the SAE Level 5 clearly uses driver-manageable roads, and weather conditions. The Tesla HW3 Sensor suite isn't intended to handle driver-manageable weather conditions, and anyone with FSD that lives in areas with challenging weather knows this.

    So I say HW3 isn't even an intent at L5 driving, and it's just marketing by Tesla/Elon in trying to claim it somehow is. So it's already failed as its not a serious attempt at L5.

    And, actually achieving Level 5 is more than just design intent as the design has to actually work to get approval. I would be absolutely astonished if Tesla achieved any kind of approval from regulatory/insurance/customers to do L4 driving let alone L5 with HW3.

    To get approval it has to meet certain Safety thresholds like Elon himself has used the 10x better than a human for the safety goal.

    So even if we ignore regulatory it's unlikely to ever get Elons approval.
     
  5. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    Huh? Are you saying Tesla doesn't intend the sensor suite achieve level 5? That makes no sense, as Elon said so himself.

    Again, if Tesla intends it to be level 5 and it doesn't have any ODD and has no intervention expectation, it's level 5. That's the sae definition.

    Safety is totally separate from the sae definition.

    Anyway, it seems to me that you're doing the mental gymnastics on this topic, lol.
     
    • Like x 1
  6. mspisars

    mspisars Active Member

    Joined:
    May 23, 2014
    Messages:
    2,030
    Location:
    Charlotte, NC
    #46 mspisars, Jan 22, 2021
    Last edited: Jan 22, 2021
    Also, have you watched any of the FSD Beta videos, especially guy from Michigan and Kim from Rhode Island. They seem to deal with snow and weather just fine.
    Unlike Waymo, Tesla is now in ~23 states plus DC. There is NO restriction to when to use FSD beta or where to use it.
     
  7. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    #47 S4WRXTTCS, Jan 22, 2021
    Last edited: Jan 22, 2021
    But, its critical towards achieving any kind of autonomous driving whether its L3, L4, or L5.

    And, in your original post you said achieve.

    You know HW3 doesn't have sensor redundancy

    You know HW3 has issues with sensors being obstructed in adverse, but driver manageable conditions.

    You know Elon is very much a believer of fail until you succeed. Sometimes you still try when you absolutely know you don't have what it will take, but there is enough things you can learn along the way that it doesn't matter.

    The mental gymnastics is about what driver-manageable means. I don't know what kind of mental gymnastics you're doing to think the SAE is not using human drivers as a benchmark for the ODD for L5.
     
  8. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    #48 S4WRXTTCS, Jan 22, 2021
    Last edited: Jan 22, 2021
    What's interesting about some of the videos is the car will even say that a sensor is obstructed or "bad weather detected" and yet it will still allow them to continue on.

    So I'm curious what the actual delivered product will have as a threshold for weather. I don't think its going to be so lenient to us non-EAP people if/when released.

    On twitter I'm trying to find the WA State FSD beta people to follow, but I haven't found them. There is at least 1 WA state person, and at least 1 Oregon one according to the map of known states with FSD beta (where you got the 25 number from).
     
  9. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    It literally says no benchmarks in the definition. The SAE definition removes any subjectivity from the levels. That's their goal.

    Here you are, saying the sae definition uses humans as a benchmark for level 5 performance when the definition specifically says it doesn't.

    Driver-manageable simply means the road *can* be driven on by the public. There's no implied performance criteria in that.
     
    • Like x 2
  10. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    Driver-manageable sounds a lot like a benchmark to me.

    Don't you have better things to do like following a car so you can use AP?
     
  11. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    If there were any benchmarks or performance comparisons, the sae definition would make that clear in a 35-page description. Instead, you are here making your own interpretation of it, which is the opposite of what that document is trying to do. It seems they were aware people would make performance comparisons, so they specifically added a line to clear that up.

    After all, your idea of driver manageable is different than Lewis Hamilton's, is different than mine, different with a drunk person, etc. That's exactly what the SAE wanted to avoid.
     
  12. mspisars

    mspisars Active Member

    Joined:
    May 23, 2014
    Messages:
    2,030
    Location:
    Charlotte, NC
    #52 mspisars, Jan 22, 2021
    Last edited: Jan 22, 2021
    This guy makes pretty maps... https://twitter.com/jeadly/status/1349512344888602627
    he has 23 states plus DC (not the 25 I mentioned)
    [​IMG]
    Hawaii is not on that map but has FSD Beta!
     
    • Helpful x 1
  13. Redbrick

    Redbrick Member

    Joined:
    Jun 27, 2020
    Messages:
    195
    Location:
    Babylon 5
    So say if tomorrow tesla removes the need for driver to keep your hands on the wheel, and autopilot nags...then those with fsd immediately get L4?
     
  14. mspisars

    mspisars Active Member

    Joined:
    May 23, 2014
    Messages:
    2,030
    Location:
    Charlotte, NC
    #54 mspisars, Jan 22, 2021
    Last edited: Jan 22, 2021
    I knew I had a real reason for 25 states.
    That same thread is a response to a guy getting it in New York state https://twitter.com/stephenpallotta/status/1349483235231805440
    And further down, in the same thread a guy says he got it in Pennsylvania!
     
    • Informative x 1
  15. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    No, because level 4 requires some fallback feature, like pulling over.
     
  16. Bladerskb

    Bladerskb Senior Software Engineer

    Joined:
    Oct 24, 2016
    Messages:
    2,068
    Location:
    Michigan
    There's a prompt that shows up that says something similar to "FSD Beta can't complete this maneuver please take over." Ofcourse im paraphrasing.
    Parking areas are not parking lots and is not the intent of the SAE language. Parking lot is defined as a big place with durable or semi durable surface.

    This is why they said "areas" not "lots". They knew what

    There are different types of parking areas, parking spaces on side streets, regular unmarked side street parking, grassfields, driveways, gated community, private condos, etc. There are also different types of driveways. Them using the word "parking lot" would eliminate all of that.

    [​IMG]

    To think that L5 doesn't cover the biggest usecase of driving which is helping the elderly and disabled. Usecases that L4 parking systems would cover.Usecases that L4 system in certain communities would cover.

    Waht you are saying that L4 would actually be more functional than a L5 system. When in a nutshell, the definition of L5 is L4 without limits or L4 without restrictions. You can't have a L4 system doing things and covering use-cases that L5 doesn't cover.

    For example: Showing up to a concert or an event and the parking area is not some lot (durable or semi-durable surface)...
    [​IMG]

    [​IMG]

    FSD Beta gives an error with message similar to can't complete maneuver please take over (obviously i'm paraphrasing.
     
  17. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    I haven't seen that lately, but ya, I vaguely remember it before.

    As for the driveway thing, I still disagree with you. Private driveways are not publicly accessible. I gave examples of private areas that are intended to be publicly accessible.

    As for your elderly example, that's subjective.
     
  18. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    What sets the expectations for the ODD for L5?

    We know it can't be limited as that's what L4 is.

    We know it can't be unlimited as that's not realistic.

    What could possibly be used to set the expectations for an L5 autonomous driving robocar? If only we knew who or what was driving cars around today we could use that as a reference point. So designers of such a system knew how to set the ODD design intent.
     
  19. powertoold

    powertoold Active Member

    Joined:
    Oct 10, 2014
    Messages:
    1,901
    Location:
    USA
    If we are to use the sae definitions, the expectation is no ODD, no requirement that the user intervene or take over. Black and white definition.

    Yes, there are other subjective expectations for a level 5 car, but that's separate from the sae definition.
     
  20. S4WRXTTCS

    S4WRXTTCS Well-Known Member

    Joined:
    May 3, 2015
    Messages:
    5,305
    Location:
    Snohomish, WA
    Yet, the SAE Level 5 allows for exceptions as stated:

    "However, there may be conditions not manageable by a driver in which the ADS would also be unable to complete a given trip"

    All these pesky "drivers" being used to determine the expected ODD limits, and ruining our black and white definition.

    I would have preferred:

    "Neither snow nor rain nor heat nor gloom of night stays these autonomous beast from the swift completion of their appointed trips."

    Maybe go with a Cybertruck in a snow storm picture next to it.
     
    • Funny x 1

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC