TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

Tracking FSD Feature Complete

Discussion in 'TSLA Investor Discussions' started by Wiki, May 26, 2019.

Tags:
Thread Status:
The first post in this thread is a WikiPost, and can be edited by anyone with the appropriate permissions.
  1. diplomat33

    diplomat33 Active Member

    Joined:
    Aug 3, 2017
    Messages:
    2,195
    Location:
    Terre Haute, IN USA
    #61 diplomat33, Jun 14, 2019
    Last edited: Jun 14, 2019
    So you are saying that the software that we have now in our cars is exactly the same as the software in development, just with some features turned off? In other words, Tesla has nothing in development right now that is not also in our cars as well? Somehow I doubt that.

    And I watched the video again. AP is turned off at the intersection whereas the FSD demo did show NOA handling the intersection. So the hacker was not able to completely recreate the entire FSD demo. That's because the FSD demo had additional "intersection" software that our cars don't have yet. Unless I misread the video?
     
  2. EVNow

    EVNow Well-Known Member

    Joined:
    Sep 5, 2009
    Messages:
    5,955
    Location:
    Seattle, WA
    I think they are merging back the dev branch frequently with the production branch. Otherwise we shouldn't see all the features we saw in the demo, in the hacked software as well.

    Apparently the turning was not good (either too sharp or not sharp enough) - so he was turning it off at intersections. There are a couple of videos where he did let it turn on its own.

     
    • Helpful x 1
  3. diplomat33

    diplomat33 Active Member

    Joined:
    Aug 3, 2017
    Messages:
    2,195
    Location:
    Terre Haute, IN USA
    Thanks. Very helpful.
     
  4. EVNow

    EVNow Well-Known Member

    Joined:
    Sep 5, 2009
    Messages:
    5,955
    Location:
    Seattle, WA
    Another explanation for having all the FSD dev code fleetwide but disabled is to collect data in the "shadow mode". While shadow mode potential is huge, not sure what they actually do.
     
    • Like x 1
  5. JusRelax

    JusRelax Member

    Joined:
    Apr 24, 2019
    Messages:
    164
    Location:
    Los Angeles, CA


    upload_2019-6-24_12-29-53.png
     
    • Informative x 1
  6. diplomat33

    diplomat33 Active Member

    Joined:
    Aug 3, 2017
    Messages:
    2,195
    Location:
    Terre Haute, IN USA
    Note sure if this has been mentioned already but here is the list of self-driving behaviors that Waymo cars are already competent in (taken from Waymo Safety Report 2018, Appendix A on page 36-37). I feel like this would be a good list to use to judge Tesla's FSD progress:

    Set of Behavioral Competencies Recommended by NHTSA
    1 Detect and Respond to Speed Limit Changes and Speed Advisories
    2 Perform High-Speed Merge (e.g., Freeway)
    3 Perform Low-Speed Merge
    4 Move Out of the Travel Lane and Park (e.g., to the Shoulder for Minimal Risk)
    5 Detect and Respond to Encroaching Oncoming Vehicles
    6 Detect Passing and No Passing Zones and Perform Passing Maneuvers
    7 Perform Car Following (Including Stop and Go)
    8 Detect and Respond to Stopped Vehicles
    9 Detect and Respond to Lane Changes
    10 Detect and Respond to Static Obstacles in the Path of the Vehicle
    11 Detect Traffic Signals and Stop/Yield Signs
    12 Respond to Traffic Signals and Stop/Yield Signs
    13 Navigate Intersections and Perform Turns
    14 Navigate Roundabouts
    15 Navigate a Parking Lot and Locate Spaces
    16 Detect and Respond to Access Restrictions (One-Way, No Turn, Ramps, etc.)
    17 Detect and Respond to Work Zones and People Directing Traffic in Unplanned or Planned Events
    18 Make Appropriate Right-of-Way Decisions
    19 Follow Local and State Driving Laws
    20 Follow Police/First Responder Controlling Traffic (Overriding or Acting as Traffic Control Device)
    21 Follow Construction Zone Workers Controlling Traffic Patterns (Slow/Stop Sign Holders)
    22 Respond to Citizens Directing Traffic After a Crash
    23 Detect and Respond to Temporary Traffic Control Devices
    24 Detect and Respond to Emergency Vehicles
    25 Yield for Law Enforcement, EMT, Fire, and Other Emergency Vehicles at Intersections, Junctions, and Other Traffic Controlled Situations
    26 Yield to Pedestrians and Bicyclists at Intersections and Crosswalks
    27 Provide Safe Distance From Vehicles, Pedestrians, Bicyclists on Side of the Road
    28 Detect/Respond to Detours and/or Other Temporary Changes in Traffic Patterns

    Examples of Additional Behavioral Competencies Tested by Waymo
    29 Moving to a Minimum Risk Condition When Exiting the Travel Lane is Not Possible
    30 Perform Lane Changes
    31 Detect and Respond to Lead Vehicle
    32 Detect and Respond to a Merging Vehicle
    33 Detect and Respond to Pedestrians in Road (Not Walking Through Intersection or Crosswalk)
    34 Provide Safe Distance from Bicyclists Traveling on Road (With or Without Bike Lane)
    35 Detect and Respond to Animals
    36 Detect and Respond to Motorcyclists
    37 Detect and Respond to School Buses
    38 Navigate Around Unexpected Road Closures (e.g. Lane, Intersection, etc.) 39 Navigate Railroad Crossings
    40 Make Appropriate Reversing Maneuvers
    41 Detect and Respond to Vehicle Control Loss (e.g. reduced road friction)
    42 Detect and Respond to Conditions Involving Vehicle, System, or Component-Level Failures or Faults (e.g. power failure, sensing failure, sensing obstruction, computing failure, fault handling or response)
    43 Detect and Respond to Unanticipated Weather or Lighting Conditions Outside of Vehicle’s Capability (e.g. rainstorm)
    44 Detect and Respond to Unanticipated Lighting Conditions (e.g. power outages)
    45 Detect and Respond to Non-Collision Safety Situations (e.g. vehicle doors ajar)
    46 Detect and Respond to Faded or Missing Roadway Markings or Signage 47 Detect and Respond to Vehicles Parking in the Roadway

    This would be a pretty good list for "feature complete". At a minimum, I think Tesla needs to do 1-28 since they are recommended by the NHTSA. Hopefully, Tesla is paying attention to these recommendations by the NHTSA to help guide their development goals.
     
    • Informative x 4
  7. willow_hiller

    willow_hiller Member

    Joined:
    Apr 3, 2019
    Messages:
    717
    Location:
    Maryland
    That's a great list, @diplomat33 .

    It seems like quite a mountain to climb, but at the same time I'm realizing that in addition to the "what" of feature complete, the industry seems to be missing a good sense of "how well." For example, NOA might check the box for "High -Speed merge" but at the moment we don't seem to have a measure by which to quantify how well an autonomous system needs to perform a merge before that feature is considered complete.
     
    • Like x 1
  8. diplomat33

    diplomat33 Active Member

    Joined:
    Aug 3, 2017
    Messages:
    2,195
    Location:
    Terre Haute, IN USA
    #68 diplomat33, Jun 25, 2019
    Last edited: Jun 25, 2019
    Thanks. And yes, you make a great point about "how well". It's like that TV ad "just ok is not good enough". Certainly, any autonomous vehicle needs to be better than "just ok" in these features.

    In terms of measuring the "how well", I think that is where disengagement reports and Tesla's fleet data come in. For a feature like "high speed merges", Tesla can collect and analyse thousands of real world examples per day from the fleet to measure how good NOA is at that feature and then use that data to improve the feature. And I don't know what the exact number might be, but presumably we could quantify the "how well" by measuring disengagements during that feature. For example, "high speed merges interrupted by disengagement" divided by "total number of high speed merges".

    Yes, the list does seem quite daunting. In fact, it might explain why Waymo has not released their cars in full robotaxi mode yet. As impressive as this list might seem, Waymo probably knows that it is not a complete list and there are probably still behaviors that their cars lack. Plus, Waymo cars may still want to get even better at some of the features on their list. Two reasons why Waymo would not consider their cars ready yet for full deployment without a driver.
     
  9. willow_hiller

    willow_hiller Member

    Joined:
    Apr 3, 2019
    Messages:
    717
    Location:
    Maryland
    I agree there would be one denominator (total number of maneuvers attempted). I would be interested in two different numerators, however. First, as you suggested "maneuvers interrupted by disengagement," but maybe secondly "maneuvers resulting in a collision." I think regulators would be willing to stomach a high number of disengagements in general if the vehicle always safely pulled over to the side of the road; if the data show the technology is causing injury or damage to property, on the other hand, that's a different story.
     
    • Like x 1
  10. EVNow

    EVNow Well-Known Member

    Joined:
    Sep 5, 2009
    Messages:
    5,955
    Location:
    Seattle, WA
    This is great. I was thinking about asking whether anyone knows the list of features Waymo has implemented !

    I'll split these into specific & platform features and "edge" cases and updated the first post.

    I think a generic way to track the features would be to turn this thread into a Feature parity with Waymo/Cruise thread (may be once Tesla announces FC).
     
    • Like x 2
  11. EVNow

    EVNow Well-Known Member

    Joined:
    Sep 5, 2009
    Messages:
    5,955
    Location:
    Seattle, WA
    #71 EVNow, Jun 25, 2019
    Last edited: Jun 25, 2019
    2x better than humans ?

    For eg., Tesla is claiming they have done x # of lane changes (in millions) without a crash. I think in lane changes (may be merges too) cars have an inherent advantage, since they see all sides all the time. Their reaction time can be better too.

    ps : Interesting thing about the Waymo report is they are saying they test all these competencies - but don't say how well they do in them. They could have said how many such situations happened in 2018, for eg. The problem is Waymo, Cruise and others have such puny numbers compared to Tesla. Waymo says they completed 10 M miles of Av testing, Tesla says the fleet has done 1B miles of AP !
     
    • Like x 1
  12. diplomat33

    diplomat33 Active Member

    Joined:
    Aug 3, 2017
    Messages:
    2,195
    Location:
    Terre Haute, IN USA
    Glad I could help. :)

    We could do a table like this to keep track of Tesla's progress:

    upload_2019-6-25_13-15-24.png
     
    • Helpful x 1
  13. EVNow

    EVNow Well-Known Member

    Joined:
    Sep 5, 2009
    Messages:
    5,955
    Location:
    Seattle, WA
    I don't like the way those features / capabilities are … not organized. So, I'll do something similar to what I have now, but expand the capabilities to include all those listed features.
     
    • Helpful x 1
  14. EVNow

    EVNow Well-Known Member

    Joined:
    Sep 5, 2009
    Messages:
    5,955
    Location:
    Seattle, WA
    @diplomat33 I've now added all the NHTSA and Waymo extra features, under appropriate headings. Some of them are repetitions of what I already had in more detail, but I added them for completeness.

    Does anyone know if Cruise also publishes something similar ? I guess they are doing all the things NHTSA is recommending and may be some more of their own.
     
    • Helpful x 1
    • Like x 1
  15. EinSV

    EinSV Active Member

    Joined:
    Feb 6, 2016
    Messages:
    3,637
    Location:
    NorCal
    @EVNow great idea for a thread. One suggestion: the color scheme is not intuitive at all, which makes the chart unnecessarily hard to understand.

    Maybe adopt the automotive "standard" of red-yellow-green with "green" meaning "feature complete," red meaning "clearly not complete" and yellow somewhere in between.

    Readers could then quickly and easily process the info in the chart.
     
    • Like x 1
  16. EVNow

    EVNow Well-Known Member

    Joined:
    Sep 5, 2009
    Messages:
    5,955
    Location:
    Seattle, WA
    #76 EVNow, Jun 27, 2019
    Last edited: Jun 27, 2019
    Thanks, something to consider. I've 4 levels of "done" - so used 4 green saturations. Not done is light orange. May be I'll include a legend or turn 3 levels of pre-wide release "done" to yellow.
     
  17. diplomat33

    diplomat33 Active Member

    Joined:
    Aug 3, 2017
    Messages:
    2,195
    Location:
    Terre Haute, IN USA
    Thanks for adding the NHTSA and Waymo features.
     
  18. diplomat33

    diplomat33 Active Member

    Joined:
    Aug 3, 2017
    Messages:
    2,195
    Location:
    Terre Haute, IN USA
    Isn't feature 2.3 "perform car following (including stop and go)" the same as what TACC does? So, I would think AP already does that now.
     
  19. EVNow

    EVNow Well-Known Member

    Joined:
    Sep 5, 2009
    Messages:
    5,955
    Location:
    Seattle, WA
    @EinSV I've updated the color scheme.

    @diplomat33 That's correct. I've updated 2.3. Also, I was wondering whether "merge" (2.6) for city roads is done. I use AP in one city road where there is a merge, but not often enough to figure out whether it is aware of other cars and handles them.
     
    • Like x 1
  20. diplomat33

    diplomat33 Active Member

    Joined:
    Aug 3, 2017
    Messages:
    2,195
    Location:
    Terre Haute, IN USA
    I am not sure about merging on city streets.
     
Thread Status:
The first post in this thread is a WikiPost, and can be edited by anyone with the appropriate permissions.

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC