TMC is an independent, primarily volunteer organization that relies on ad revenue to cover its operating costs. Please consider whitelisting TMC on your ad blocker and becoming a Supporting Member. For more info: Support TMC

2016 California Autonomous vehicle disengagement reports

Discussion in 'Autopilot & Autonomous/FSD' started by RubberToe, Feb 1, 2017.

  1. LosAltosChuck

    LosAltosChuck Member

    Joined:
    Aug 1, 2017
    Messages:
    90
    Location:
    Los Altos
    Just catching up to this thread. It should be noted that the DMV report only covers disengagements on public CA roads. Tesla tests in other places, including non-public tracks where such reporting is not required. For example, substantial testing is done by them and others at Moffett Field in Mountain View on the runway and taxiways. (I've shared the runway with them during testing, so can state this with certainty.)

    To use that report as a measure of Tesla's (or any other manufacturer's) testing program would be highly misleading.
     
    • Informative x 1
  2. zmarty

    zmarty Member

    Joined:
    Jun 25, 2017
    Messages:
    220
    Location:
    Kirkland, WA
    I get what you are saying, but I disagree a little. In the end real life testing is what makes or breaks a system. I am looking forward to the 2017 report coming up in 2018. I am very curious how much testing on public roads they have done this year. I currently do not feel so good about the entire Autopilot program at Tesla, I think they are struggling and nowhere near the competition. Hopefully throwing money at the problem will help, but I doubt it.
     
    • Helpful x 2
  3. LosAltosChuck

    LosAltosChuck Member

    Joined:
    Aug 1, 2017
    Messages:
    90
    Location:
    Los Altos
    My point isn't that track testing is a substitute. It is that the DMV report doesn't come close to being a measure of the amount of testing any of the companies have done. Testing is done on local tracks (eg, Moffett), on non-CA roads (eg, Nevada), and a lot is done by simulation. Bay Area roads are convenient, but are also not the most appropriate when testing new systems. A previous commenter suggested that Tesla was in the dark ages compared to Google, providing a link to simulation work being done at Google. Nothing at the link made any suggestion that Tesla was *not* doing its own simulation work. I'm certain they are, they just haven't been public about what they are doing.

    All I'm suggesting is that there is not nearly enough public information available from any of the companies to compare testing and validation efforts, or to make a judgement about Tesla. There are hints and suggestions, but they don't establish fact.

    I drive my AP2 equipped MS in Autopilot mode as much as possible, but I do so knowing it is "beta" software, and I am thus a tester. I treat it with the same respect I treat the autonomous technologies I build. AP2 fails on me frequently enough that I've learned many of its weaknesses, and thus am training myself as I'm sure the data I'm generating is contributing to training the system.

    The one odd thing that occurs to me is this: since this is beta software, and we are all, thus, testers, why doesn't Tesla have to report the disengagements *we* experience on California roads? :)
     
    • Like x 1
    • Funny x 1
  4. zmarty

    zmarty Member

    Joined:
    Jun 25, 2017
    Messages:
    220
    Location:
    Kirkland, WA
    Probably because AP2 does not fall under the definition of "self driving" as per the DMV. It's just a driver aid.
     
    • Like x 1
  5. zmarty

    zmarty Member

    Joined:
    Jun 25, 2017
    Messages:
    220
    Location:
    Kirkland, WA
    Also, I made the stone ages comment. I find it extremely disingenuous that the only public testing they have done last year was for that video, and they had to try the same route for days to get decent footage and they disengaged hundreds of times. Google is at 5,000+ miles between disengagements. Tesla is at 3+ miles.

    I want FSD and I want it now :)
     
  6. LosAltosChuck

    LosAltosChuck Member

    Joined:
    Aug 1, 2017
    Messages:
    90
    Location:
    Los Altos
    But you have no evidence that it was the only public testing. The DMV report reflects only the public testing done on open CA roads. They wouldn't report disengagements at, say, Gomentum Station in Concord, where there are rumored to be testing, or in Nevada, where they have permits to test.

    I'd like to see more transparency, of course. I'd really like to know when I'm going to see substantial improvement, even in AP2. Mysterious, random updates with little information on content bothers me.
     
    • Like x 1
  7. zmarty

    zmarty Member

    Joined:
    Jun 25, 2017
    Messages:
    220
    Location:
    Kirkland, WA
    Sure, but if they tested 1 million miles on private roads, and then they came on public roads and had hundreds of disengagements in 3 days like they did, I would still say they are way behind. Based on the disengagement report, this is a fail either way (if they tested or not on private roads beforehand). In the end real life is what matters, so let's wait for the 2017 report.
     
    • Helpful x 2
  8. AnxietyRanger

    AnxietyRanger Well-Known Member

    Joined:
    Aug 22, 2014
    Messages:
    9,451
    Location:
    EU
    I think there is ample evidence to know Tesla has at least been at this FSD game a much shorter time than major competition. While this in itself is no proof of being behind, combined with the rest of what we can see, it does raise legitimate concerns and questions.

    Of course we do know Tesla is trying to do this the easy way, i.e. with minimal sensor fusion (while competition looks to mesh 360 radar and 360 lidar with vision), which might allow them moving quicker in some regards. With less sure results, though...
     
    • Like x 1

Share This Page

  • About Us

    Formed in 2006, Tesla Motors Club (TMC) was the first independent online Tesla community. Today it remains the largest and most dynamic community of Tesla enthusiasts. Learn more.
  • Do you value your experience at TMC? Consider becoming a Supporting Member of Tesla Motors Club. As a thank you for your contribution, you'll get nearly no ads in the Community and Groups sections. Additional perks are available depending on the level of contribution. Please visit the Account Upgrades page for more details.


    SUPPORT TMC