I know the deadline for companies to submit their autonomous driving report to the CA DMV was Jan 1, 2020. That was a month ago. So when does the CA DMV release those reports to the public?
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
When does the CA DMV autonomous driving report come out?
- Thread starter diplomat33
- Start date
Daniel in SD
(supervised)
It was Feb 13th last year: California’s self-driving car reports are imperfect, but they’re better than nothing
Yup middle of February last year posted here: Autonomous Vehicle Disengagement Reports 2018
And as from The Verge article about reports being imperfect, it's easy to make misleading conclusions.
A couple companies have already written downplaying disengagements:
Cruise: The Disengagement Myth
Embark: 2019 Disengagement Report Update
And as from The Verge article about reports being imperfect, it's easy to make misleading conclusions.
A couple companies have already written downplaying disengagements:
Cruise: The Disengagement Myth
Embark: 2019 Disengagement Report Update
Thanks. I am aware of the blogs downplaying the disengagement report. But I am still curious what the numbers will be for Cruise and others. I am also curious if Tesla reports the disengagements from their autonomy day demo or weasel out again.
I checked the CA DMV website for an update and found a new list of all the companies that received autonomous driving (with safety driver) permits from the DMV this month.
As of Feb 5, 2020, there are now 66 companies that received permits to test autonomous driving (with safety driver) in California:
As of Feb 5, 2020, there are now 66 companies that received permits to test autonomous driving (with safety driver) in California:
- Volkswagen Group of America
- Mercedes Benz
- Waymo LLC
- Delphi Automotive
- Tesla Motors
- Bosch
- Nissan
- Cruise LLC
- BMW
- Honda
- Ford
- Zoox, Inc.
- Faraday & Future Inc.
- Baidu USA LLC
- Valeo North America, Inc.
- NIO USA, Inc.
- Telenav, Inc.
- NVIDIA Corporation
- AutoX Technologies Inc
- Subaru
- Udacity, Inc
- Navya Inc.
- Renovo.auto
- PlusAi Inc
- Nuro, Inc
- Udelv
- Apple Inc.
- Pony.AI
- TuSimple
- WeRide Corp
- SAIC Innovation Center, LLC
- Almotive Inc
- Aurora Innovation
- Nullmax
- Samsung Electronics
- Continental Automotive Systems Inc
- Voyage
- CYNGN, Inc
- Roadstar.Ai
- Changan Automobile
- Lyft, Inc.
- Phantom AI
- Qualcomm Technologies, Inc.
- SF Motors Inc.
- Toyota Research Institute
- Apex.Al
- Intel Corp
- Ambarella Corporation
- Gatik AI. Inc.
- DiDi Research America LLC
- TORC Robotics Inc
- Boxbot Inc
- EasyMile
- Mando America Corporation
- Xmotors.ai, Inc.
- Imagry Inc.
- Ridecell Inc.
- ThorDrive Inc
- Helm.AI Inc
- Argo AI, LLC
- Qcraft.ai
- Atlas Robotic, Inc.
- Deeproute.ai Ltd
- Kaizr, Inc
- Leonis Technologies North America LLC
- Uber Advanced Technologies Group
Last edited:
2019 CA DMV Disengagement report is out!!
2019 Autonomous Vehicle Disengagement Reports
Tesla only reported autonomous miles from 1 VIN number and reported 12.2 miles in March 2019 and 0 disengagements. So it looks they simply reported the FSD demo from Autonomy Investor Day!
2019 Autonomous Vehicle Disengagement Reports
Tesla only reported autonomous miles from 1 VIN number and reported 12.2 miles in March 2019 and 0 disengagements. So it looks they simply reported the FSD demo from Autonomy Investor Day!
Daniel in SD
(supervised)
Zero disengagements puts Tesla far ahead of its competitors!
So I guess they're claiming that all those demo rides they did weren't autonomous?
I made a table showing the results of the disengagement report in an easily readable format:
Yup, Tesla is way ahead in miles per disengagement!Zero disengagements puts Tesla far ahead of its competitors!
Code:
Manufacturer #Dis. Miles Miles/Disengagement
Tesla, Inc. 0 12 #DIV/0!
Baidu USA LLC 6 108300 18050
Waymo LLC 110 1454137 13219
CRUISE LLC 68 831040 12221
AutoX Technolog 3 32054 10685
PONY.AI, INC. 27 174845 6476
Nuro 34 68762 2022
Zoox, Inc 42 67015 1596
PlusAI, Inc. 2 1880 940
AImotive Inc. 26 6056 233
WeRide Corp 39 5917 152
Apple Inc. 64 7544 118
Aurora Innovati 141 13429 95
Qualcomm Techno 37 2164 58
SAIC Innovation 40 2230 56
Drive.ai Inc 75 3974 53
Nissan North Am 47 2277 48
Nullmax 70 2430 35
Phantom AI, Inc 43 1125 26
Lyft 1667 42931 26
SF Motors, Inc. 140 3493 25
NVIDIA 655 7218 11
Telenav, Inc. 3 22 7
Mercedes0Benz R 2054 14238 7
BMW of North Am 8 21 3
Udelv, Inc 444 707 2
Valeo North Ame 92 100 1
Toyota Research 2947 1817 1Zero disengagements puts Tesla far ahead of its competitors!
I know you are being tongue and cheek. The fact is that these numbers do show how far behind Tesla is. Only 12 miles is pathetic.
I also wish Tesla fanboys would stop dismissing Waymo and Cruise. They are the clear leaders of autonomous driving. Cruise did 800k of real autonomous miles, mostly in the city I assume since they test in SF, with a disengagement rate of 12,221 miles per disengagement.
I would say that Waymo and Cruise definitely have L4 autonomous driving!
I guess not. Weird. So the demo rides that had some disengagements somehow did not get reported. Weird.
Daniel in SD
(supervised)
I would love to see videos of all their disengagements and their simulation results of what would have happened had the disengagement not occurred. As it stands I'll believe it when they do a million miles without a test driver.
And of course I agree that they're way ahead but it's hard to say how close they are to having a viable product.
I wonder how much of these other companies miles are required in order to gather data? I guess it would make sense for them to gather data AND test at the same time. Tesla's biggest advantage is their fleet gathering data with over 2 billion miles logged. If you look at it purely from total amount of data collected (and distribution) perspective, Tesla is still far ahead. It still is concerning though that Tesla has not logged any other real world miles other than one run "Hey look, we can do it." for autonomy day.
I agree videos of the disengagements would be very informative. However, if we look at the causes of the disengagements, we do get some useful hints.
Cruise's disengagements were caused by the following:
"precautionary takeover to address perception, AV made unsuccessful left turn"
"precautionary takeover to address perception, other road user behaving poorly"
"precautionary takeover to address planning, precautionary takeover to address controls, AV made unsuccessful left turn"
"precautionary takeover to address planning, third party lane encroachment"
"precautionary takeover to address planning, AV lane change issues"
"precautionary takeover to address planning, third party lane obstruction"
So we can see that the Cruise's AV encountered various issues like another car behaving poorly or a lane obstruction or the car was not making a left turn quite right but all the disengagements were precautionary, meaning that the safety driver disengaged to be extra careful and safe, not necessarily because of an actual accident or failure. We can also see that most of the issues were planning not perception, meaning the issue was not the car not seeing the right thing but the car not planning the right path. So Cruise is still trying to work out some path planning in these edge cases.
Considering that they had only 68 precautionary disengagements out of 800k autonomous miles, I think we can conclude that Cruise's AV can handle city driving extremely well and Cruise is just trying to work out some edge cases where the AV does not quite handle the planning part well enough. So yeah, I would say that they have L4 autonomy, just not L4 autonomy that can handle 100% of the edge cases yet.
Waymo had 110 disengagements over 1.4M autonomous miles. So the disengagement rate was 13k miles per disengagement.
Waymo's causes of disengagements:
"Disengage for unwanted maneuver of the vehicle that was undesirable under the circumstances"
"Disengage for a perception discrepancy for which a component of the vehicle's perception system failed to detect an object correctly"
"Disengage for a hardware discrepancy for which our vehicle's diagnostics received a message indicating a potential performance issue with a hardware component of the self-driving system or a component of the base vehicle"
"Disengage for incorrect behavior prediction of other traffic participants"
"Disengage for a recklessly behaving road user"
Waymo obviously has a lot of autonomous miles and a good disengagement rate. They clearly have L4 autonomy that works extremely well most of the time. However, their disengagements seem more troubling to me as it appears they have some hardware failures and perception failures.
Waymo's causes of disengagements:
"Disengage for unwanted maneuver of the vehicle that was undesirable under the circumstances"
"Disengage for a perception discrepancy for which a component of the vehicle's perception system failed to detect an object correctly"
"Disengage for a hardware discrepancy for which our vehicle's diagnostics received a message indicating a potential performance issue with a hardware component of the self-driving system or a component of the base vehicle"
"Disengage for incorrect behavior prediction of other traffic participants"
"Disengage for a recklessly behaving road user"
Waymo obviously has a lot of autonomous miles and a good disengagement rate. They clearly have L4 autonomy that works extremely well most of the time. However, their disengagements seem more troubling to me as it appears they have some hardware failures and perception failures.
Daniel in SD
(supervised)
Interesting that none of the disengagements were initiated by the vehicle. I agree that they do appear close to deployment. I'm just always skeptical!
You can see all the accident reports here: Report of Traffic Collision Involving an Autonomous Vehicle (OL 316)
I haven't read them but I bet there's some interesting information.
Daniel in SD
(supervised)
I've looking at a sampling of the Cruise accidents. They mostly involve the car getting rear ended. What's interesting is that in many of the cases the safety driver disengages before getting hit. I interpret this to mean that they've instructed the safety drivers to take over when the car behaves in unpredictable manner (like phantom braking).
There are also a surprising number of accidents while the car is being driven by a human driver. I'm curious why they are driving around in "conventional" mode. It makes me suspicious.
There are also a surprising number of accidents while the car is being driven by a human driver. I'm curious why they are driving around in "conventional" mode. It makes me suspicious.
insaneoctane
Well-Known Member
12 miles. hmmmmm. Literally the fewest miles among all those other companies. Either Tesla kept the nag turned on for all their testing miles so that they don't have to report or they just test across the state border, right?
Daniel in SD
(supervised)
They had a little person in the frunk driving the demo rides.
There's no way they would violate the autonomous testing regulations which say nothing about nags. Cruise and Waymo also have nags BTW.
The car getting rear ended could just be the other car not stopping in time. It does not necessarily mean the Cruise AV did anything wrong.
Similar threads
