When does the CA DMV autonomous driving report come out?

[S4WRXTTCS]

At least it seems like many (most?) companies are complying but if Tesla is just skirting the law or willfully violating it, you think that's going to look very good in the eyes of CA and other state regulators? CA DMV could pull their permit. I'm not a lawyer but perhaps they could even sue Tesla and others not in compliance.

I was just stating what Tesla is doing. I'm not supporting them giving the CA DMV the middle finger, and I'm simply stating that it is what they're doing.

For the record I completely agree with everything you wrote in that quoted section.

At this point the ball is in the CA DMV hands to choose what to do about it.

 

[S4WRXTTCS]

If everyone thinks it's so stupid, why did they comply and report all their miles? Cruise CEO criticized the disengagement report as not being the best metric to measure FSD progress but Cruise still reported 800K miles!!! Heck some companies have horrible disengagement rates but they still reported them.

Because the industry gives off the "we'll comply, but we don't think this makes any sense whatsoever" vibe. We only know what the Cruise CEO says openly about it. Imagine what he says behind closed doors.

Because I was trying to convey the sentiment of what Elon Musk likely thinks of it.

Because I'm personally I'm against it so that obviously plays a role in using language that is admittedly a bit harsh.

 

[diplomat33]

Because the industry gives off the "we'll comply, but we don't think this makes any sense whatsoever" vibe. We only know what the Cruise CEO says openly about it. Imagine what he says behind closed doors.

Because I was trying to convey the sentiment of what Elon Musk likely thinks of it.

Because I'm personally I'm against it so that obviously plays a role in using language that is admittedly a bit harsh.

Personally, I think the real reason Tesla is not reporting any meaningful autonomous miles is because they don't have real autonomous driving yet, not because Elon does not like the disengagement report process. After all, if Tesla had super good autonomous driving, I am sure Tesla would be reporting a ton of miles and bragging about how great their disengagement rate is.

 

[S4WRXTTCS]

Personally, I think the real reason Tesla is not reporting any meaningful autonomous miles is because they don't have real autonomous driving yet, not because Elon does not like the disengagement report process. After all, if Tesla had super good autonomous driving, I am sure Tesla would be reporting a ton of miles and bragging about how great their disengagement rate is.

I don't think they're racking up the autonomous driving miles. Like I don't think they have more than 100,000 miles of autonomous driving. But, I don't think they would have done autonomy day without at least having a done the route(s) lots of times.

You don't risk making a bad impression with investors. Even the Cyber truck reveal was rehearsed a bunch of times. They only screwed up because they didn't test EXACTLY what they did during the reveal.

I do agree with you in that they're pretending to be an autonomous player while they really aren't. But, I also think they intentionally left out a lot of miles from the report. Not L2 miles driven by normal drivers, but autonomous miles driven by safety drivers with no hands on the steering wheel restrictions. The entire autonomy day as I understand it was done without hands on the steering wheel except during take over events. That itself was likely hundreds of miles with some disengagements reported (by people riding in them).

 

[diplomat33]

Here are two quotes from the letter Tesla attached to their disengagement report:

"“For Reporting Year 2019, Tesla briefly tested an autonomous vehicle in autonomous mode on public roads in California. In April, we operated one vehicle in autonomous mode to record one demo run on a 12.2-mile route around Tesla’s Palo Alto headquarters. The route covered surface streets and highways. We did not experience any autonomous mode disengagements during this run and, as a result, do not have any disengagements to report for Reporting Year 2019.”

"“For background on how Tesla develops software capability, first, using industry best practices, we perform hardware and software in-the-loop testing, system-level and regression testing, simulations, test track and/or on-road testing (not autonomous), and a battery of cross-functional reviews, hazard analysis, risks assessments, and failure modes and effects analysis. Second, we rely heavily on fleet learning. Tesla is the only participant in the AVT program with a fleet of customer-owned vehicles in the hundreds of thousands. Virtually all customers consent to Tesla running developmental feature software, including for AV capability, in “shadow mode” during their normal driving operation. Features in “shadow mode” run silently in the background without actuating any vehicle controls whatsoever, which enables Tesla to test how features will perform in real-world driving conditions before we deploy them to the customer fleet. As a result, we are able to collect billions of miles of anonymized driving data remotely over the air, including on targeted roadways and in driving situations that we later use to train AV features to perform safely, consistently, and predictably.”

Just sharing for informational purposes.
https://electrek.co/2020/02/27/tesla-reports-fully-autonomous-miles-first-time-years/

 

[Daniel in SD]

It's not a million miles, but Cruise posted another video of their car driving 1 hour in autonomous mode on city streets with no disengagements. It certainly gives a good idea of the kinds of scenarios that the car can handle:

I've watched some of that before and it's quite impressive but it's less than 20 miles! I think it can be deceptive because you can probably judge a human driver pretty well by sitting in the passenger seat for 20 miles through San Francisco.

I still can't understand the logic behind only reporting 12 miles of testing. They're basically thumbing their nose at the DMV. They gave test rides to the public on Autonomy Day (which did reportedly have disengagements).
I expect that the DMV won't do anything unless there is a complaint, moving violation, or accident.

 

[S4WRXTTCS]

From the disengagement reports itself the only one I'm really interested in spending the time looking into is the Toyota Research Institute.

Just what in the heck were they doing? They had 2,947 disengagements over 1,817 miles.

That's one jumpy safety driver. Or they we researching some specific technology that was really designed to be taken over from.

 

[diplomat33]

From the disengagement reports itself the only one I'm really interested in spending the time looking into is the Toyota Research Institute.

Just what in the heck were they doing? They had 2,947 disengagements over 1,817 miles.

That's one jumpy safety driver. Or they we researching some specific technology that was really designed to be taken over from.

Definitely a jumpy safety driver, I think, because they list a lot of causes for the disengagements as "Safety Driver proactive disengagement".

 

[DrDabbles]

They think highly of themselves, but they're an ineffective dumping ground of idiots.

You may be misunderstanding the purpose of the HR department. It's to prevent employees from creating situations that damage the company.

I'd much prefer it if Tesla didn't keep selling people features that don't exist and are likely not going to be available during the expected life of the car.

I agree. Which is why I've suggested rolling new features out as they exist. Ex: emergency stop at traffic control devices if the driver doesn't attempt to stop in time.

 

[run-the-joules]

You may be misunderstanding the purpose of the HR department. It's to prevent employees from creating situations that damage the company.

Yeah, I know. They tend to create those situations a lot themselves, in my experience.

 

[cwerdna]

BTW, to actually support what S4WRXTTCS said about "everyone" , there's Waymo doesn't like California's benchmark for self-driving research and Everyone hates California’s self-driving car reports now...

 

[Daniel in SD]

BTW, to actually support what S4WRXTTCS said about "everyone" , there's Waymo doesn't like California's benchmark for self-driving research and Everyone hates California’s self-driving car reports now...

I agree, it seems that a few companies have hit a rate of disengagements where the metric is now meaningless. It’s not like it’s much of a regulatory burden though, they already have to thoroughly analyze every disengagement anyway.

 

[diplomat33]

I agree, it seems that a few companies have hit a rate of disengagements where the metric is now meaningless. It’s not like it’s much of a regulatory burden though, they already have to thoroughly analyze every disengagement anyway.

I don't think disengagements themselves are a bad way to measure autonomous driving. After all, we want an autonomous driving system that can drive safely for the longest period of time without requiring a human to intervene or turn it off. So we want a good disengagement rate. Now, disengagements are not the only way to measure autonomous driving but they will certainly be one piece of it. The key is that you need context. Any stat taken out of context won't mean much. In the case of disengagements, you need context too. Specifically, you need to know what caused the disengagement. Some disengagement causes will be more innocent than others.

You also need to know the ODD of the system to get a sense of where and when the autonomous driving system can be used. A disengagement rate in city driving will mean something different than the disengagement rate in highway driving. You also need to know how often the same route was taken. Disengagement rate for a single 3,000 miles coast to coast trip won't mean the same as the disengagement rate for 3,000 miles of the same 500 miles route done over and over again. Also, if the disengagement rate is measured over several months where the company was changing and improving their system, then the disengagement rate will be more an average. Maybe your disengagement rate at the start was really bad and it got better by the end as you fixed issues. So knowing all these details will great help in better understanding what the disengagement rate actually means.

 

[S4WRXTTCS]

BTW, to actually support what S4WRXTTCS said about "everyone" , there's Waymo doesn't like California's benchmark for self-driving research and Everyone hates California’s self-driving car reports now...

The Verge article was actually written earlier on the same day as my comment so I probably came across the headline while browsing a google or apple news feed. It likely seeded my subconscious.

But, I had the other one beat.