Navigant: GM Cruise, Waymo Lead Tesla And Others On Full Autonomy

[sootless]

Personally, I think it is much more important to look at how self-driving capability is being pursued than where they are today. I believe that the problem can only be resolved with AI/Big Data. Implementations using older traditional methods will show good initial results but progression is unlikely. Even those using AI without a data collection and analysis system will eventually hit a dead end.

Solving Autonomous driving Using traditional if/then/else programming is feasible to a point. It is feasible with a team of developers to handle 90-99.5% of the usual scenarios and shows initial results quickly. It is very difficult, if not impossible to handle all of the corner cases using traditional methods. Progression is eventually blocked.

The winner, in the end, will be the first to master AI coupled with Big Data scale pattern recognition. I believe that the transition from AP1 to AP2 was Tesla's move from traditional methods to AI. In the AI/Big Data arena I give Tesla the edge. Tesla collects far more data than anybody and seems to have a good handle on the issues and a plan to tackle them. I think Elon stated they collect 20x-100x the data of any other. Who else has a fleet of cars with an LTE network collecting and analyzing disengagements?

 

[electronblue]

@sootless Your message comes with two assumptions, one that is iffy and another one that is plain wrong:

1) All players are investing heavily into AI, MobilEye included (the company and chip range behind AP1), so nobody is really just doing rules — then again everyone, Tesla included, so far drives with traditional code
2) The iffy part: Will Tesla’s data gathering — the type than can be gleaned from consumer cars — be meaningful in training NNs? Will it offer a meaningful competitive advantage when offset with Tesla’s possible disadvantages or areas where others are ahead?

Interesting times for sure.

 

[OPRCE]

"CAPABILITY"!
Do you think that HE hasn't used HW3?

"Resultant deaths"?
That's bs and you know it.

1. I'm sure he has used HW3 but check the context in updated link, he is talking there about the NoAP tech then currently on the road and just about to be pushed, namely no more stalk-confirm for ALC.

2. Whence do you think Tesla drivers get the idea that they are driving L3 vehicles which require no human supervision as they attend to their email/Harry Potter movies while flying down the highway @80mph only to be decapitated by slicing under a crossing 18-wheeler or flambeed on a gore-point?

The correct answer is from Musk's deliberate equivocation [for the purpose of boosting sales] on the subject over several years up to and including the present.

 

[Cheburashka]

Musk has stated FSD will remain L2 until approved by regulators after billions of fleet miles for validation, which even in his optimistic imagination was not projected to be before end of 2020.

The epic fail here is that Level 2 is not FSD. Not even close.

If it tries to kill you even once out of every 100 drives it's still terrible.

If we look at the current state of NoA and EAP and compare that to Waymo's disengagements, it's easy to see why Tesla is way way behind.

 

[OPRCE]

The epic fail here is that Level 2 is not FSD. Not even close.

If it tries to kill you even once out of every 100 drives it's still terrible.

If we look at the current state of NoA and EAP and compare that to Waymo's disengagements, it's easy to see why Tesla is way way behind.

Right, but it has also been evident for ages that Tesla had no other feasible path to develop/release its pseudoFSD except piecemeal and under the same L2 nag regime as for EAP, as Musk has now clarified will be the case.

Taken together with the redefinition of FSD as limited to a few specific features to be delivered within a specified timeframe, it is at least a sign that some realism is starting to bite as a bit of pressure is applied to the vapourware.

 

[Brando]

GM is often first. Or at least before Tesla.
- making vehicles including Semi
- electrics; 1960s prototype - even electric Corvair; EV1
- improved buying experience of fixed prices, better "sales people/training: Saturn vs dealers

GM consistent about decreasing market share.

GM's share of the US market has decreased since its peak of 50.7 percent in 1962, falling to 17 percent in 2016.

 

[strangecosmos]

Navigant’s report costs $4,000, so unless somebody pays that, we won’t know exactly what this ranking is based on. It’s unclear to me ATM whether Navigant has any AI/robotics expertise in-house or interviewed AI/robotics experts for this report. If not, then why should Navigant’s ranking matter more than any random person’s ranking?

My view on the topic of autonomous vehicle competition is expressed in depth in this thread: Algorithms, compute, data: a mental model for thinking about AV competition It’s not based on expert interviews, but it’s also not $4,000.

If Tesla was even as far as say Nissan (not a leader) and why is it that they keep shipping out software that can't reliably stop for stopped vehicles (e.g. firetrucks and police cars) and can't read stop lights nor reliably detect pedestrians and bicyclists?

Nissan’s ADAS software has the same problem with stopped vehicles as Autopilot.

We have no idea what Nissan’s fully autonomous software is like beyond a short demo along a planned route.

Since Tesla's about hype, how come they've yet to demonstrate that they're at least equivalent to Nissan or Cruise Automation w/reporters on board? Or, how about a closed beta taxi program like Waymo had?

Seems pointless in Tesla’s case. If it’s production-ready, push it to customers. If it’s not production-ready, who cares?

Waymo and Cruise, who are seeking/have received outside investment, need demos because they have no other way of showing their work to the outside world. Tesla has 400,000+ customers with HW2 cars.

Have you looked at the California public road disengagement reports at Testing of Autonomous Vehicles for Tesla vs. Waymo/Google and Cruise Automation for each year?

Only safety-critical disengagements have to be reported, i.e. if the safety driver didn’t take over, the car would have crashed. Waymo’s safety-critical disengagement rate in California is once per 11,000 miles, compared to around 500,000 miles per crash in human driving. But based on two anecdotes from Waymo beta testers in the Phoenix area, there is a non-safety-critical disengagement around every 5 rides. Assuming rides are at most 20 miles on average, that’s a disengagement rate of at least once per 100 miles.

Two anecdotes is not a statistically significant sample size, but Waymo doesn’t disclose the true disengagement rate so it’s unfortunately the best info we have at the moment.

Waymo One is still in closed beta so we can only get more info if existing riders decide to share it.

 

[strangecosmos]

Paywalled, but in the preview you can see that Waymo riders made a complaint through the mobile app for 40% of rides: With Waymo Robotaxis, Customer Satisfaction Is Far From Guaranteed

The reporter shared an example on Twitter: Amir Efrati on Twitter (I recommend following Amir Efrati by the way; he does shockingly juicy investigative reporting on autonomy)

This is not to throw shade at Waymo, just to give more context to the 11,000 miles per safety-critical disengagement figure. It's not like the Waymo cars only make a mistake once per 11,000 miles.

 

[Sambas]

Seems pointless in Tesla’s case. If it’s production-ready, push it to customers. If it’s not production-ready, who cares?

Agree. Tesla haters (or paid by haters) need to drive a Tesla today to get a first hand experience on autonomous driving better than wasting time on power point presentations. No one outside of Mobileye employees has driven a car powered by their new system so until then no one knows how good it is compared to tesla. And Waymo!! man that is one ugly looking van they have to pay me to own one. That hump doesn't fit in my home garage I'm sure of that.

 

[cwerdna]

Only safety-critical disengagements have to be reported, i.e. if the safety driver didn’t take over, the car would have crashed. Waymo’s safety-critical disengagement rate in California is once per 11,000 miles, compared to around 500,000 miles per crash in human driving.

Where does the above bolded stat come from? I've never heard that and find it hard to believe it's that high. And, it's going to vary a lot by country. I suspect in countries like China, it's terrible but much better in Germany than the US.

But based on two anecdotes from Waymo beta testers in the Phoenix area, there is a non-safety-critical disengagement around every 5 rides. Assuming rides are at most 20 miles on average, that’s a disengagement rate of at least once per 100 miles.

Two anecdotes is not a statistically significant sample size, but Waymo doesn’t disclose the true disengagement rate so it’s unfortunately the best info we have at the moment.

Waymo One is still in closed beta so we can only get more info if existing riders decide to share it.

Even if they were true, where is Tesla's equivalent data in a similar setting, handling pedestrians, traffic lights, stop signs, etc.?

I never said Waymo is anywhere near perfect yet and I'd seen stories like 'I hate them': Locals reportedly are frustrated with Alphabet's self-driving cars from last year.

Seems pointless in Tesla’s case. If it’s production-ready, push it to customers. If it’s not production-ready, who cares?

Waymo and Cruise, who are seeking/have received outside investment, need demos because they have no other way of showing their work to the outside world. Tesla has 400,000+ customers with HW2 cars.

Judging by the types of updates Tesla pushes out that affect autopilot, they've pushed out many updates that aren't ready for production.

 

[Engr]

Last year, GM's cruise was ranked very high because they promised to produce cars without steering wheels in 2019. Awesome.!. The author believes GM & Waymo's tech are quite equal (despite other journalist reporting otherwise). Heck, he ranked Ford as no. 1 & relegated Waymo to the 7th position in 2017. WTF.!? Ford where simply dabbling in AV & will not make any meaningful stride untill 2018 with the Argo AI investment. There's a reason why AV companies don't brag about this report. It's a silly ranking from someone looking for attention.

 

[diplomat33]

Any thoughts on how Tesla will change in Navigant's rankings as a result of yesterday's event? I definitely think that Tesla will move way up in the rankings. Tesla has made significant progress on self-driving both in terms of hardware but also software and Tesla has a huge advantage of being able to deploy the software quickly to a very large number of cars already on the road. I say that puts Tesla in the "contender" category.

 

[WarpedOne]

Nope, Navigant will continue to refer to experts and maintain that lidar is a must.

 

[diplomat33]

Nope, Navigant will continue to refer to experts and maintain that lidar is a must.

Sad but true.

 

[malcolm]

Any thoughts on how Tesla will change in Navigant's rankings as a result of yesterday's event?

Won't change while Tesla continue to involve customers, which is too controversial for those who prefer a glossy press release and curated demos/test drives. But as Andrej pointed out, you need that real world data - both from the external sensors on the car and the genuine reactions of real drivers through the controls of the vehicle.

For me, the interesting take-away isn't the limitations of LIDAR (although it doesn't help) - rather it's the (over?) reliance on simulated driving.

 

[Ulmo]

I don’t know about GM, but I do consistently see Waymo here in the Bay Area. They’re usually autonomous minivans driving on local roads, respecting traffic lights and signs. They drive pretty aggressively, but they also seem to brake for even the slightest adjustment, instead of making use of the padding between the car in front. Kind of annoying if you’re behind a Waymo.

I've never seen them, or if I have, only once or twice near Mountain View in very limited form and with drivers in them driving; where are they more precisely?

 

[Ulmo]

Any thoughts on how Tesla will change in Navigant's rankings as a result of yesterday's event? I definitely think that Tesla will move way up in the rankings. Tesla has made significant progress on self-driving both in terms of hardware but also software and Tesla has a huge advantage of being able to deploy the software quickly to a very large number of cars already on the road. I say that puts Tesla in the "contender" category.

It seems that the purpose of Navigant is to say good things about the anti-Tesla group and bad things about Tesla; it would not be Navigant fulfilling its purpose of existence to ever admit to anything good about Tesla. Therefore, if Navigant ever said something good about Tesla being superior, it would amount to a white flag, an announcement of failure, surrender, and as a strategic signal of negotiation in some sort of negotiable role, but still not as a true signal of accurate quality. Soon thereafter, the Navigant group would be recycled.

 

[Engr]

It seems that the purpose of Navigant is to say good things about the anti-Tesla group and bad things about Tesla; it would not be Navigant fulfilling its purpose of existence to ever admit to anything good about Tesla. Therefore, if Navigant ever said something good about Tesla being superior, it would amount to a white flag, an announcement of failure, surrender, and as a strategic signal of negotiation in some sort of negotiable role, but still not as a true signal of accurate quality. Soon thereafter, the Navigant group would be recycled.

The Navigant researcher, Samuel, have some sort of a personal vendetta against Elon. I'll be surprised if this presentation changes his perception at all. It doesn't help that Elon tends to lash back against the media & so it creates that -ve perception. While the media are quick to use this ranking, none of the AV companies is commending this as a research or looking at it like it's a corvetted ranking.