How else to interpret 2016 CA autonomous driving results?

[u00mem9]

I usually get a laugh out of the ability of fanboy culture on this site to rationalize almost any bad Tesla news. (Though lately it seems the messiah has left the building now that he is working cooperatively with the new administration!). But now, I come to you in my time of need.

Help me see the new results showing google averaging 5,000miles between driver intervention as anything other than clear indication that Tesla's AP2.0 bluster is anything more than hype...

(Tesla averaged 3miles per intervention and did a total of 550 miles vs Google 630,000)

 

[zambono]

The only thing I would say, is that 500 miles is very low for any real tests, and when I read the article the other day I thought the same. Seems more like developer testing just to do the video of the AP presentation. Tesla has the capability of thousands of miles of testing a day if they wanted too unlike the others except Waymo. Just deploy the software on 20 vehicles and let them go with 20 employees.

 

[McRat]

The Bolt in November ran 2264 miles with 6 disengagements. 377 miles per disengagement.

 

[jldf310]

I saw a good writeup here: The Numbers Don’t Lie: Self-Driving Cars Are Getting Good, Fast

I'm not defending Tesla (I love the "it seems the messiah has left the building" comment) - but the article does point out that the results aren't an "apples to apples" comparison and lists some reasons for the differences in the numbers.

 

[Maximapolak]

They have different distances, different road types, etc. What's there to compare?

Under Tesla, you see unknown road type. Who knows if most, if not all, the disengagements were on those road types.

Apples to Oranges.

 

[bradhs]

Drove 5 South to 805 South to 8 East from Villa De La Valle to El Cajon Blvd without disengaging AP. Could I have done that in my wife's BMW 750? Sure, but a HUGE difference. In the Tesla I didn't do much except for hold on to the steering wheel. In the BMW I would have had to hold on to the steering wheel and DRIVE THE CAR even with the auto steer engaged. It's not in the same league as the Tesla, not even close, you cannot compare them.

It's an apples to oranges comparison sprinkled with statistical comparison data that is not even in the same league. Don't waste you brainpower trying to understand it as it will never make sense.

 

[u00mem9]

Drove 5 South to 805 South to 8 East from Villa De La Valle to El Cajon Blvd without disengaging AP. Could I have done that in my wife's BMW 750? Sure, but a HUGE difference. In the Tesla I didn't do much except for hold on to the steering wheel. In the BMW I would have had to hold on to the steering wheel and DRIVE THE CAR even with the auto steer engaged. It's not in the same league as the Tesla, not even close, you cannot compare them.

It's an apples to oranges comparison sprinkled with statistical comparison data that is not even in the same league. Don't waste you brainpower trying to understand it as it will never make sense.

You're talking about autopilot 1.0. I think we all agree it's at the top of its class.

This discussion is about full autonomous capability. It looks like Google is basically ready to deliver the goods. 5000 miles average without intervention is really impressive.

Meanwhile Tesla is actually accepting payment for full self driving right now (and has been since sept.) while it seems they are struggling to recover to the original mobileye equipped capability of AP1.

There was a lot of mocking the Google effort when Tesla launched Autopilot, but now it seems like Google is way ahead, and the partners that have signed on with them have made a good decision.

 

[MorrisonHiker]

You're talking about autopilot 1.0. I think we all agree it's at the top of its class.

This discussion is about full autonomous capability. It looks like Google is basically ready to deliver the goods. 5000 miles average without intervention is really impressive.

Meanwhile Tesla is actually accepting payment for full self driving right now (and has been since sept.) while it seems they are struggling to recover to the original mobileye equipped capability of AP1.

There was a lot of mocking the Google effort when Tesla launched Autopilot, but now it seems like Google is way ahead, and the partners that have signed on with them have made a good decision.

I believe the vast majority (if not all) of Google's self-driving was at low speeds:
Google Explains Why Its Self-Driving Cars Only Go 25 MPH (For Now)
Google's self-driving car pulled over for driving too slowly - Roadshow

Full Self Driving wasn't announced by Tesla until October 19th: All Tesla Cars Being Produced Now Have Full Self-Driving Hardware

 

[McRat]

I believe the vast majority (if not all) of Google's self-driving was at low speeds:
Google Explains Why Its Self-Driving Cars Only Go 25 MPH (For Now)
Google's self-driving car pulled over for driving too slowly - Roadshow

Full Self Driving wasn't announced by Tesla until October 19th: All Tesla Cars Being Produced Now Have Full Self-Driving Hardware

November 2015 not 2016.

 

[MorrisonHiker]

November 2015 not 2016.

When I search for Google Self Driving speeds in 2016, the articles I found still mentioned 25-30 mph. Can you point me to something showing they are capable of faster speeds?

 

[McRat]

When I search for Google Self Driving speeds in 2016, the articles I found still mentioned 25-30 mph. Can you point me to something showing they are capable of faster speeds?

They do not break down speeds or miles travelled per type in this report format. However it does list that 12 of the 124 disengagements in 600k mi in 2016 were listed as Highway. One of the cars covered 21,500 miles (3k in a single month), which would be pretty difficult on public roads at a peak speed of 25mph.

https://www.dmv.ca.gov/portal/wcm/c...toWaymo_disengage_report_2016.pdf?MOD=AJPERES

 

[NerdUno]

Drove 5 South to 805 South to 8 East from Villa De La Valle to El Cajon Blvd without disengaging AP. Could I have done that in my wife's BMW 750? Sure, but a HUGE difference. In the Tesla I didn't do much except for hold on to the steering wheel. In the BMW I would have had to hold on to the steering wheel and DRIVE THE CAR even with the auto steer engaged. It's not in the same league as the Tesla, not even close, you cannot compare them.

It's an apples to oranges comparison sprinkled with statistical comparison data that is not even in the same league. Don't waste you brainpower trying to understand it as it will never make sense.

Perhaps the other car manufacturers are actually holding back their solutions until they are actually reliable AND safe.

 

[McRat]

I have a hunch that the most serious challenge is tight urban environments.

Cadillac has been working on their system for years now, but apparently at initial release, it will only work on interstates:

http://www.gminsidenews.com/articles/58553-2/

 

[calisnow]

View attachment 213013 View attachment 213012 I usually get a laugh out of the ability of fanboy culture on this site to rationalize almost any bad Tesla news. (Though lately it seems the messiah has left the building now that he is working cooperatively with the new administration!). But now, I come to you in my time of need.

Help me see the new results showing google averaging 5,000miles between driver intervention as anything other than clear indication that Tesla's AP2.0 bluster is anything more than hype...

(Tesla averaged 3miles per intervention and did a total of 550 miles vs Google 630,000)

@u00mem9 - okay here is my defense of Tesla:

1 - Tesla's neural net, based on Nvidia's Drive PX2 is/was close to brand new in October 2016. Tesla is literally the first implementation of Nvidia's hardware. Nvidia's system is designed for end to end, unsupervised (industry term - I didn't invent it - I do not know the *precise* meaning of "unsupervised" as used WRT artificial neural network learning) learning - and so in October it would have been a "baby" so to speak.

2 - Tesla's system can also, AFAIK, run in a shadow simulation mode, where a human being drives on the streets and the cameras in the car "pretend" to drive - and then compare what they did and would have done vs what the human being did. Therefore Tesla could be testing autonomous driving right now in shadow mode on company cars, but the data is not showing up in California's public information because the "automated driving" is being done in a simulation.

3 - Neural networks rapidly improve - so the first few months of data should show decreasing error rates until the curve levels off and the error rates asymptotically approach 0.

 

[calisnow]

@u00mem9

Further context for Tesla's position in October:

Remember Musk was in the position of needing to drum up support for his acquisition of Solar City (which was being panned by analysts and the press) - a vote was upcoming when this self driving video was released, if I remember correctly.

So I imagine the neural nets were brand new yet Musk was pushing the team to get a demo video of the car driving on streets ready to push out to the public. If this had not been the case they could have done much more testing on private test courses before releasing a video to the public - but they were on a time crunch. They could have released a video of a test course but it would not have been nearly as impressive.

Finally - regardless of where the nets are now, the more cars Tesla sells the larger the fleet of AP 2 hardware out there learning - so pumping sales with a demo video was probably a higher priority than getting low error rates. And rightly so, IMHO, from the perspective of company management. Because with this technology the more cars they sell, the better the networks get.

The real prize is not the S/X - it's getting self driving right on the 3. A bunch of pissy S owners on a message board is not a real corporate problem. Hundreds of thousands of 3 owners with self driving systems that don't work WOULD be a real corporate problem.

Like it or not the S/X crowd are the trainers of Tesla's self driving system - and the real customers are the mass market Model 3 buyers. Tesla needs as many S/X as possible with AP 2 hardware training the networks before the first deliveries of the Model 3 take place - so that initial Model 3 PR is good.

I hate to call the S/X buyers actually irrelevant to the big picture - but we kind of are.

 

[gordo]

What's to defend? Basically it states that as of Dec 31(?) only Google had done any sort of appreciable fully autonomous drive testing. The government requires number disclosure, so they disclosed. Why try to draw conclusions from a measly ~500 miles worth of calibration data? If you extrapolate conclusions about a project from a tiny data set at a nascent phase, you will often draw wildly inaccurate assumptions about the project overall. So just relax and come back in the summer and I guarantee you, it will be a different story.

 

[u00mem9]

Remember Musk was in the position of needing to drum up support for his acquisition of Solar City (which was being panned by analysts and the press) - a vote was upcoming when this self driving video was released, if I remember correctly.

I essentially agree with you, but what you are saying is they turned the hype knob up to 11 for the sake of manipulating investors. I know it's a post-fact world now and all that, but I still live where that's called a lie.

 

[Bladerskb]

They have different distances, different road types, etc. What's there to compare?

Under Tesla, you see unknown road type. Who knows if most, if not all, the disengagements were on those road types.

Apples to Oranges.

This is embrassing. Are you saying that tesla is testing on some top secret difficult road that google or even other companies don't have access to?

How can you be this naive? They have testing in the same cities. this is apples to apples.
Just accept the facts and move on, whats so hard? If this came out with tesla having 5,000 miles per disengagement and completely destroying the competition you will be singing from the rooftop. But since its the other way around, its irrelevant.

How convenient, that we can just pick and choose what statistic to pay attention to.

Drove 5 South to 805 South to 8 East from Villa De La Valle to El Cajon Blvd without disengaging AP. Could I have done that in my wife's BMW 750? Sure, but a HUGE difference. In the Tesla I didn't do much except for hold on to the steering wheel. In the BMW I would have had to hold on to the steering wheel and DRIVE THE CAR even with the auto steer engaged. It's not in the same league as the Tesla, not even close, you cannot compare them.

It's an apples to oranges comparison sprinkled with statistical comparison data that is not even in the same league. Don't waste you brainpower trying to understand it as it will never make sense.

It will never make sense because it doesn't fit your narative?

What you described is a level 2 driver assistance. What we are talking about is a level 5 fully self driving cars that is supposed to be able to drive anywhere without a human.

Its not apples to oranges, ITS THE SAME GOSH DARN APPLE!

You tesla fans amaze me

I believe the vast majority (if not all) of Google's self-driving was at low speeds:

No those are just the koola cars.

 

[jaguar36]

If you actually look at the report that Tesla submitted and not just the overall summary you'll see that they only did testing in October (and a bit in Novemeber, aka right around the time they were making the demo video. Since then they have had ZERO testing miles.

So from this we can draw two conclusions. They only did it to make a fancy video and aren't doing any more development work and are basically just ripping everyone off with the FSD hardware which they don't ever intend on following through with.

Or.. They are doing all of the developmental testing somewhere else. Either on private roads, or in a different state. (or most likely, both)

It seems quite obvious to me that the disengagements in that report are purely related to making the demo video (most of them also being in the rain), and are not at all indicative of the overall state of Tesla's development.

 

[stephenstohn]

View attachment 213013 View attachment 213012 I usually get a laugh out of the ability of fanboy culture on this site to rationalize almost any bad Tesla news. (Though lately it seems the messiah has left the building now that he is working cooperatively with the new administration!). But now, I come to you in my time of need.

Help me see the new results showing google averaging 5,000miles between driver intervention as anything other than clear indication that Tesla's AP2.0 bluster is anything more than hype...

(Tesla averaged 3miles per intervention and did a total of 550 miles vs Google 630,000)

Impressive! I think I'll go and buy a Google car. Oh, wait...