Waymo One launches

[electronblue]

I should add many limited scope Level 3-4 projects ”like” freeways too for their simplicity.

So in the Waymo vs. Tesla comparison a much more pertinent point is Waymo has been working on a car that can go from start to finish autonomously. Irrespective of the SAE Level this is a hard nut to crack and one where city streets are the more important problem because they can not be avoided when driving from start to finish autonomously.

Simplicity of freeways is no help for such a car because it will also have to be able to join the freeway autonomously unlike other systems that can just rely on a driver to join the freeway.

 

[diplomat33]

It's interesting that Waymo does not like freeways but Tesla welcomes freeways.

Why the difference?

Waymo's goal is to have a urban self-driving taxi service that will make money for the company. This requires a self-driving car that can handle urban environments, hence city streets. Tesla is starting from the perspective of an auto maker with customers already driving the car on various roads. For Tesla, it made to sense to start with a driver assist system that would help their customers. So Tesla started with TACC and Auto Steer which is more useful on freeways than city streets.

 

[mongo]

That was hardly my point though. As I said, a prototype of a Level 4/5 car. We know it can self-drive so certainly Waymo’s are Level 4 cars and I would say aim to be Level 5 hence I framed it as Level 4/5 car/prototype.

But my point was a Level 4/5 prototype going from start to finish can not avoid city streets but it can avoid freeways that are hard to join and where speed kills. Hence it makes sense to avoid them if needed.

A Level 2 car on the other hand likes freeways for their simplicity because it does not have to worry about self-driving to get there or about being in charge. So it makes sense a Level 2 car avoids city streets and ”likes” freeways.

Aye. Good points.
But isn't it more accurate to say it is intended to be level 4/5, but is currently level 2 (or level 3 if the driver doesn't need to pay attention).

 

[Bladerskb]

Beats the heck out of me. I'd expect a big improvement (although it may be offset from HW3 roll out to validate on larger pool)

Here's my point of view. The contentions that outsiders have with AP is not that it will improve and get better. But that it will be level 5 in the time period that Elon vowed and guaranteed. Which was initially end of 2017 / Early 2018.

This has lead to everyone with a Tesla position to preach that Tesla is 3+ years ahead of the competition. Some say 5, some say 10.
You can't go anywhere without some uninformed fan telling you that Tesla is a decade ahead in autonomy.

Yet where AP is at today is just an incremental improvement from AP1. Which everyone expected We have yet to see the exponential explosion that Trent keeps preaching about.

Take for example, if i told you i could run a mile in 5 minutes and you say you can do it in 2 minutes.
After every .025 miles you ran you made it known through an over the air update.
While me, i keep it all inhouse and say nothing about it.

After 2 minutes your latest OTA shows you at .1 mile. So not only did you not run a mile in 2 minutes but you ran only .1 miles, meaning it will take you 8 extra minutes, 10 total to get to 1 mile. For you to even get to 1 mile in 5 minutes would require a breakthrough. Me on the other hand based on my projected 1 miles in 5 minutes if i was on track would be internally at .4 miles. 4x ahead of you.
Yet because i don't do OTA updates, people believe you are way ahead based on what they can see feel and touch because they refuse to do research or any critical thinking.

This is exactly what's playing out with AP. Tesla fans believe that AP is decade ahead of the competition but aint really doing the math which is that if the competition is claiming to release a SDC in 2021 then they have to be at or ahead of the publicly available offering by Tesla.

Elon is already a year late into his Level 5 promise, for him to get that deployed in 2019. The rate of improvement must be exponential.
So you either believe some magical exponential update will happen like Trent and Level 5 will be achieved in a manner of months or you admit that Tesla won't have Level 5 in 2019 or 2020 or 2021...

Looks like you do believe in an exponential update so we will see. Hopefully you don't backtrack when i bring this up next year.

 

[electronblue]

But isn't it more accurate to say it is intended to be level 4/5, but is currently level 2 (or level 3 if the driver doesn't need to pay attention).

That’s pretty fair. I’m not sure calling it Level 2 is entirely accurate as they have driven them without a driver but again in this context the Level number was not the point anyway.

 

[Bladerskb]

According to the SAE. All L3, L4 or L5 test cars are identified at their appropriate design specifications.

For example, a L4 test car isnt a L2 just because a driver is there taking over during development/testing.

The levels are actually assigned by the systems design not by how well they perform. Manufacturer and regulators are the ones who decide what level of performance is sufficient for them to deploy.

Some people say that Elon doesn't like or use levels (like l4 or l5) but uses 1x, 2x, 10x better than a human driver instead. But actually the levels dont define performance, they only define system design. You can theorically have a level 5 that crashes every 100 miles. But ofcourse no one will buy it and the manufacturer would drown in lawsuits. (That's if the regulators allow it to be sold)

SOURCE: SAE 2018 DOCUMENT

 

[mongo]

Here's my point of view. The contentions that outsiders have with AP is not that it will improve and get better. But that it will be level 5 in the time period that Elon vowed and guaranteed. Which was initially end of 2017 / Early 2018.

This has lead to everyone with a Tesla position to preach that Tesla is 3+ years ahead of the competition. Some say 5, some say 10.
You can't go anywhere without some uninformed fan telling you that Tesla is a decade ahead in autonomy.

Yet where AP is at today is just an incremental improvement from AP1. Which everyone expected We have yet to see the exponential explosion that Trent keeps preaching about.

Take for example, if i told you i could run a mile in 5 minutes and you say you can do it in 2 minutes.
After every .025 miles you ran you made it known through an over the air update.
While me, i keep it all inhouse and say nothing about it.

After 2 minutes your latest OTA shows you at .1 mile. So not only did you not run a mile in 2 minutes but you ran only .1 miles, meaning it will take you 8 extra minutes, 10 total to get to 1 mile. For you to even get to 1 mile in 5 minutes would require a breakthrough. Me on the other hand based on my projected 1 miles in 5 minutes if i was on track would be internally at .4 miles. 4x ahead of you.
Yet because i don't do OTA updates, people believe you are way ahead based on what they can see feel and touch because they refuse to do research or any critical thinking.

This is exactly what's playing out with AP. Tesla fans believe that AP is decade ahead of the competition but aint really doing the math which is that if the competition is claiming to release a SDC in 2021 then they have to be at or ahead of the publicly available offering by Tesla.

Elon is already a year late into his Level 5 promise, for him to get that deployed in 2019. The rate of improvement must be exponential.
So you either believe some magical exponential update will happen like Trent and Level 5 will be achieved in a manner of months or you admit that Tesla won't have Level 5 in 2019 or 2020 or 2021...

Looks like you do believe in an exponential update so we will see. Hopefully you don't backtrack when i bring this up next year.

I wouldn't say exponential, I'd say generational. AP2 isn't going to make a huge leap (well, I guess anything is possible). AP 3 having a multiple of processing power will be unlike AP2.

For the race analogy, one could compare bicycling to waiting for a (delayed) train. Or a rocket that does not have a greater than one thrust to weight ratio to one that does.

For example, a L4 test car isnt a L2 just because a driver is there taking over during development/testing.

I argue, if the driver is needed when the car is doing it's level 4 thing, it is really level 3. But sure, getting to/ from the level 4 designated area would require a driver and not make it a level 3 or lower.

 

[Bladerskb]

I wouldn't say exponential, I'd say generational. AP2 isn't going to make a huge leap (well, I guess anything is possible). AP 3 having a multiple of processing power will be unlike AP2.

For the race analogy, one could compare bicycling to waiting for a (delayed) train. Or a rocket that does not have a greater than one thrust to weight ratio to one that does.

What makes you so sure that the train will ever arrive before the bicyclist gets to the finish line?
Infact what makes you think that the bicyclist doesn't also have its own train or that there isn't another rocket with capable trust?

or that the bike has the speed and power of a train?

BMW says Level 5 self-driving car for public could happen by 2021

On Tuesday, BMW executives said they could sell a car to the public that would be capable of driving itself—without a steering wheel. The only factors keeping them from selling a completely self-driving car? Government regulations and high-resolution maps.

“We are on the way to deliver a car in 2021 with level 3, 4 and 5,” Frickenstein told a panel discussion in Berlin, explaining the vehicle will have different levels of autonomy, depending on how and where it is used.

Dr. Dirk Wisselmann, senior expert for autonomous driving at BMW, tells me that the automaker's first level 3 car will have the technical capabilities for level 4 or 5 highway driving. "We can create a software update (for the car) and inform our drivers, 'We are are very confident on this road. We are very sure nothing can happen. You can sleep if you want to.'" He makes sure to note that this is a best-case scenario.

"The idea is that we really have to develop a car today on a highly sophisticated level." The other is a fleet vehicle with level 4 (and potentially level 5) capabilities that will be part of BMW's upcoming branded ride-hailing service.
The automaker plans on delivering both of these vehicles in 2021.

“Everybody looks at all the Audis and Teslas and GMs and such. Our level 3 will be much beyond that, not only in performance but also in reliability. We’re working very hard so that it works when it’s foggy, when it’s snowing,” Froehlich said. The sensors would be self-cleaning, so they don’t get blinded by falling snow, for example.

“It wouldn’t be right now to take it step-by-step, making the arduous way from the low second level to the third. Due to the system leaps, the additional requirements in terms of redundancy, computing power and connectivity for fully automated driving, we will be able to fully master Level 5 right from the outset in 2021.”

“From then on, we will be able to offer automation solutions worldwide in a flexible range of level 3 to level 5 – depending on customer demands and the respective legal framework, just like the modular systems that you already know from our e-mobility solutions.”

Mr Froehlich added that BMW’s autonomous systems would be modular, which would allow its future models to be capable of all levels of autonomous driving, despite the fact that he believes the technology will easily outpace autonomous vehicle regulation.

“I developed very modular (systems), like I did on powertrains. I don’t know how much Level 4 systems will be required. Will ride sharing work or not? So I am flexible too. Level 3, 4, 5 is modular. So the redundancy of the Level 4 system is simply my Level 3 system I have developed anyway.

“The software module, the stack, is modular, so I do not invent the wheel new like others do. They have a Level 3 system that is completely independent of the 4 system.

“So Level 4 will be also ready in 2021, the car (iNext) is capable for Level 4, but I do not know today where I will pilot this because I do not know where the regulators will move. I personally think that China will be the fastest and I think Europe will be the slowest.”

Now I'm not one that believes anything an automaker says. But because BMW is using mobileye for sensing, mapping and driving policy, it puts more weight to what they are saying.

 

[mongo]

What makes you sure that the train will ever arrive before the bicyclist gets to the finish line?
Also what makes you think the bicyclist doesn't also have its own train or that there isn't another rocket with capable trust?

What makes either of us think analogies have any meaning?FWIW, I was attempting to illustrate the step change of AP 2.x to AP 3, not Tesla to others.

Honest opionion: isn't this statement wrong?

“We are on the way to deliver a car in 2021 with level 3, 4 and 5,” Frickenstein told a panel discussion in Berlin, explaining the vehicle will have different levels of autonomy, depending on how and where it is used.

Level 5 can't be use case dependent, right?

Most of those quotes read to me as if the sensor and processor will be set up to support 3-5, but the software will be level 3 in 2021.

 

[Bladerskb]

What makes either of us think analogies have any meaning?FWIW, I was attempting to illustrate the step change of AP 2.x to AP 3, not Tesla to others.

totally missed that. But my point was that Tesla isn't the only company aiming for level 5. Although its the loudest.

Honest opionion: isn't this statement wrong?

Level 5 can't be use case dependent, right?

You're right.

But I think what they are doing or trying to say rather is that they have small teams working on different features and some features represent a particular level. Then they can sell separate systems starting with the level 3 highway software to customers (with level 5 capable hardware) and level 4 for ride hailing service in specific cities, while also testing level 5 features that they are still developing.

They might also offer an customer update in a particular city/cities or highway for level 4 driving at a upgrade cost (who knows) until level 5 software is ready.

At-least that's what i think from their statements.

All of this is still very fluid which is why Mobilete's Amnon doesn't talk about it (Level 5 aspect). If you notice Amnon is the anti-hype of the SDC industry. He only talks about things that are rock solid with signed production contractual agreements.

Most of those quotes read to me as if the sensor and processor will be set up to support 3-5, but the software will be level 3 in 2021.

True but its worth pointing out that the software will be modular. So it won't be a level 3 based software according to them. But one complete system that you can locked down particular features and release them.

"The software module, the stack, is modular, so I do not invent the wheel new like others do. They have a Level 3 system that is completely independent of the 4 system. So Level 4 will be also ready in 2021"

 

[lunitiks]

 

[electronblue]

Thank you keeping us updated on BMW @Bladerskb. Interesting.

I take it that BMW especially refers to Audi’s splintered Level 3/4 roll-out plan: Traffic-jam Pilot (Level 3), Highway Pilot (Level 4) and AI Zone automated parking (Level 4) — and BMW plans to differ from that.

BWM’s approach to roll-out sounds more generic as a technology platform though they also seem to plan on limiting it technically into areas and subfunctions. Makes sense with their even closer collaboration with MobilEye.

Exciting stuff.

 

[Bladerskb]

@electronblue

Not just that but Mercedes have a traffic jam pilot aimed for 2020 which is also a separate system.

“We’re making autonomous driving safe.” An interview about progress in development work | Daimler

Not all Level 3/ Level 4 systems are the same.

Based on what i can gather;

Audi Level 3 - Limited to Max 37 mph or Less, No lane change.
Mercedes Level 3 - Traffic Jam (No details but probably limited to 37 MPH and no lane change either)

Audi Level 4 - Limited to 37.3 mph to 80.8 mph, Makes Lane Change but might not handle highway interchanges (at-least I haven't seen interchanges demoed and the specs and limitation to 37mph gives the indication that it wont...)

But yeah things like working in adverse weather, handling all highway interchanges, making lane changes in dense low speed traffic, etc, differentiates one self driving system from another.

I totally believe if Mobileye has their way in implementing their driving policy unfeathered, then the BMW's system will be able to handle everything on the highway.

 

[YYZ-IAD]

The most interesting thing that will come out of Waymo One are the vlogs by riders and journalists, since (I believe) Waymo One users will not be under a non-disclosure agreement. I hope there are a lot of vlogs.

Just saw that some Arizona media folks followed some Waymos for 170 miles and captured some clips:

I was in Phoenix area - so purposely went to Chandler to find and follow a Waymo One vehicle. Success!

Observations:
1) Moved at posted speed quickly - not the conservative luddite that has been reported by some. Moves well with traffic.
2) Safety impressive based on a few cases a) It was first car in line making left at T intersection with traffic light. Stopped at red. When light turned green, it hesitated. Two cars coming from the right blew through red light. It finally started when intersection was clear. b) there was a raised pedestrian crosswalk on quiet street. It noticeably slowed down in anticipating of the "speed bump" created by raised crosswalk

Overall, very impressive. Clearly massive testing efforts have paid off. Geofencing to Chandler area adds to safety - since detailed HD maps of area are part of the "sensor" package.

 

[strangecosmos]

Some positive anecdotes from a Waymo One rider:

Naysayer-Debunking Highlights From The Recent Waymo One User AMA : SelfDrivingCars

 

[diplomat33]

Some positive anecdotes from a Waymo One rider:

Naysayer-Debunking Highlights From The Recent Waymo One User AMA : SelfDrivingCars

Thanks for sharing. I don't dismiss what Waymo has done. It is a great accomplishment. It sounds like Waymo has created a self-driving system that can navigate specific city roads, handling almost all the driving, very effectively. And considering that the driving is on complex city streets, with busy intersections, lots of traffic, pedestrians, cyclists etc..., what Waymo has accomplished is excellent.

But I did notice this quote:
"If I had to put a number on it, I would say they disengage the auto drive mode once in every five rides or so, and even then it is only for a few seconds before they put it back into auto drive mode."

1 disengagement every 5 rides actually seems like a lot to me, especially considering that each ride is probably not very long. It's not like each ride is a 200 mile trip. Each ride is maybe 20 miles or so, driving around town. But maybe, since the disengagements were just a couple seconds and were apparently not safety related, they were not disengagements that were the fault of the system?

 

[electronblue]

@diplomat33 The operative word being ”they disengage” here. So the safety driver chooses to disengage as opposed to the system freaking out for example. Many of them could well be ”flow of traffic” type of scenarios.

 

[diplomat33]

@diplomat33 The operative word being ”they disengage” here. So the safety driver chooses to disengage as opposed to the system freaking out for example. Many of them could well be ”flow of traffic” type of scenarios.

Yeah, I realize that the disengagements don't necessarily mean that the system can't handle whatever the situation was.

 

[strangecosmos]

1 disengagement every 5 rides actually seems like a lot to me, especially considering that each ride is probably not very long. It's not like each ride is a 200 mile trip. Each ride is maybe 20 miles or so, driving around town. But maybe, since the disengagements were just a couple seconds and were apparently not safety related, they were not disengagements that were the fault of the system?

I think a safety-related disengagement means that, if the safety driver didn’t disengage, there would be a risk of a collision. Most disengagements are probably not that serious. But I think all Waymo One’s disengagements are the fault of the system. If the system were performing perfectly, why would the safety driver disengage?

Here’s an example of a disengagement as recounted by a Waymo rider:

“Richardson said the incident he remembers most vividly occurred when Waymo was traveling behind a bus. Just after an intersection, the road temporarily widened to give buses room to stop at a bus stop without blocking the flow of traffic. The bus in front of Richardson's Waymo pulled off to the right at the bus stop, leaving the road in front of the Waymo vehicle clear. But the Waymo stopped—perhaps expecting the bus to pull back out in front of the Waymo vehicle at any moment. After about three seconds, Richardson says, the safety driver took over control and steered the Waymo past the bus.”
​

Source: We finally talked to an actual Waymo passenger—here’s what he told us

A disengagement once every 5 trips is way too high for a fully autonomous robotaxi. But this makes me think about Tesla’s development of a human-supervised system. If Tesla drivers can maintain the same vigilance as Waymo safety drivers, then a disengagement once every 5 trips wouldn’t be a problem. The software would offer Tesla owners some amount of convenience, novelty, and coolness, and possibly added safety. (Human + machine could potentially be safer than either human or machine alone.)

The bar for a fully autonomous robotaxi might be a disengagement once every 50,000 miles — how often a car breaks down on average — or 500,000 miles — how often a car crashes on average. But the bar for a supervised system with a safety driver is much lower. If it needs to be disengaged every 100 miles, that’s okay.

A big question is whether this is still workable as the system gets better. A higher disengagement rate is actually better in a sense — it keeps the driver on their toes. If the disengagement rate drops to once per 6,000 miles, that could be dangerous. If your car only needs to disengaged every 6 months on average, how are you supposed to keep paying attention?

It could be safe if the failure modes are harmless. Like if the failure mode is just the car sitting at a green light, that’s annoying, not dangerous. If the failure modes aren’t on the highway, but are in residential or urban areas with a 30 mph / 50 km/h speed limit, that would also limit the potential for danger.

Launching a human-supervised system, similar to Waymo’s, at the scale of 1 million+ vehicles would create some interesting possibilities. Let’s say each Tesla owner spends an average of 45 minutes driving per day (roughly in line with the American average). With a fleet of 1 million Teslas, that’s 45 million minutes per day, or 31,000 years of driving per year. This is the kind of scale you need to do reinforcement learning (RL). I believe the RL training phase of AlphaStar (which followed the imitation learning phase) generated 60,000 years of combined experience across all the different agents.

The reward for Teslas could be based on miles between disengagement, or trips between disengagement, or something like that. The training signal would come from humans taking over control.

 

[strangecosmos]

For Tesla, there could be three distinct training phases:

1. Imitation learning​

2. Simulated reinforcement learning​

3. Real world reinforcement learning
​

Or Tesla could skip Phase 2 entirely. Who knows— might be too hard to bridge the sim2real gap.

I've been thinking that Tesla would use IL to boostrap to RL in simulation. But possible it will bootstrap directly to RL in the real world.

We know from Drago Anguelov's recent talk that Waymo uses a combo of IL and hand coding. Perhaps Tesla is doing the same. (I don't know; I'm just guessing.) If you can build a "scaffolding" that gets human-supervised Full Self-Driving into 1 million+ cars, then maybe you can use that scaffolding to kick off real world reinforcement learning at massive, simulation-level scale.