Waymo

[Bitdepth]

Was having a discussion in another thread and a scenario came up about a traffic intersection being controlled by a traffic cop. How would Waymo handle an intersection where the lights are out and a cop is waving people?

In the same clip you can see Waymo react to 2 different traffic control personnel, and I don't believe remote assistance is involved in the process as stated by @Daniel in SD.

Are they able to recognize a School bus stopped on a multi lane highway ? Where the School Bus is in the far right lane and the Waymo is in the left lane

Waymo does yield traffic on multi lane roads, you can see it yield for an emergency vehicle in the oncoming lane so i assume they would do the same for school buses.

@JJRicks videos are a good archive of many things he encountered when riding in a waymo.

JJRicks.com - Videos-Obsolete

All videos listed on this website are licensed under a Creative Commons Attribution 4.0 International License.

www.jjricks.com

 

[Daniel in SD]

In the same clip you can see Waymo react to 2 different traffic control personnel, and I don't believe remote assistance is involved in the process as stated by @Daniel in SD.

No way to know and I'm sure the system can handle it smoothly nearly 100% of the time. I think I remember the Cruise CTO saying that they use remote assistance every 10 miles or so on average (of course that was a while back). Waymo might be different but if I were using remote assistance that frequently this is one of the cases that I would monitor.
I'm not trying to denigrate these systems, there's nothing wrong with using remote assistance as you're still getting a huge reduction in labor costs.

 

[Bitdepth]

No way to know and I'm sure the system can handle it smoothly nearly 100% of the time.

Remote assist is used when the car cannot respond to a given situation and needs further assistance. The capability is built into the system to read and respond to hand gestures from cyclists going far back as 2016 and traffic control personnel.

Utilizing key point and pose estimation for the task of autonomous driving

Historically, computer vision relies on rigid bounding boxes to locate and classify objects within a scene; however, one of the limiting factors in detection, tracking, and action recognition of vulnerable road users, such as pedestrians and cyclists, is the lack of precise human pose...

blog.waymo.com

While the Waymo Driver can detect various gestures from raw camera data or lidar point clouds, like a cyclist or traffic controller’s hand signals, it is advantageous for the Waymo Driver to use key points to determine a person's orientation, gesture, and hand signals. Earlier and more accurate detection allows the Waymo Driver to better plan its move, creating a more natural driving experience.

Moreover, if remote assist is involved, it shows on the rider screen. As we have seen in JJricks videos, and they always call in to keep the rider informed on what is happening.

I think I remember the Cruise CTO saying that they use remote assistance every 10 miles or so on average (of course that was a while back). Waymo might be different but if I were using remote assistance that frequently this is one of the cases that I would monitor.

I remember that also when Cruise CTO said it occurs between 5 - 10 miles but they are two different companies in different stages of development. What they use remote assistance for may vary based on geography as well as how sufficiently advanced their software is at tackling these issues on their own. We have seen Waymo handle cones many times and need assistance 1 time in the JJRicks videos. What I noticed is that when remote assist was needed, they were talking to JJRicks and there was notification on the rider screen to let him know.

I'm not trying to denigrate these systems, there's nothing wrong with using remote assistance as you're still getting a huge reduction in labor costs.

Remote assist will be and is a necessity with an autonomous taxi service. You cannot guarantee your vehicle will operate perfectly and you definitely need a way to remotely fix any arising issues. It is clear that in JJricks videos, the few times that the car was reacting to traffic control personel, remote assist was not involved to approve the decision.

 

[ThomasD]

Having hundreds of remote operators for the thousands of robotaxis operating all over a State or even keeping track of cars that travel from one state to another sounds expensive. Waymo can read and interpret signs? '

 

[Daniel in SD]

Having hundreds of remote operators for the thousands of robotaxis operating all over a State or even keeping track of cars that travel from one state to another sounds expensive. Waymo can read and interpret signs? '
View attachment 845949
View attachment 845950

The previous state of the art was having a driver in every car even when the vehicle is waiting for a fare. It’s a huge efficiency improvement over that. And as the technology advances you can reduce the scenarios where remote assistance is used.

 

[diplomat33]

Having hundreds of remote operators for the thousands of robotaxis operating all over a State or even keeping track of cars that travel from one state to another sounds expensive.

Waymo is not going to have hundreds of remote operators and tracking every car. That would not be scalable. They only use remote assistance now because their autonomous driving might encounter a situation where it needs guidance. But the goal is to eventually remove all remote assistance completely. As Waymo improves their FSD, they will need remote assistance less and less until eventually, they won't need remote assistance at all.

It is also important to note that remote assistance never controls the car and cannot make split second interventions to prevent a crash. So the car still has to handle all safety critical decisions on its own or reach what is called a minimum risk condition. Remote assistance is only for non-safety issues. So when we see videos of driverless Waymo rides, Waymo trusts that the car can drive safely. Waymo only has remote assistance "just in case" the car is unsure about something. But even when the car is unsure, it will still drive safely.

There is a natural progression as the FSD gets better. Put simply:

• First, you use safety drivers in the car when your FSD is not safe enough.
• Then, when your FSD is safe enough, you can remove the safety driver. But you use remote assistance "just in case" it encounters a non-safety issue.
• Then eventually, your FSD does need remote assistance at all.

Waymo can read and interpret signs? '

I believe so.

 

[ThomasD]

How far away are these companies from operating with zero assistance on any road surface in any weather? Will people with no cell service be able to use a landline to call for a self driving Taxi?

 

[Daniel in SD]

How far away are these companies from operating with zero assistance on any road surface in any weather? Will people with no cell service be able to use a landline to call for a self driving Taxi?

Nobody can answer that. It would be easier to estimate when someone will walk on Mars, a much easier engineering problem.
By the time L5 happens landlines will no longer exist. I’m willing to bet on that. Haha.

 

[diplomat33]

How far away are these companies from operating with zero assistance on any road surface in any weather?

That is going to take longer. We still got work to do to get autonomous driving to operate on urban and highways with zero assistance. And robotaxi companies may never do rural areas because there is not enough money to be made compared to cities. Once reliable autonomous driving moves to personal cars, then it will eventually work everywhere, even rural, unpaved roads.

Will people with no cell service be able to use a landline to call for a self driving Taxi?

Not right now. You use an app on your smart phone to request a robotaxi ride so you can't use a landline.

 

[Doggydogworld]

Having hundreds of remote operators for the thousands of robotaxis operating all over a State or even keeping track of cars that travel from one state to another sounds expensive.

Hundreds of monitors for thousands of robotaxis, roughly a 1:10 ratio, is fine for scaling. You've eliminated 90% of the driver cost and you'll have a clear path to whittling the other 10% down to 2% or whatever.

Of course the Tesla Robotaxis operating on every road on earth since April 2020 need zero remote monitoring. That's tough competition!

 

[diplomat33]

Starting tomorrow, Waymo is starting rider-only (driverless) testing with their trusted testers, in downtown Phoenix. Before, it was just employee only. This is one step closer to public driverless.

https://twitter.com/x/status/1564284920700121088

 

[Bitdepth]

@JJRicks we need you to come back and start testing again.

 

[ThomasD]

Trusted tester = Keep your mouth shut even if the car messes up

 

[diplomat33]

Trusted tester = Keep your mouth shut even if the car messes up

Very funny. LOL.

"Trusted tester" is just the name Waymo gives to their Early Access program. Waymo invites some people to ride a driverless Waymo to get feedback before it goes full public.

 

[diplomat33]

@JJRicks we need you to come back and start testing again.

Looks like he will.

https://twitter.com/x/status/1564292210673143809

 

[ThomasD]

So can the trusted testers film their adventures

 

[ThomasD]

Maybe the end of private car ownership is the future

 

[Doggydogworld]

So can the trusted testers film their adventures

They're under NDA. But since they're "Trusted" Testers I'm sure Waymo allows them to film as long as they pinkie pledge not to post it on YouTube

 

[diplomat33]

So can the trusted testers film their adventures

No. But like I said, this is early access testing. It is common for early access to be under some sort of NDA.

"Trusted Tester" is just the name of their early access program. They are called "trusted tester" because Waymo is letting them ride in their robotaxi service before the general public in exchange for feedback.

 

[Bitdepth]

https://twitter.com/warren_craddock

@diplomat33 has shared some posts from Warren Craddock and it is a really good place to start when understanding ADS. Most of us aren't experts in the field and is good to have a basic understanding so we can communicate effectively. There are some things I had the wrong understanding on that is cleared up by some of his post so I will share some especially because I recently had an unpleasant discussion about perception, sensor fusion and how it is not the most difficult part of solving autonomous driving.

AV Myth #7: "If two sensors disagree, there's no way to figure out which one to believe." Fact: The disagreements are the *entire point* of using multiple types of sensors. If they never disagreed, then they'd be 100% redundant, and would offer no additional value. 1/n

Radar does well in fog, because fog is essentially transparent at radio wavelengths. Cameras do well in daylight, because the sun is bright. Lidar does well at night, because it produces its own light. Each sensor sees the world differently, and they disagree all the time. 2/n

The disagreements between sensors are intentional and desirable. The disagreements mean the AV gets much richer data than it would from any individual sensor. The sensors are much stronger together than they are individually. 3/n

Sensor fusion is a mature field. Medical imaging is a prominent example, where PET and CT scanners are bundled together. The PET system detects positrons, but can't see the body itself. The CT system sees the body, but can't detect positrons. 4/n https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1987321/

Sensor fusion is never about trusting one sensor over another. Instead, data from multiple sensors is combined and fed into a single neural net, which learns to make the best use of the salient features provided by all the sensors. 5/n https://arxiv.org/pdf/2005.09202.pdf

An aside: If you believe neural nets are sufficient to drive cars more safely than humans, then you must also believe that neural nets are able to competently combine inputs from multiple sensors -- a vastly simpler task. 6/n

Some of the most exciting work in the AI research community today is explicitly multi-modal, e.g. DeepMind's Gato. An AI system can be much stronger when it is given text, images, sound, etc. all associated with the same event. 7/n https://www.deepmind.com/publications/a-generalist-agent

This is of course also true for humans: we learn more effectively when we're taught something in a way that uses all our senses. Even kindergarteners understand that. 8/n

In summary, we use multiple sensing modalities (cameras, radars, lidars) precisely *because* they sometimes disagree. Together, they provide a fuller and more complete picture of the world than they do individually. And we put it all into the same neural nets, anyway! 9/9