Waymo

[diplomat33]

2. understand = reactive at best, not anticipate

I don't know if I agree. Understanding is the first step. Once you have a better understanding of how things work then you can better anticipate. If you read the next part of the tweet, it says to help the Waymo Driver better predict the actions of others. IMO, anticipation is part of prediction.

Specifically, Waymo wants to better understand how NY traffic "works", ie how do cars, pedestrians, cyclists etc interact with each other and with the road. Then, they can use that data to train their prediction NN. The whole point is for the Waymo Driver to be a better driver because it does a better job of predicting the actions of others, especially in a dense urban setting when there are a lot of road users. If the Waymo Driver is better at prediction, it could anticipate that a pedestrian will jay walk, that a cyclist will dash between two cars, or that a car will cut across multiple lanes to get into a parking spot etc...

 

[voyager]

I skipped a phase. Understand? Whose understanding? Machine learning?
Will it contribute to the level of situational awareness needed for anticipating?

 

[diplomat33]

I skipped a phase. Understand? Whose understanding? Machine learning?
Will it contribute to the level of awareness needed for anticipating?

Waymo wants to better understand traffic and road users in NYC in order to improve perception and prediction of actions of other road users in dense urban environments:

"Our focus in NYC was around understanding traffic and road features, which will help us improve our Drivers ability to perceive and predict the actions of other road users in dense urban areas"

If you can better predict the action of other road users, yes, I think that leads to better anticipation.

 

[voyager]

If the Waymo Driver is better at prediction, it could anticipate that a pedestrian will jay walk, that a cyclist will dash between two cars, or that a car will cut across multiple lanes to get into a parking spot etc...

Way too many variables why a 'machine' will never learn to cope with all situations like a human driver can (not under the influence of any substance, fatigue). Let's differentiate the pedestrian for instance. How to 'assess' when he is about to jaywalk, indicate that the next car is his cab ride, greeting someone he knows on the other side of the road, bullying an AV into phantom braking?

 

[diplomat33]

Way too many variables why a 'machine' will never learn to cope with all situations like a human driver can (not under the influence of any substance, fatigue).

Maybe not all situations perfectly, all the time, but we can get pretty close. The goal is for AVs to have the best prediction of as many variables as possible. The more data, the better the training, the more capable the prediction can be and the more variables it can handle. And computers are able to process a lot of variables quickly.

How to 'assess' when he is about to jaywalk, indicate that the next car is his cab ride, greeting someone he knows on the other side of the road, bullying an AV into phantom braking?

That is precisely what Waymo is working on. Waymo talked about this very thing in their blog last month.

Utilizing key point and pose estimation for the task of autonomous driving

Historically, computer vision relies on rigid bounding boxes to locate and classify objects within a scene; however, one of the limiting factors in detection, tracking, and action recognition of vulnerable road users, such as pedestrians and cyclists, is the lack of precise human pose...

blog.waymo.com

 

[voyager]

So, when Waymo (or any developer) says it is working on this,
you think that they are already underway solving this?.. I don't.
You and I will never agree. Let's leave it at that.

 

[diplomat33]

So, when Waymo (or any developer) says it is working on this,
you think that they are already underway solving this?.. I don't.
You and I will never agree. Let's leave it at that.

Yes I do because the blog literally gives examples that Waymo is underway solving this. How someone can see proof and not believe the proof, is beyond me. So yes, they are "underway solving this". Of course, it is not 100% solved yet but it is underway.

 

[ThomasD]

The densest part of NYC is The Upper East side where the population is 168,000 per square mile. So WAYMO only plans to operate in a 2 square mile area

 

[diplomat33]

The densest part of NYC is The Upper East side where the population is 168,000 per square mile. So WAYMO only plans to operate in a 2 square mile area

No, I don't think that is correct. From Google maps, the area is about 7 miles long and 1 mile wide. So about 7 sq mi. That's an approximation. But it is definitely bigger than 2 sq mi. And the area stretches down to Wall Street.

Here is Waymo's testing area in NYC:

 

[EVNow]

Yes I do because the blog literally gives examples that Waymo is underway solving this. How someone can see proof and not believe the proof, is beyond me. So yes, they are "underway solving this". Of course, it is not 100% solved yet but it is underway.

So, when Waymo (or any developer) says it is working on this,
you think that they are already underway solving this?.. I don't.
You and I will never agree. Let's leave it at that.

We’ll never know how much they have solved, though. 10% or 90%.

I have mentioned this before, for all companies, it’s difficult to know whether what they are showing is something new they came up with last week or something mature. We can carefully listen and parse the words, but it’s going to be an in exact science.

 

[diplomat33]

We’ll never know how much they have solved, though. 10% or 90%.

I have mentioned this before, for all companies, it’s difficult to know whether what they are showing is something new they came up with last week or something mature. We can carefully listen and parse the words, but it’s going to be an in exact science.

Sure, we don't know how much is solved yet but it is silly to act like maybe it is just some idea that they have not even started on yet. 'The blog tells us:

While we incorporate this exciting technology onboard our vehicles to help the Waymo Driver navigate the real world...

So "key points" is not something brand new they just came up last week. It is already being incorporated into the cars now.

The Waymo Driver uses real-time data from our sensor suite, including our lidars, which feed into our neural-network models to localize key points in three dimensional space.

We know how it works. The Waymo Driver feeds real-time data from the sensor suite into NN models to localize key points.

Additionally, we’ve optimized our system to run onboard the vehicle in real-time, with high precision and low latency, to enhance its behavior-prediction models and allow the Waymo Driver to quickly and safely handle any situation.

It takes time to optimize a system like that. This is another clue that Waymo has been working on it for awhile now. It is obviously not something they just came up with last week if they have optimized it to run onboard the vehicles in real-time.

Here are a handful of examples of how key points are helping the Waymo Driver navigate San Francisco.

Waymo then provides some clips of key points "in action". So we have some examples of what it is doing now in cars.

While the Waymo Driver can detect various gestures from raw camera data or lidar point clouds, like a cyclist or traffic controller’s hand signals, it is advantageous for the Waymo Driver to use key points to determine ...

This tells us that the Waymo Driver can already detect various gestures now with just raw camera or lidar data but that key points improve performance.

Bottom line: if we take the blog at face value, which we have no reason not to, I think we can say that key points are working in cars now and are fairly advanced already. To act like we have no idea if maybe key points are just some new idea they came up with last week and have not even started on, is silly IMO.

 

[EVNow]

@diplomat33 I’m sure it’s being used now …

For some time now, key points have been providing the Waymo Driver a more nuanced understanding of the world around it, creating a more predictable and comfortable driving experience for all road users, including our Waymo One riders.

But on some items, it’s not clear what the status is.

While the Waymo Driver can detect various gestures from raw camera data or lidar point clouds, like a cyclist or traffic controller’s hand signals, it is advantageous for the Waymo Driver to use key points to determine a person's orientation, gesture, and hand signals.

 

[diplomat33]

But on some items, it’s not clear what the status is.

I think that sentence is explaining why Waymo is using key points. It is simply saying that Waymo can do gesture detection without key points but that key points enhance gesture detection.

The other quote you have, makes it clear key points have been in the cars for awhile ("for some time now"): .

For some time now, key points have been providing the Waymo Driver a more nuanced understanding of the world around it, creating a more predictable and comfortable driving experience for all road users, including our Waymo One riders.

The quote also explains how key points improve the Waymo Driver ("creating a more predictable and comfortable driving experience").

 

[diplomat33]

Good clip that explains how the Waymo Driver handles a case of a cyclist cutting in.

 

[ThomasD]

Waymo has been testing in some cities for 3+ years. They still aren't transporting passengers in those cities. What is the issue in those cities?

 

[diplomat33]

Waymo has been testing in some cities for 3+ years. They still aren't transporting passengers in those cities. What is the issue in those cities?

The simple answer is that solving autonomous driving is incredibly difficult. To transport passengers safely without any driver in the car requires the autonomous driving to be 99.99999% reliable. There are a lot of cases that you need to train the car to handle. And you cannot deploy until you are absolutely sure that the car can drive completely safely on its own. Waymo is good but not quite there yet.

Frankly, it is a bit silly to ask "why isn't Tesla there yet?" or "Why isn't Waymo there yet?" The fact is that nobody is there yet. Nobody has solved FSD yet. It is a very difficult problem to solve. Computers are not intelligent like humans. Driving is not easy for computers. Sure, you can get the computer to do the basics like stay in the lane, don't hit stuff, stop at red lights, make turns. But driving is more complicated than that. There can be all kinds of situations that require decisions like when to yield to another car or figuring out if a pedestrian really intends to cross the street or is waiting for someone. And there are cases like traffic lights going out where you have to navigate without traffic lights or construction that requires you to break the normal rules of driving, like driving in the wrong lane but only when the worker holds up a make shift "go" sign. There are millions of edge cases like that, that your AV has to be able to handle safely without human intervention. On top of that, rain, snow, fog and other conditions can make it harder to see things.

 

[EVNow]

Waymo has been testing in some cities for 3+ years. They still aren't transporting passengers in those cities. What is the issue in those cities?

My guess is they are just trying to get data from those cities and / or their disengagement rates in those cities are not good enough.

Ofcourse starting the service say in Kirkland area near Seattle, where Google has a large office - is also a business decision.

 

[Doggydogworld]

Waymo has been testing in some cities for 3+ years. They still aren't transporting passengers in those cities. What is the issue in those cities?

They don't have a business model that works in those cities. San Francisco has enough customers to make it work, if Waymo can get out of their own way. Easier said than done.

They also need to improve their tech, but that's a secondary issue. With a working business model they could ramp and get 100x more feedback to help improve the tech. That was the original plan, but they couldn't attract anywhere near enough customers in their first market to have any hope of reaching critical mass.

 

[diplomat33]

They don't have a business model that works in those cities. San Francisco has enough customers to make it work, if Waymo can get out of their own way. Easier said than done.

They also need to improve their tech, but that's a secondary issue. With a working business model they could ramp and get 100x more feedback to help improve the tech. That was the original plan, but they couldn't attract anywhere near enough customers in their first market to have any hope of reaching critical mass.

Yes, the business model requires customers, which requires dense urban driving, which requires they "solve dense urban driving". So tech is the primary issue. If they solve "dense urban driving", they can deploy where lots of customers are, and their business model will work. So it is not a business model issue, it is a tech issue. Solve the tech issue, solve the business model.

 

[EVNow]

Yes, the business model requires customers, which requires dense urban driving, which requires they "solve dense urban driving". So tech is the primary issue. If they solve "dense urban driving", they can deploy where lots of customers are, and their business model will work. So it is not a business model issue, it is a tech issue. Solve the tech issue, solve the business model.

Isn’t it easier to change the business model to adopt to the current tech ?

Reminds me of all the people who say, we should address climate change, but only when the tech is “ready”.