Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

Waymo

This site may earn commission on affiliate links.
They are not recording landmarks

Yes they are. Landmarks are simply defined as LIDAR map points that are consistent over multiple mapping passes. In a dynamic environment where you have parked cars, people, obstructions, you have to use LIDAR to map a particular location multiple times. Then you can use NNs or heuristic code to process each LIDAR map to remove points that are inconsistent through multiple passes. This is basically the localization point map.

When I say landmark, I don't mean they choose a building or house or street sign as a "landmark." Perhaps I'm using the word incorrectly, but you get my drift.
 
Yes they are. Landmarks are simply defined as LIDAR map points that are consistent over multiple mapping passes. In a dynamic environment where you have parked cars, people, obstructions, you have to use LIDAR to map a particular location multiple times. Then you can use NNs or heuristic code to process each LIDAR map to remove points that are inconsistent through multiple passes. This is basically the localization point map.

When I say landmark, I don't mean they choose a building or house or street sign as a "landmark." Perhaps I'm using the word incorrectly, but you get my drift.
You said;
5) There can be a lot of "noise" in the trigulation process because the exact positions of the landmarks can change over time or there can be obstructions. For example, trees grow and block certain landmarks, or there can be construction on houses / buildings that are used as landmarks.
You are implying they choose certain buildings or houses to use as landmarks. And like I said from every literature including patents and information I have seen they do not choose certain building or houses as landmarks. They create ground truth localization data by scanning an environment and recording the X,Y Z, Orientation, Pitch, Yaw and location of each point. Of course, they filter out dynamic objects from the scan and over multiple passes validate that as ground truth. They localize by comparing points up to a certain distance from the car by comparing the change between the ground truth and subsequent lidar then apply the new transformation of the points to localize the car. A mismatch is what they used to localize not a precise point to point matching.
All these factors can be made a lot worse with rain droplets on the LIDAR sensor and/or streets / reflections. As you increase the noise from the LIDAR sensor input from reflections / cars / (anything blocking view of landmarks), its localization ability is worse, leading to these jitters.
How localization affects steering wheel jitter from a technical level is information i have not seen discussed anywhere. Perhaps it would do us all well to share such information less refrain from making statements that have no technical backing but pure conjecture or our part.
In the videos we've seen, the steering wheel has a very characteristic frequency oscillation. I think it's related to the LIDAR localization sampling frequency.
What frequency does the lidar sample at and how does that affect steering wheel jitter? If it is a result of lidar localization, shouldn't it be present the entire drive?
Edit: BTW, when you see or hear about Elon or Karpathy making fun of Waymo and how it's on "rails," this is what they're laughing about.
An ADS that navigates on the road amongst other road agents is by industry definition not on rails. It is not following a rail line and using a map to localize is not a rail. The funny thing is the company that does not have an autonomous driving system laughing at other companies that actually have autonomous driving systems. Karpathy has abandoned Tesla.
 
Last edited:
The LIDAR HD map localization layer is the figurative rail.
Figurative rail, a figurative crutch, a figurative fool's errand and figuratively doomed to fail.

A HD map is a prior knowledge. Localizing using prior knowledge as well as Lidar sensor or camera sensor is just SLAM. Not rails, that is why you ought to venture out Elon Musk's talking points.
Sure, within that HD map, the car can move freely within the pre-defined drivable space. But once again, you get my drift.
I don't get your drift, there is no room to drift within a rail (pun intended). You are either on rails like the bots in Tesla factories or you are not on rails and are free to move and engage dynamically with every road agent on an actual public road without human supervision. Every autonomous car has to create a map and determine what is a drivable and non-drivable space, what lanes to take etc. Having prior knowledge of those things makes sense.
 
So essentially Waymo builds a virtual reality world based on HD scans and that is where it drives itself. While it does that, it also uses real time scan to account for any differences and adjust. In other words, it plans the trip using HD maps but as it moves along, it adjusts based on real time scans. Did I get that right?
 
So essentially Waymo builds a virtual reality world based on HD scans and that is where it drives itself. While it does that, it also uses real time scan to account for any differences and adjust. In other words, it plans the trip using HD maps but as it moves along, it adjusts based on real time scans. Did I get that right?

Pretty close.

I would not call HD maps a virtual reality world. They are simpler than that. HD maps are a 2D representation of the static driving area. HD maps can also include information like speed limits, one way streets, no parking zones between certain hours etc.. Yes, the Waymo uses real-time scans with cameras, lidar and radar to detect any differences and adjust.

The real-time scans with cameras, lidar and radar also detect moving objects like other vehicles, pedestrians, cyclists etc... The Waymo takes information from the HD maps and the real-time scans to build a complete perception of everything around it, both the static and the dynamic. The Waymo then makes predictions about what other objects will do, like other vehicles, pedestrians etc...

Finally, the Waymo takes the map, the real-time scans and the predictions and makes a decision about what path to drive in order to drive safely. The car then tells the steering wheel and pedals what to do to execute the path.
 
  • Like
Reactions: enemji
If you looked at all companies as critically as you look at Tesla, truth will be clear.

I'm actually just as critical, you just don't see it. I have said for some time now (for several years) that I don't believe a traditional automaker would be able to solve L4 driving, EVER. Not now, not in 100 years. They just don't have the technical capability and when they try to hire engineers their antiquated management squander it. I got a-lot of slack for this on reddit. But its true and continue to be true for several years. Notice that all the people who are making progress in AV are tech companies/startup. And if a traditional automaker is making progress its due to acquiring another company (GM/Cruise, VW-Ford/Argo AI, etc).

I think you should look in the mirror. Have you ever tried to just remove Tesla out of the equation. Don't make it about Tesla.
Don't make it about tesla versus whoever or feel like you are defending Tesla.

IF you white-label your AV analysis rather than having a conclusion and walking backwards. You would find out that its others that are being equally critical and its you that isn't being critical at all on Tesla.

Lets look at the below logic.
It makes no sense if you believe this logic is critical thinking. Yet this is what a-lot of Tesla fans exhibits.

Tesla using a 2011 and 2014 ACC forward radar.

Tesla fans: Tesla Leapfrogs Self-Driving Competitors With Radar That's Better Than Lidar

Tesla removes their old radar

Tesla fans: Tesla removed radar because Radar is DOOMED. IF you have radar on your car, it makes the car more DANGEROUS! Sensor fusion sucks. If radar and vision disagrees what do you do?


Tesla adds back a better radar

Tesla fans: Sensor fusion of vision and radar will make Tesla FSD 10-100x better than a human driver.

Again you think the dance above is critical thinking?

  • Notice how not a single tesla fan pointed out that other SDC companies use state of the art radars that are over a thousand times better than Tesla's radar. That Tesla is using an ancient old 2011 and 2014 radar that was only meant for ACC on highways and not for the ODD of autonomous driving.


  • That other SDC companies have 360 radar coverage and can therefore handle all driving scenario, while Tesla only covered forward and hence doesn't cover the majority of driving scenario. For example: Chuck's left turn or any turn or lateral movement for that matter.

  • SOTA Ultra High resolution imaging 4D Radars Waymo and Mobileye are using gives you 500k points per second, the radar that Tesla used gives you 400 pps. Not even going into other benefits like way better contrast, etc.

  • With Tesla's old radar, they can't feed it to a NN to do object classification. With SOTA radar that SDC companies are using, the resolution is so high that they can actually train a NN with it to do object classification.

  • Teslas fans all went "sensor fusion is bad, sucks, doesn't work, no one knows how to do it". Not a single one of them went. Hmm. Tesla is trying to fuse 1 million pixels with 40 scattered radar points from their old obsolete ACC radar. Maybe that's the problem. Rather than fusing 1 million camera pixels with 1 million radar points from SOTA 4D imaging radars. Not a single one ever stopped and considered why it was hard for them. They all said. "Lidar = Doomed. Radar = Doomed. Sensor fusion = doesn't work"

We are living in a world where we have to fight for reality because people have let their personal allegiance take over their critical thinking faculty.

 
Pretty close.

I would not call HD maps a virtual reality world. They are simpler than that. HD maps are a 2D representation of the static driving area. HD maps can also include information like speed limits, one way streets, no parking zones between certain hours etc.. Yes, the Waymo uses real-time scans with cameras, lidar and radar to detect any differences and adjust.

The real-time scans with cameras, lidar and radar also detect moving objects like other vehicles, pedestrians, cyclists etc... The Waymo takes information from the HD maps and the real-time scans to build a complete perception of everything around it, both the static and the dynamic. The Waymo then makes predictions about what other objects will do, like other vehicles, pedestrians etc...

Finally, the Waymo takes the map, the real-time scans and the predictions and makes a decision about what path to drive in order to drive safely. The car then tells the steering wheel and pedals what to do to execute the path.
Thanks. So in a real driving situation, it plans its path using the HD maps but real time information ALWAYS overrides the planned information.
 
So essentially Waymo builds a virtual reality world based on HD scans and that is where it drives itself. While it does that, it also uses real time scan to account for any differences and adjust. In other words, it plans the trip using HD maps but as it moves along, it adjusts based on real time scans. Did I get that right?

No, that's not right.

Waymo drives around the same streets / routes multiple times, builds HD map (basically like train tracks with multiple forks).

Afterwards, once the train tracks are laid, Waymo deploys its cars in "autonomous" mode. The auto-cars then uses the real-time LIDAR information to localize itself with the previously built HD maps. These HD maps are manually / automatically labeled with all sorts of information: lane directions, stop signs, traffic lights, stop lines, speeds, etc. (everything).

Once the auto-car localizes itself, it drives around based on the previously built HD map. It can only drive on the drivable space previously defined.

When a destination is entered for the auto-car, it builds a route based on the train tracks already laid out by the prior HD map. The auto-car follows this route, and on the way, it may encounter obstacles or other cars in the way. The routed path can adjust for these obstacles within the previously defined drivable space.

Most of this is speculation since Waymo doesn't spell it out explicitly. But we get it (it's not magic).
 
Thanks. So in a real driving situation, it plans its path using the HD maps but real time information ALWAYS overrides the planned information.
It’s not a matter of planning based on HD maps. All data from sensors and maps goes into every aspect of driving.

An overview of the architecture.


9ZgI2ci.png
 
Thanks. So in a real driving situation, it plans its path using the HD maps but real time information ALWAYS overrides the planned information.

Waymo uses both maps and real-time information to plan the path. But yes, real-time information will always override the map information, for example a construction zone.

No, that's not right.

Waymo drives around the same streets / routes multiple times, builds HD map (basically like train tracks with multiple forks).

Afterwards, once the train tracks are laid, Waymo deploys its cars in "autonomous" mode. The auto-cars then uses the real-time LIDAR information to localize itself with the previously built HD maps. These HD maps are manually / automatically labeled with all sorts of information: lane directions, stop signs, traffic lights, stop lines, speeds, etc. (everything).

Once the auto-car localizes itself, it drives around based on the previously built HD map. It can only drive on the drivable space previously defined.

When a destination is entered for the auto-car, it builds a route based on the train tracks already laid out by the prior HD map. The auto-car follows this route, and on the way, it may encounter obstacles or other cars in the way. The routed path can adjust for these obstacles within the previously defined drivable space.

Waymo does not build "rails" or "train tracks". Waymo does not just localize itself with lidar and drive on "train tracks". We know Waymo uses many methods to localize that don't depend on lidar. And yes, Waymo does build a HD map as a reference. But Waymo is not stuck on the HD map like on a "rail". Waymo uses cameras, lidar and radar to detect the static and dynamic and plot a path in real-time. And Waymo can drive without maps. So Waymo is not restricted to "rails" that were mapped before.

Please, please, stop spreading misinformation. You can be a Tesla fan if you want but deliberately spreading misinformation that misleads others is not cool. You would not want others to spread misinformation about Tesla, would you? So please stop spreading misinformation about Waymo.

Most of this is speculation since Waymo doesn't spell it out explicitly. But we get it (it's not magic).

Waymo does spell it out very clearly. There is no need to lie or speculate.

And if you don't know, then ask others who do know or say nothing. Don't spread misinformation.
 
Last edited:
Most of this is speculation since Waymo doesn't spell it out explicitly.

No. Waymo spells it out very clear. At last year's at Scale AI TransformX Conference, around the 5:28 mark, Anguelov describes step by step how Waymo handles an intersection:

1) Use prior map AND sensors (cameras, radar, lidar) to build perception of what is around the car.
2) Make predictions of what other road agents will do.
3) Determine the appropriate path in real-time.
4) The steering wheel and pedals execute the planned path.

We also know that #1, #2 and #3 use ML. So we know very well how Waymo works. We know they don't premap with lidar, localize with lidar and only follow "rails" from previous map. That is an incorrect description of their approach.

I11k6nu.png


Later in the talk, around 11:40, he further describes the Waymo approach. We also describes Waymo's 3D point clouds with vision, their prediction ML and more. So we know. There is no mystery.

 
Last edited:
When did Karpathy say that?

At the 2020 CVPR conference here:


Karpathy shows a Waymo making a turn at an intersection and says that Waymo premaps, localizes to the cm with lidar and is on "rails" so it knows how to make the turn. He completely misrepresented the Waymo approach. But sadly, many Tesla fans assumed it was true since it was coming from Karpathy and repeat the myth to this day.

I don't know if Karpathy genuinely does not know how HD maps work (unlikely) or was just repeating Elon's talking points (likely). Bu either way, I think it was rather unethical to lie about the competition like that at a major conference to make your system look better. He easily could have just left Waymo out of it and talked about Tesla's approach. That would have been fine. It seems he wanted to push Elon's talking point that "HD Maps and lidar bad, Tesla's vision-only good" but he was dishonest in how he did it.
 
Last edited:
But sadly, many Tesla fans assumed it was true since it was coming from Karpathy and repeat the myth to this day.

I don't know if Karpathy genuinely does not know how HD maps work (unlikely) or was just repeating Elon's talking points (likely).

Karpathy knows

Few people here know what's involved in creating HD maps, kinda like how few people understand what the sae levels mean
 
At the 2020 CVPR conference here:


Karpathy shows a Waymo making a turn at an intersection and says that Waymo premaps, localizes to the cm with lidar and is on "rails" so it knows how to make the turn. He completely misrepresented the Waymo approach. But sadly, many Tesla fans assumed it was true since it was coming from Karpathy and repeat the myth to this day.

I don't know if Karpathy genuinely does not know how HD maps work (unlikely) or was just repeating Elon's talking points (likely). Bu either way, I think it was rather unethical to lie about the competition like that at a major conference to make your system look better. He easily could have just left Waymo out of it and talked about Tesla's approach. That would have been fine. It seems he wanted to push Elon's talking point that "HD Maps and lidar bad, Tesla's vision-only good" but he was dishonest in how he did it.
Thanks. I guess I should have asked "when did he laugh about it", that's the part of @powertoold 's claim that jumped out at me.

Karpathy says HD map localization makes things easier for Waymo and others, which is obviously true (otherwise why bother?). He says it doesn't scale well, which might be true. It's certainly true that HD map scaling would be much, much harder for Tesla than a robotaxi operator.

He shouldn't say "rails", of course. Sure, they can localize very precisely. But it implies every Waymo follows the exact same path, down to the centimeter, and can't deviate from it. That's nonsense and Karpathy knows it.
 
Thanks. I guess I should have asked "when did he laugh about it", that's the part of @powertoold 's claim that jumped out at me.

I don't think they literally laugh. @powertoold is just saying Elon and Karpathy figuratively laugh at Waymo's FSD approach.

He shouldn't say "rails", of course. Sure, they can localize very precisely. But it implies every Waymo follows the exact same path, down to the centimeter, and can't deviate from it. That's nonsense and Karpathy knows it.

Yes, that is the part I object to. I don't object to saying Waymo builds HD maps with lidar or has accurate localization on the maps. That part is obviously true. The "rails" part is wrong. And yes, Karpathy should know better.