Autonomous Car Progress

[daniel]

HD maps are safer for sure, if properly created and maintained.

Karpathy doesn't think HD maps are scalable, and even if they are scaled, they need to be maintained, which may be either cost prohibitive or not reliable enough given human limitations. The funny thing is that many companies are using vision to create rough HD maps, which begs the question: once your vision is good enough, why even use HD maps?

Because vision is processor-intensive. And of course with connectivity, the fleet of cars could constantly update the maps, feeding any discrepancies back to the server. It will take less local processing power to confirm a map detail using vision than to resolve that detail from scratch from raw vision data. So the on-board computer checks and confirms map details, and if the map is wrong in any detail the on-board computer resolves the item and sends a report to the server. Thus the map speeds up the whole process and the fleet constantly upgrades the map. Each car benefits from earlier cars via the map.

 

[willow_hiller]

So the on-board computer checks and confirms map details, and if the map is wrong in any detail the on-board computer resolves the item and sends a report to the server. Thus the map speeds up the whole process and the fleet constantly upgrades the map. Each car benefits from earlier cars via the map.

Doesn't this process make HD maps redundant? The entire point of HD maps is that they're supposed to contain information that the fleet cannot obtain easily via visual confirmation. If the HD map does not supplement the visual information with new and novel data, what use is it?

If an HD map can be easily created via visual information at a pace where it's safe for the vehicle to encounter scenarios where the HD map is wrong, then the vision system alone is sufficient.

 

[powertoold]

Thus the map speeds up the whole process and the fleet constantly upgrades the map. Each car benefits from earlier cars via the map.

Sounds like SD maps are adequate, which is what Tesla is doing.

 

[diplomat33]

Eh, if you mean level 2, I agree lol.

Some new Waymo video:

No. It would still be Level 4.

 

[diplomat33]

Sounds like SD maps are adequate, which is what Tesla is doing.

Since Tesla is doing L2, I agree.

 

[renim]

Back to HD maps, it seems nissan skyline (infiniti) elsewhere comes with hands off propilot 2.0 in Japan.

HD maps is central to their approach, as to how/what is stored, who knows, there are many ways to skin a cat so to speak.

One aspect of propilot 2 is that there are 5 radars on the nissan, a forward radar and 4 corner radars. The 'HD' map could merely be a data log from an earlier lidar trip correlated with radar/GPS results. Much speculation, but it seems each HD map needs to generated bespoke to each car manufacturer's system, no use for 3rd party HD maps....

2: Hands-off driving is possible when driving in a single lane, on the condition that the driver remains attentive on the road ahead and is prepared to immediately take manual control of the steering wheel when conditions of the road, traffic and vehicle require it.
*3: The hands-off feature is not available in tunnels where a GPS signal cannot be established, on expressways that have two-way traffic, on winding roads, in tollgate areas or merging lanes. When entering a road section where hands-off driving is not available, the system will alert in advance so the driver can take manual control of vehicle steering.

 

[diplomat33]

As @powertoold said, I think HD maps could quickly become unsafe if they're not updated in a timely manner. Right now, it seems like Tesla has issues maintaining a really basic HD map of just speed limits (a secondary road near my house has had 40 MPH as the speed limit for over a year since it changed to 35 MPH). Imagine if instead of just a speed limit, your car also used that out of date HD map to anticipate intersections, curbs, crosswalks, etc. It would constantly be expecting a curb in a certain area, and struggling with the neural network equivalent of cognitive dissonance. Past a certain point, it would be much safer for the vehicle to realize the map is out of date, and dispense with it entirely.

Sure, if you let your HD maps get really bad, that would be potentially unsafe. But I don't think it would be that hard to update maps when something major changes. Waymo has 20M autonomous miles so far and from what I gather, the maps are not wrong very often. it seems from the tweet, that the main thing that will cause the HD maps to be wrong is major road construction. And like Waymo wrote, their cars are designed to operate in those special instances where the HD maps are wrong.

Doesn't this process make HD maps redundant? The entire point of HD maps is that they're supposed to contain information that the fleet cannot obtain easily via visual confirmation. If the HD map does not supplement the visual information with new and novel data, what use is it?

If an HD map can be easily created via visual information at a pace where it's safe for the vehicle to encounter scenarios where the HD map is wrong, then the vision system alone is sufficient.

No. HD maps will still provide information that cannot be obtained by vision. For example, the HD map could include traffic information or that a lane is closed because of a festival. But the main point of HD maps is to add safety by adding redundancy, so you are not depending on just one sensor. It will be safer to have 2 sensors (HD maps + Vision) rather than just 1.

 

[renim]

Maybe all a HD map needs to include is data about what can be ignored.

So when a vehicle sensor notices a gore point, or a truck accross lanes, and its not in the HD map as on route, the car takes action to avoid the fatality.

 

[willow_hiller]

No. HD maps will still provide information that cannot be obtained by vision. For example, the HD map could include traffic information or that a lane is closed because of a festival. But the main point of HD maps is to add safety by adding redundancy, so you are not depending on just one sensor. It will be safer to have 2 sensors (HD maps + Vision) rather than just 1.

HD maps only make sense if they contain information that cannot be determined by the vision system. Traffic is a good example (or a full road closure, for the purpose of routing around it), but I don't think a lane closure is.

1. The first vehicle encounters a lane closure that is not on the HD map. It's traveling at 60 MPH. Is it able to determine that the lane is closed via nothing but vision within time to avoid damage? If yes, it beams the state of the lane back to HQ and the lane is now marked as closed on the HD map. Go to step 2. If no, then the vehicle cannot safely operate without an HD map.

2. The lane is now marked as closed on the HD map. Every vehicle that passes it uses the same advanced vision system as the first car to check whether the lane is still closed. "Yep, still looks closed to me" or "No, it's open now, I better tell the HD map to update."

In this case, the HD map hasn't provided any new information that the vehicles must be equipped to ascertain themselves within a safe period of time. It's not providing redundancy, and it's not improving safety.

And in the event of an HD map containing a full road closure... if all of the autonomous vehicles route around it, will it ever be marked as open again if the cars never pass that way?

 

[Mardak]

HD maps can tell the car where to stop in the middle of an intersection if they have to wait before completing an unprotected left turn. That is not information you can directly get from vision.

Pretty sure those HD maps from the Chinese company are showing stop lines based on actual stop lines. I would think a vision system can pick up the stop line relatively easily:

 

[diplomat33]

HD maps only make sense if they contain information that cannot be determined by the vision system. Traffic is a good example, but I don't think a lane closure is.

1. The first vehicle encounters a lane closure that is not on the HD map. It's traveling at 60 MPH. Is it able to determine that the lane is closed via nothing but vision within time to avoid damage? If yes, it beams the state of the lane back to HQ and the lane is now marked as closed on the HD map. Go to step 2. If no, then the vehicle cannot safely operate without an HD map.

2. The lane is now marked as closed on the HD map. Every vehicle that passes it uses the same advanced vision system as the first car to check whether the lane is still closed. "Yep, still looks closed to me" or "No, it's open now, I better tell the HD map to update."

In this case, the HD map hasn't provided any new information that the vehicles must be equipped to ascertain themselves within a safe period of time. It's not providing redundancy, and it's not improving safety.

Ok, maybe lane closure is not the best example but there are plenty of cases where HD maps do provide increase safety.

And HD maps can still be useful even if they provide information that vision can also get. It adds redundancy.

For example crosswalks. Even if vision can see the crosswalk, it will more reliable if you also have the precise position of the crosswalk from HD map. Vision can miss the crosswalk if the lines are faded or if another vehicle is blocking the view of the crosswalk. But with the HD map data, you car is guaranteed to know exactly where to stop every time. Also crosswalks also tell the car where pedestrians will cross. So it gives the car information about what paths the pedestrians are more likely to take.

Pretty sure those HD maps from the Chinese company are showing stop lines based on actual stop lines. I would think a vision system can pick up the stop line relatively easily:
View attachment 568688

What if the stop lines are faded or blocked from view by other vehicles? With HD maps, your car will be able to stop precisely where it should, even if vision can't see the stop lines.

 

[willow_hiller]

Vision can miss the crosswalk if the lines are faded or if another vehicle is blocking the view of the crosswalk. But with the HD map data, you car is guaranteed to know exactly where to stop every time.

If a vision system cannot pick up a faded crosswalk, then it can reliably operate more safely than a human driver in every case where that HD map is accurate and up to date, but it will also reliably operate less safely than a human driver in every case where the HD map is out of date.

On average, if the HD map is maintained well, then the fleet will drive more safely than a human. But is ever driving less safely than a human acceptable for L5?

 

[diplomat33]

"HD maps enable vehicles to see beyond the driver’s field of view, providing an accurate representation of the road ahead and information on the surrounding environment. Though not limited to use only within autonomous vehicles, this technology will be particularly beneficial to vehicles with automated features.

Although self-driving vehicles will not rely solely on HD maps, this technology can significantly enhance the functionality of autonomous driving features. “Lower levels of autonomy can function without an HD map, but it is not as safe or smooth,” Willem Strijbosch, Head of Autonomous Driving at TomTom, told M:bility.

“If there is very dense traffic ahead, you can ‘see’ through all of those cars, and understand what is around the corner or even two kilometres ahead of you. It does not matter if it is snowing or pouring down with rain, the map remains clear and increases the safety of these assistance systems.”

The goal is to improve an autonomous vehicle’s awareness and to provide precise information on where it is on the road, often referred to as ‘contextual awareness.’ Looking ahead, Strijbosch affirms that Level 3 and Level 4 autonomous vehicles must have HD mapping technology – it is not up for debate, he suggests: “Level 2 vehicles are on the road today without a HD map, but all of the car manufacturers are looking to integrate these maps to make their systems smoother, and ultimately, safer.”

HD maps – the key to autonomous driving success? | Automotive World

 

[willow_hiller]

HD maps enable vehicles to see beyond the driver’s field of view, providing an accurate representation of the road ahead and information on the surrounding environment.

A sinkhole appears in the road just around a sharp bend (out of the field of view). It's new so it's not on any HD map. Are we willing to let the first vehicle that encounters the situation crash into the sinkhole so that all future vehicles can avoid it? Or do we focus our efforts on training a vision system that can identify a sinkhole within a fraction of a second of it coming into view so all vehicles can avoid it without prior knowledge?

 

[diplomat33]

A sinkhole appears in the road just around a sharp bend (out of the field of view). It's new so it's not on any HD map. Are we willing to let the first vehicle that encounters the situation crash into the sinkhole so that all future vehicles can avoid it? Or do we focus our efforts on training a vision system that can identify a sinkhole within a fraction of a second of it coming into view so all vehicles can avoid it without prior knowledge?

I don't think a sinkhole suddenly appearing is something that you use HD maps for. Of course, you devote effort to vision. HD maps are for other situations.

Again, you are not driving on just HD maps alone. You drive with HD maps AND Perception!

But even with vision, there are situations where HD maps will still be useful to increase safety even more. So even if you have great vision, HD maps are not obsolete.

 

[willow_hiller]

I don't think a sinkhole suddenly appearing is something that you use HD maps for. Of course, you devote effort to vision. HD maps are for other situations.

Again, you are not driving on just HD maps alone. You drive with HD maps AND Perception!

But even with vision, there are situations where HD maps will still be useful to increase safety even more. So even if you have great vision, HD maps are not obsolete.

I was just trying to come up with an extreme example that is difficult for a vision system to catch, but has potentially life threatening consequences.

I've got a logical progression, let me know at what point we don't agree anymore:

1. If an HD map improves safety, that means it contains useful information that cannot be ascertained through vision alone (e.g. forewarning of obstacles)
2. If information is needed to improve safety, then the vehicle is less safe without it
3. If an HD map contains information that cannot be ascertained through vision alone and is needed to improve safety, then the vision system is incomplete
4. If the vision system is incomplete, the vehicle is less safe without an HD map
5. If the vehicle is less safe without an HD map, then the vehicle will be less safe in situations where the HD map is incomplete

The existence and usefulness of an HD map means that the vision system is incomplete, and therefore less safe in all situations where the HD map fails.

 

[diplomat33]

1. If an HD map improves safety, that means it contains useful information that cannot be ascertained through vision alone (e.g. forewarning of obstacles)

True but incomplete. HD maps can also improve safety by providing useful information that can be ascertained through vision alone but where vision might be unreliable in some cases. HD Maps can also provide a "second opinion" where you need to make sure that vision is accurate. Having that "second opinion" will improve safety by reducing the situations where the car fails because the vision failed.

2. If information is needed to improve safety, then the vehicle is less safe without it.

True.

3. If an HD map contains information that cannot be ascertained through vision alone and is needed to improve safety, then the vision system is incomplete.

False. The vision system can be complete but the HD map might provide information that is not possible to get from vision alone.

4. If the vision system is incomplete, the vehicle is less safe without an HD map.

True.

5. If the vehicle is less safe without an HD map, then the vehicle will be less safe in situations where the HD map is incomplete.

True.

The existence and usefulness of an HD map means that the vision system is incomplete, and therefore less safe in all situations where the HD map fails.

False. You can have complete vision and still find HD maps useful.

 

[daniel]

Doesn't this process make HD maps redundant? The entire point of HD maps is that they're supposed to contain information that the fleet cannot obtain easily via visual confirmation. If the HD map does not supplement the visual information with new and novel data, what use is it?

If an HD map can be easily created via visual information at a pace where it's safe for the vehicle to encounter scenarios where the HD map is wrong, then the vision system alone is sufficient.

No, because the HD map data is not "easily" created. It takes a lot of processor power. If the local processor only has to create data where the visual information does not match what's on the map than it's only creating a tiny amount of information on the fly. If you had an infinite amount of processing power and speed locally, the map would be redundant. The map is useful precisely because local processing power is limited.

We basically see this in the Waymo car with no driver controls at all. It only operates where it has the map data, but Tesla is very far from this with its Level 2 systems.

 

[willow_hiller]

No, because the HD map data is not "easily" created. It takes a lot of processor power. If the local processor only has to create data where the visual information does not match what's on the map than it's only creating a tiny amount of information on the fly. If you had an infinite amount of processing power and speed locally, the map would be redundant. The map is useful precisely because local processing power is limited.

Is the car able to determine that the HD map is accurate without using its full processing power? Because it sounds like these cars need to be constantly vigilant for areas of the HD map that are out of date.

To me it sounds like they would be running at full processing power regardless of the presence of an HD map.

 

[Mardak]

What if the stop lines are faded or blocked from view by other vehicles? With HD maps, your car will be able to stop precisely where it should, even if vision can't see the stop lines.

How is that safe behavior to rely on HD maps when vision does not see the stop line? If these turn lanes are removed because of a new traffic pattern, following the HD map to "stop precisely where it should" would place the vehicle right in the way of other vehicles potentially leading to an accident.

In fact the street view screenshot shows construction of what seems to be a subway line taking up additional space for workers, so after construction completion, the turn lanes may very well be moved to a different location. Vision failing to see the stop line could be because the line is now somewhere else from reopened lanes, so relying on HD maps is less safe in this case.