Autonomous Car Progress

[EVNow]

So if HD maps aren't scalable what mapping are they going to use? You can't always rely on GPS

Normal "SD" maps - which themselves can't be easily kept updated - as we see with all the Tesla map issues. So thinking magically HD maps will be all updated to be useful everyday is just fantasy.

We hear this kind of conflicting pretzel argument ...
- HD maps are absolutely required
- HD maps are "free"
- But, HD Maps don't need to be kept upto date
- Since, the car will drive just fine if HD Map is not up to date or is wrong

 

[ZeApelido]

Personally, I think you need real world testing during development. But I am intrigued by the idea of trying to "solve FSD" with simulation only. Simulation is becoming very realistic both physically and in agent behavior. And we've seen AI like AlphaGo learn how to play. So the idea of a driving AI learning how to drive on its own entirely in a simulation, is thought provoking to me.

It's not going to work. It's simply that you and others like @Bladerskb have no technical expertise in the data science and algorithm development domain so you swoon to the simulation hype because "oh those are fancy!".

Meanwhile in every other real world algorithm / machine learning project, the idea that you could rely only on simulated data would get you laughed out of the room.

Think about that, every other real world project needs lots of real data to discover all the random findings and previously undiscovered phenomema, but somehow the hardest challenge - self driving cars - does not?

Literally - this isn't a game. Simulation only can word in games because the rules and possible outcomes are extremely well defined. It's a totally different world.

Real world "AI" needs lots of real data X lots of simulations (to fill in gaps in solution spaces to not overfit).

No experienced machine learning expert would agree with you.

 

[diplomat33]

It's not going to work. It's simply that you and others like @Bladerskb have no technical expertise in the data science and algorithm development domain so you swoon to the simulation hype because "oh those are fancy!".

Meanwhile in every other real world algorithm / machine learning project, the idea that you could rely only on simulated data would get you laughed out of the room.

Think about that, every other real world project needs lots of real data to discover all the random findings and previously undiscovered phenomema, but somehow the hardest challenge - self driving cars - does not?

Literally - this isn't a game. Simulation only can word in games because the rules and possible outcomes are extremely well defined. It's a totally different world.

Real world "AI" needs lots of real data X lots of simulations (to fill in gaps in solution spaces to not overfit).

No experienced machine learning expert would agree with you.

This is a classic strawman! You are falsely pretending that Waabi and I are suggesting only using simulated data, with no real world data at all, so that you can ridicule it. But I never said that! I support using real world data. Also, Waabi is using real world data. They say so clearly here:

Like Cruise, Waabi bases its virtual world on data taken from real sensors, including lidar and cameras, which it uses to create digital twins of real-world settings.

So Waabi collects real world data. They simply believe that once you have a digital fac simile of the real world, you can do most testing and training in the digital copy. I do not agree with their approach but that is what they are claiming.

I was also very clear that I support real world testing. I simply said that I was intrigued by Waabi's approach. Not the same thing!

 

[Daniel in SD]

Normal "SD" maps - which themselves can't be easily kept updated - as we see with all the Tesla map issues. So thinking magically HD maps will be all updated to be useful everyday is just fantasy.

We hear this kind of conflicting pretzel argument ...
- HD maps are absolutely required
- HD maps are "free"
- But, HD Maps don't need to be kept upto date
- Since, the car will drive just fine if HD Map is not up to date or is wrong

So you're saying that Waymo and Cruise can achieve what appears to be safe driverless operation because their HD maps are up to date 100% of the time? Presumably when the HD maps are out of date they do not drive as fine as they normally do but when you average that performance with the vast majority of the time when the maps are up to date you do get better overall performance.

 

[diplomat33]

Waymo posted a new blog explaining how they use "key points" to understand gestures and body poses for the purpose of predicting behavior of road users. Put simply, instead of using bounding boxes, camera vision detects key points on the human body and uses those key points to predict behavior. Waymo uses both camera vision and lidar to detect "key points".

Key points are a compact and structured way to convey human pose information otherwise encoded in the pixels and lidar scans for pedestrian actions. These points help the Waymo Driver gain a deeper understanding of an individual's actions and intentions, like whether they’re planning to cross the street. For example, a person's head direction often indicates where they plan to go, whereas a person's orientation tells you which direction they are already heading. While the Waymo Driver can recognize a human's behavior without using key points directly using camera and lidar data, pose estimation also teaches the Waymo Driver to understand different patterns, like a person propelling a wheelchair, and correlate them to a predictable future action versus a specific object, such as the wheelchair itself.

The Waymo Driver generates key points in the "wild" for all nearby road users, which is orders of magnitudes harder as our Driver often encounters up to hundreds of pedestrians at a single intersection, many of which can be occluded by other objects.

The Waymo Driver uses real-time data from our sensor suite, including our lidars, which feed into our neural-network models to localize key points in three dimensional space. Waymo created its own methodologies to generate high-quality labels to identify the joints in a 3D space, which enabled training human pose models to further improve the safety of the Waymo Driver. This also means that Waymo's key point technology doesn’t identify an individual person, but rather aggregates data points and provides us with a better capability to recognize that a person exists and where they may be going, which is especially beneficial for partially visible pedestrians that might be stepping out of a vehicle or sitting near the road. Additionally, we’ve optimized our system to run onboard the vehicle in real-time, with high precision and low latency, to enhance its behavior-prediction models and allow the Waymo Driver to quickly and safely handle any situation.

Some example of how Waymo uses "key points": predicting behavior of large crowds, detect gestures of traffic controllers or cyclists, detect indecisive pedestrians crossing the street, and detect partially occluded objects like detecting a pedestrian from jus a leg sticking out from behind a car.

Utilizing key point and pose estimation for the task of autonomous driving

Historically, computer vision relies on rigid bounding boxes to locate and classify objects within a scene; however, one of the limiting factors in detection, tracking, and action recognition of vulnerable road users, such as pedestrians and cyclists, is the lack of precise human pose...

blog.waymo.com

 

[Nocturnal]

"I use beta less than ever...". I find that odd. But since this is a subjective topic, I'll refrain from dragging this into a contest in this thread. So I'll just share my experience. In re-reading my note below, I do sound like a Fanboy, but considering we bought our first Tesla because of all this technology and design, it's expected. I have high tolerance for solving technical challenges, but that's just me.

I still use it always, streets and all intersections included. The only time I haven't is if I'm late for something as I can easily drive faster myself. However, I don't like it when it's taken away (exceeding 80 mph for example). I literally trust the car more than myself in most cases, and absolutely prefer the car change lanes for me (that was many versions ago actually). I still need to nudge it at intersections as it's still timid (good thing). And of all the left hand turns it's taken on complex intersections, I've never seen it miss the turn (either too wide or hit a curb, and as other cars are turning with me). So far, 100% accuracy, and I thought that would be the hardest part (those left turns).

For the first year, in fact, this worried me because I saw it misalign to a new lane while going straight through intersection (if the road curved)... so how could it possibly do a full 90 turn and hit the correct lane? Well folks... it does it 100% for me. And Tesla team did this by solving an accuracy problem with nearby object. So it was a single vision accuracy improvement that suddenly allowed for accurate turns and no more curbs (so far). Add in lately how it anticipates the timing of other cars in the future... it's nearly a miracle on wheels.

The game for me is trying to make the people believe I'm driving the car, so that means having to nudge the accelerator occasionally (however, the video sample of V11 seems to not need that anymore). The steering wheel could use some damping at times, but other times I've seen that wheel spin VERY FAST and stop perfectly on track. Folks... I CANNOT DO THIS MANEUVER (without some over/undershooting)! Top it off with the fact that it was turning into a parking lot and there were cars right there exiting. It was actually the moment I saw my car as a robot having extreme precision.

I'm a bit of a daredevil in how I allow the car to play out even when I see it hesitate. This requires a LOT of trust in Tesla combined with high confidence in my co-piloting skills. And I say skills because it is learned. This means as it gets better and I get better, it is safer - for me at this time in the program. I'm certain others do not have the risk tolerance I have, so I expect some fear (or annoyance) out there.

Didn't want to clog up the roundtable thread.

Since the last few updates my car will randomly veer into the curb or other lanes while driving. I'm pretty comfortable taking a risk and intervening at the last second. With earlier versions I could see the errors and felt like they were ok. I'm happy to take over if the car get's confused etc.

I tried 10.2 for the first time this morning. Just 1 mile into my commute back from dropping off the kids it swerved HARD towards the curb. It does this on most drives now. At this point it's hard for me to justify testing it more. FSD just seems to have become worse for me since maybe 10.5.

Now it's just not usable for me. I can't risk a head on collision. I also used to trust AP nearly 100%, but now I find it not usable on the rural highways that I formerly used it the most on due to what I believe is moving from radar to vision.

 

[SOULPEDL]

Ok, I've seen some of the swerving, I think. It happened in a parking lot and pulled right in front of a truck coming at me. All slow speed stuff and I only think about what I should do... like let it play out or not. But then I tell my wife "Not ready yet" which we all believe is true.

There is an exit from the bank where the car seems to be in a hurry to get over to the far left lane (3) and yet the next turn is a right just a mile away. That's a bug. At first I intervened as I thought it was headed for the curve. However, I let it play out, same spot, and it correct. So maybe it wasn't going to hit your curb, but it was surprising for sure.

I guess I have a high tolerance having never ever been in an accident at age 61. Spun out and gone off plenty of roads, but never hit an object or a car, ever.

 

[Nocturnal]

Ok, I've seen some of the swerving, I think. It happened in a parking lot and pulled right in front of a truck coming at me. All slow speed stuff and I only think about what I should do... like let it play out or not. But then I tell my wife "Not ready yet" which we all believe is true.

There is an exit from the bank where the car seems to be in a hurry to get over to the far left lane (3) and yet the next turn is a right just a mile away. That's a bug. At first I intervened as I thought it was headed for the curve. However, I let it play out, same spot, and it correct. So maybe it wasn't going to hit your curb, but it was surprising for sure.

I guess I have a high tolerance having never ever been in an accident at age 61. Spun out and gone off plenty of roads, but never hit an object or a car, ever.

It very well could be a numbers game. I haven't been driving that much lately. It's possible it doesn't like my area, though it's in a very straightforward suburban area with nicely marked streets in a grid.

 

[EVNow]

So you're saying that Waymo and Cruise can achieve what appears to be safe driverless operation because their HD maps are up to date 100% of the time? Presumably when the HD maps are out of date they do not drive as fine as they normally do but when you average that performance with the vast majority of the time when the maps are up to date you do get better overall performance.

I'm obviously not saying that. I've no idea what they do - but obviously keeping HD maps up-to-date in a tiny area is easier than on Millions of miles of roads.

The argument is simple - If you don't need up-to-date HD maps for driving safely, why do you need them at all.

I've not seen any company make the argument that HD Maps are viable for Consumer AVs and present a plan on when they are going to sell consumer AVs using those HD Maps at scale.

 

[AKAlex]

I'm obviously not saying that. I've no idea what they do - but obviously keeping HD maps up-to-date in a tiny area is easier than on Millions of miles of roads.

The argument is simple - If you don't need up-to-date HD maps for driving safely, why do you need them at all.

I've not seen any company make the argument that HD Maps are viable for Consumer AVs and present a plan on when they are going to sell consumer AVs using those HD Maps at scale.

I think this is a reasonable argument. How much human labor is involved in keeping these maps updated? Will this approach only be profitable in dense urban environments?

 

[Daniel in SD]

I'm obviously not saying that. I've no idea what they do - but obviously keeping HD maps up-to-date in a tiny area is easier than on Millions of miles of roads.

The argument is simple - If you don't need up-to-date HD maps for driving safely, why do you need them at all.

I've not seen any company make the argument that HD Maps are viable for Consumer AVs and present a plan on when they are going to sell consumer AVs using those HD Maps at scale.

You can still drive more safely than a human even if you drive less safely than a human for 0.1% of the time that the maps are wrong. It's all about probabilities.
I've lost track of the consensus here, are AV Maps HD Maps or not? I don't see any reason a company using HD Maps couldn't transition to using AV Maps like Mobileye did.

 

[diplomat33]

I think this is a reasonable argument. How much human labor is involved in keeping these maps updated? Will this approach only be profitable in dense urban environments?

Here is what Waymo says about their updating process:

We’ve automated most of that process to ensure it’s efficient and scalable. Every time our cars detect changes on the road, they automatically upload the data, which gets shared with the rest of the fleet after, in some cases, being additionally checked by our mapping team.

The fact that they have automated most of the process to make it efficient and scalable would suggest to me that the human labor cost is probably not very high. The only time you need human labor is when you need a human to double check an update as Waymo mentions in the second sentence. They say "in some cases" being double checked by a mapping team. So not all cases require humans.

 

[diplomat33]

I've not seen any company make the argument that HD Maps are viable for Consumer AVs and present a plan on when they are going to sell consumer AVs using those HD Maps at scale.

Mobileye has.

 

[Doggydogworld]

I don't think that is the right interpretation ... there is no other way to explain why he combined
- Multiple sensors
- HD Maps .... with
- Geofencing

The explanation is clear, it just doesn't fit your narrative. Compute and sensors are too costly today and map coverage too limited today for consumer cars. But time and scale fix that. Your static world view is ironic coming from a Tesla fan.

Afterall remember .... 4 months back, GM is already working on and well into their UltraCruise effort that doesn't use HD Maps and the leadership says because HD Maps is not practical for 2 Million + miles.

Super Cruise has lidar maps but no car lidar. Ultra is the exact opposite. They're just flailing. Don't read too much into it.

Why can they map 200k miles for Super but not 2 million for Ultra? They just need 10x as many customers. Maps are the ultimate scale product. But Ultra is "more exclusive" than Super, i.e. fewer customers. You obviously can't map 2m miles for 20k customers. But the cost is trivial when spread over 20m customers.

 

[FloridaJohn]

The argument is simple - If you don't need up-to-date HD maps for driving safely, why do you need them at all.

Which scenario below are you a more confident driver?

1) On a road in a city in which you have never driven or seen before.
2) On your daily commute.

I would guess that most people would answer scenario 2. People are more confident on roads which they have driven many times before. You know which lane to be in, you know which route is the easiest/fastest, you know where the backups will start, etc.

For sure, you could drive yourself anywhere in America without that kind of intimate knowledge. You would successfully navigate the route.

That, IMO, is the difference between HD maps or not. If the car has those maps, it's like it knows all the ins and outs of your commute. If the map is wrong or out of date, it can revert back to the drive-like-you've-never-been-here-before mode. I think there would be a qualitative difference between those two modes, and I think most people would prefer the one where the car knows all the "local secrets."

 

[EVNow]

That, IMO, is the difference between HD maps or not.

Humans don’t have had maps they have very sparse, imprecise maps. I can’t tell you with cm resolution what the wife every road or lane I drive daily. I can’t even do that with my bedroom.

What is the point of high precision map if you have no idea if it’s accurate (up to date) or not ?

Then there are all these folks arguing car can automatically generate HD map and send it back home to keep it updated. If the car can generate HD Maps at run time, why do you need them beforehand ?

 

[S4WRXTTCS]

Humans don’t have had maps they have very sparse, imprecise maps. I can’t tell you with cm resolution what the wife every road or lane I drive daily. I can’t even do that with my bedroom.

What is the point of high precision map if you have no idea if it’s accurate (up to date) or not ?

Then there are all these folks arguing car can automatically generate HD map and send it back home to keep it updated. If the car can generate HD Maps at run time, why do you need them beforehand ?

Humans aren't a good comparison because we don't drive to the level of safety that autonomous cars need. We have what I'd call relaxed rules. I don't have to be that precise when I can't see the road very well. I mostly go off of what I remember from being on the same road 1000's of times.

An autonomous vehicle should contain HD maps, and a vision (Vidar/Lidar) system capable of determining where the car is in relation to the details within the HD Maps. It should be able to use all the available data to determine if the two things correlate correctly.

If a stop sign is removed it should be able to detect that, and have the information reviewed. Maybe a recent change removed the stop sign or maybe someone stole the stop sign.
If a stop sign is added it should be able to detect that, and have the map reviewed

Some issues might lead to that location being flagged as not appropriate for autonomous operation. I see no reason to allow autonomous operation to occur in areas without connectivity so things should be updated fairly quickly at least from a go/no-go perspective.

To me its not Vision versus HD maps, but instead its advocating for Vision that's augmented by HD Maps.

Maps are great for knowing all the details and rules of an intersection, and exactly where one needs to be. So an entire sequence of events can be executed in a precise manner in a way that vision only doesn't allow for.

With FSD Beta its painfully obvious (at least in WA State) that its using Visual data without having much in the way of Map data to augment that data. It has a general feeling that its reactionary, and its not just the cars its reacting to but the roads/curbs/etc. This is why it usually sucks for me at doing turns, and it can't execute quick sequences of events that are necessary for smooth driving and navigation.

If it was capable of building maps with the fleet on a continuous basis I'd likely have a really good FSD Beta commute right now where I wouldn't need to take over on a lot of turns.

Now we can say that failure is that maps aren't tightly enough integrated, but at some point it will lead to the question of how precise the maps need to be. I would argue that the maps need to be precise enough to act as temporary redundancy in case vision is temporarily blinded.

HD Maps would also help flag camera calibration malfunction.

 

[rbt123]

What is the point of high precision map if you have no idea if it’s accurate (up to date) or not ?

They serve 2 obvious purposes:

1. Knowledge of what is behind a direct obstruction. They won't try to park where a fire-hydrant is expected to be, or fail to slow-down for a school-zone despite a moving truck blocking the sign.

2. Computational efficiency, particularly in crowded environments.

Part of choosing a path for the vehicle includes some attempt at predicting the paths of the other mobile entities (pedestrians, cyclists, vehicles) for a time (up to 30 seconds). The near infinite options for a pedestrian or cyclist can be narrowed down if you have an idea of the environment they're within, even when that environment is obstructed.

Neither of those prevents driving without the map but it is far more efficient to have increased behaviour predictions for other actors on the roadway.


It's a bit like knowing that a certain bridge often has black-ice on it during certain weather conditions. A human or machine with that map information will take caution be safer because of it. The human or machine without that knowledge will almost always be perfectly fine, but it's rarely harmful to have historical knowledge about an area.

Then there are all these folks arguing car can automatically generate HD map and send it back home to keep it updated. If the car can generate HD Maps at run time, why do you need them beforehand ?

They don't need them before hand to autonomously drive through an area. Having historical and more detailed knowledge of the area improves decision making accuracy and efficiency. Decisions are still possible with strictly the visible information available.

As you say, any company which requires a detailed map in advance has a dead technology. That only works for warehouse robots (like Amazon's) and railways (like CBTC signalling).

It's the difference between just getting to the Manhattan Bridge (a tourist might be told Broadway/Bowley is the most direct route) but locals know 2nd ave is usually faster due to less congestion and not having to run around Union Square.

 

[2101Guy]

I’m watching a Doug DeMuro AMG 63 wagon review on YouTube. A nice touch? The steering wheel is capacitive touch. As long as the steering wheel senses your hand is touching it the smart cruise control remains engaged. No need to tug on the wheel every few seconds for the cruise control to know your hand is there

Listening, hi tech company Tesla?

 

[bradtem]

I am confused why people say that Tesla FSD "works" anywhere, let alone everywhere. I have not heard of any place it is said to be even close to working.

There are several approaches to making self-driving work. One approach I would call the "Tesla Master Plan" approach. As you will recall, the Tesla Master Plan was to build an expensive, niche sportscar to prove out the concepts, then later a luxury sedan and eventually scale up with more affordable cars.

This is the approach being followed by most self-driving teams -- make use of the best tools available, not worrying much about cost -- LIDAR, maps, neural nets, imaging radars, extensive simulation, you name it to get a working self-driving vehicle. Then reduce cost and expand territory to scale it out.

The one large company not following the Tesla Master Plan approach decided to try to use the lowest cost sensors, no maps, a cheap radar they later removed and primarily neural nets, and is trying to build it first cheap and at scale, rather than doing that later. In addition to using cheap sensors they even decided to stick with the sensors they were buying in volume in 2016 rather than making use of the improvements in performance and cost that come with all electronics tech.

A bit ironic, I think.