Autonomous Car Progress

[voyager]

I read Buttigieg saying that AVs need an eyesight test, sort of.
Anyway, what he basically says is that people need to pass an exam to become a licensed driver.
Now the car = the driver. It will therefore need to pass some sort of standardized test too.

 

[diplomat33]

I read Buttigieg saying that AVs need an eyesight test, sort of.
Anyway, what he basically says is that people need to pass an exam to become a licensed driver.
Now the car = the driver. It will therefore need to pass some sort of standardized test too.

Yeah, that is one idea that is going around for how regulators could standardize AV safety. If we had some sort of standardized test or series of standardized tests that AVs had to pass before being allowed to deploy, then we would have a nice way to determine when AVs are ready for deployment and every AV would have to pass the same standard. So we would know that every AV on the road passed an agreed upon standard test. The problem becomes figuring out that test. It would need to be much more rigorous than what humans have to pass to get a license. Also, the tests would not be perfect, they are never going to cover every single edge case. They would just need to cover enough scenarios to be good enough, meaning that we are satisfied that they sufficient test the AV for what we deem critical and important. Again, remember that AV safety does not need to be perfect. AVs will get into accidents. We just need the safety to be acceptable.

 

[EVNow]

Put simply, Cruise and Waymo have said that they use ML to predict paths of other road users and then based on those predictions, plot a safe path accordingly. Waymo uses a deep neural net called Multipath++ that maps out possible paths of other road users, with probabilities, up to 8 seconds into the future. Those predictions then feed into their Planner which maps out a safe path for the car. Anguelov has also explained that prediction and planning are interrelated since your actions will affect the actions of other road users but in turn their actions will affect your actions. He says Waymo is still working out the best way to combine prediction and planning together.

What does Waymo and Cruise do after they get predictions ? Run Monte Carlo or other simulations ? Predictions of other users basically limits the number of simulations needed to be run - otherwise it’s like “brute force”

IMO, the "Lean Driving Policy" video explains Mobileye's planning pretty well:

Mobileye rejects the traditional approaches of Tesla, Waymo or Cruise of using Monte Carlo or some other search algorithm to predict paths. They also reject end-to-end learning for planning. The video explains why they think these approaches are not the best. Simply put, Mobileye believes they don't guarantee good enough results, require you build models of other users which may not match reality well enough, and can be very computationally expensive. The video goes into more details.

I’m yet to watch that planning specific video, but I finished Shashua’s presentation. I’d like to hear what Waymo/Cruise say about this approach by Mobileye - otherwise it’s like uncontested marketing.

 

[EVNow]

I read Buttigieg saying that AVs need an eyesight test, sort of.
Anyway, what he basically says is that people need to pass an exam to become a licensed driver.
Now the car = the driver. It will therefore need to pass some sort of standardized test too.

He needs to go back to being a terrible mayor he was - every AV company will ace the test. We all know they can easily overfit for the test conditions.

 

[diplomat33]

What does Waymo and Cruise do after they get predictions ? Run Monte Carlo or other simulations ? Predictions of other users basically limits the number of simulations needed to be run - otherwise it’s like “brute force”

Waymo and Cruise do far more than just running monte-carlo or other sim. They take into account a lot of factors for their planning.

Here is the chart for Cruise's Planning which they call the "decision engine" (from the Under the Hood event):

 

[EVNow]

Waymo and Cruise do far more than just running monte-carlo or other sim. They take into account a lot of factors for their planning.

Here is the chart for Cruise's Planning which they call the "decision engine" (from the Under the Hood event):

I saw that part of Cruise presentation. They talked more about their "requirements" rather than "design".

You need to take into account a lot of things - and then you have to select the best of the "plausible" paths. Monte Carlo simulations is one way to do it ... Cruise doesn't say how they actually optimize (they have some "ML trajectory seeds", which is part of optimizing ... but not the only part).

 

[diplomat33]

I saw that part of Cruise presentation. They talked more about their "requirements" rather than "design".

You need to take into account a lot of things - and then you have to select the best of the "plausible" paths. Monte Carlo simulations is one way to do it ... Cruise doesn't say how they actually optimize (they have some "ML trajectory seeds", which is part of optimizing ... but not the only part).

Well if it helps, the CTO of Mobileye in his lean driving policy video says that "other AV companies" use either Monte-Carlo or Dynamic Programming to optimize their Planning. He says nobody uses Brute Force. So it is possible that Waymo and Cruise use either Monte-Carlo or Dynamic Programming for their optimization. I will look around and see if I can find reliable info directly from Waymo or Cruise.

 

[EVNow]

So it is possible that Waymo and Cruise use either Monte-Carlo or Dynamic Programming for their optimization.

What is dynamic programming ?

 

[diplomat33]

What is dynamic programming ?

In simple terms, you predict a single behavior for each agent and then use a lot of math to calculate an optimal path.

And here are the pros and cons of the Dynamic Programming approach.

For proper comparison, here are the pros and cons for Monte-Carlo:

 

[EVNow]

In simple terms, you predict a single behavior for each agent and then use a lot of math to calculate an optimal path.

And here are the pros and cons of the Dynamic Programming approach.

For proper comparison, here are the pros and cons for Monte-Carlo:

Tesla is betting MCTS is the correct approach - and Mobileye says it’s too computing intensive (and not as good quality). Version 11 would be interesting, may be Tesla thinks they have enough compute. After all they are now using even more compute .. ?

 

[diplomat33]

Tesla is betting MCTS is the correct approach - and Mobileye says it’s too computing intensive (and not as good quality). Version 11 would be interesting, may be Tesla thinks they have enough compute. After all they are now using even more compute .. ?

Well, I don't think it is as simple as MCTS is the right or wrong approach. It's how you use it that really matters. Computing power is only one factor but not the only one. The quality of perception and prediction data that goes into your planning matters a lot. You can have the best MCTS in the world but if it is using bad perception or bad prediction input, your planning probably won't be good. And you can have the best perception and prediction data and use MCTS but if it is poorly optimized then your planning output will probably be bad.

 

[Steerpike]

Driving home in heavy rain last night my M3 repeatedly reported "multiple cameras blocked or blinded", the wipers failed to keep the windscreen clear, and the headlights were too slow switching to and from main beam. I don't believe that "vision only" can ever be sufficiently reliable except under ideal conditions, meaning that Tesla is not even in this race to autonomy.

 

[EVNow]

Well, I don't think it is as simple as MCTS is the right or wrong approach. It's how you use it that really matters. Computing power is only one factor but not the only one. The quality of perception and prediction data that goes into your planning matters a lot. You can have the best MCTS in the world but if it is using bad perception or bad prediction input, your planning probably won't be good. And you can have the best perception and prediction data and use MCTS but if it is poorly optimized then your planning output will probably be bad.

Ofcourse - the question is given the same perception, whats the best planning approach.

 

[AndreP]

There are a ton of potential problems with all of the sensors depending on your climate. It's safe to say that getting autonomous vehicles working in something like harsh winter will present a huge array of challenges compared to moderate climates, especially if we're talking about true autonomy and selling a consumer vehicle without a steering wheel or pedals.

I'm driving a rental 2021 Ford F-150 in down to -40 right now for work and can't do a commute without sensors being occluded leading to warnings that assist/safety functions aren't available, the vehicle screaming at me when backing up, etc. I would need to wash/wipe them down after every drive, and things often stop functioning in the middle of a cruise. Redundancy in these situations will be absolutely massive and maintaining them will likely be quite a bit of work until they can somehow build in ways to keep the sensors functional in more situations.

No spot on the vehicle is 100% safe either, my back-up camera is on the top lip of the tailgate like 5' off the ground and is often unusable in this weather.

 

[Steerpike]

Zoox Robotaxi hardware looks like a reasonable example of what is required for dependable autonomy and their software appears to be making very advanced observations and decisions.

I'm trading my 2019 M3 in for a MY in the next few weeks but I didn't include FSD.

 

[diplomat33]

Nice 6 mn video from Zoox on how they do calibration, mapping and localization. They mention that if you don't calibrate the sensors, the car will be like cross eyed and the perception looks wonky. The video also shows what the HD maps look like. It is a cool, informative video IMO.

 

[diplomat33]

Protocol's experts answer the question: What will be the biggest milestone or news for the autonomous vehicle industry in 2022?

The moments that'll define autonomous vehicles in 2022

Regulatory clarity, a push toward sustainability and explainable AI breakthroughs could be in store, the experts say.

www.protocol.com

 

[voyager]

So, as I see it: still differeces (no agreement) when it comes to:
1. what type of on board 'intelligent stuff' with which to achieve FSD
2. which trajectory prediction may lead to better anticipation
3. which simulation and testing procedures

Am I forgetting something?
Okay. let's extrapolate. How many years will it take to come to some sort of standardization and certification?

Or do I perceive things too strictly? In other words, like we have more smartphones and mobile
operating systems (Android etc.), we will see more hi-tech roads leading towards FSD.

 

[diplomat33]

So, as I see it: still differeces (no agreement) when it comes to:
1. what type of on board 'intelligent stuff' with which to achieve FSD
2. which trajectory prediction may lead to better anticipation
3. which simulation and testing procedures

Am I forgetting something?
Okay. let's extrapolate. How many years will it take to come to some sort of standardization and certification?

Or do I perceive things too strictly? In other words, like we have more smartphones and mobile
operating systems (Android etc.), we will see more hi-tech roads leading towards FSD.

Yeah, I think you are perceiving things too strictly. Autonomous driving is not going to be uniform tech where every single AV uses the exact same hardware and software. Yes, like smart phones, there will be different AVs in form and factor, but with the same function. And we are still in the early days of autonomous driving where different companies have different ideas for the best way to achieve safe autonomous driving. We are still in that "race" between different approaches.

Eventually, we will see standards but they will not dictate the exact planning or perception algorithm that every AV must use. Standards will set the goals and requirements, like you must be able to keep a safe space from pedestrians with some mathematical equation for calculating "safe space". The specific hardware and software for achieving that goal will be up to each AV company. Remember that the standards that we have now or that are being developed make a point that they are technology agnostic, meaning that the standard must not depend on a specific technology.

 

[drtimhill]

Yeah, that is one idea that is going around for how regulators could standardize AV safety. If we had some sort of standardized test or series of standardized tests that AVs had to pass before being allowed to deploy, then we would have a nice way to determine when AVs are ready for deployment and every AV would have to pass the same standard. So we would know that every AV on the road passed an agreed upon standard test. The problem becomes figuring out that test. It would need to be much more rigorous than what humans have to pass to get a license. Also, the tests would not be perfect, they are never going to cover every single edge case. They would just need to cover enough scenarios to be good enough, meaning that we are satisfied that they sufficient test the AV for what we deem critical and important. Again, remember that AV safety does not need to be perfect. AVs will get into accidents. We just need the safety to be acceptable.

The problem with such a test is that "gaming" the test would almost certainly be rampant unless the test had sufficient variability to stop makers hard-wiring knowledge of the test as a special-case into the car (think how computer makers try to rig benchmark test results in various dubious ways). And if its not standardized then you are open to all sorts of accusations of favoritism. Not saying I'm against the idea, it makes a lot of sense .. it's just going to be tough to get it right (and fair).