Autonomous Car Progress

[ZeApelido]

The benefit of lots of real-world data. Simulations ain't coming up with all of these:

(Bottom left: Car reflection on back of tank)
(Bottom right: knocked over cones detected as red stop light)

 

[diplomat33]

The benefit of lots of real-world data. Simulations ain't coming up with all of these:

(Bottom left: Car reflection on back of tank)
(Bottom right: knocked over cones detected as red stop light)
View attachment 553601

Why not? Why couldn't simulations be programmed to include these scenarios?

 

[ZeApelido]

Why not? Why couldn't simulations be programmed to include these scenarios?

You cannot generate events you have never experienced before.

Simulations can generate interesting new combinations of previously seen events, but not new ones.

Sorry this is just fundamentals of machine learning data handling.

 

[diplomat33]

You cannot generate events you have never experienced before.

Simulations can generate interesting new combinations of previously seen events, but not new ones.

Sorry this is just fundamentals of machine learning data handling.

I am not sure that is correct. I listened to Tom Boyd who is the VP of Simulations at Cruise. There are 2 kinds of simulations. There is replay which is what you are talking about, where you just replay existing events to the car. But there are simulations that you build, a lot like making a computer game simulation. You can build new scenarios in those.

Here is what Cruise's 3D simulations look like. I believe they can add new scenarios to these simulations.

To be clear, I am not suggesting that you don't need any real-world data or real-world testing. Of course you do. Simulations do not replace real world data. But I do believe it is possible to generate new scenarios in simulations.

 

[daniel]

Sure, but unless v2v is necessary for fsd, then it follows that statement you mentioned.

Nope. "The perfect is the enemy of the good" is saying that "perfect" is unattainable. The best we can hope for is "good." It is not saying that you should never do something unless it's necessary. We do lots of things that help make something better but that are not necessary to get a job done.

FSD itself is not necessary to provide transportation. But it has the potential to make transportation safer. V2V might not be necessary to achieve FSD, but if it makes FSD measurably safer then it's a good thing. Again, we're not trying to make FSD perfect because that's an impossible goal. We're trying to make it as safe as we reasonably can, and it would be foolish to dismiss out of hand a potential tool merely because that tool itself is not perfect.

 

[ZeApelido]

I am not sure that is correct. I listened to Tom Boyd who is the VP of Simulations at Cruise. There are 2 kinds of simulations. There is replay which is what you are talking about, where you just replay existing events to the car. But there are simulations that you build, a lot like making a computer game simulation. You can build new scenarios in those.

Here is what Cruise's 3D simulations look like. I believe they can add new scenarios to these simulations.

To be clear, I am not suggesting that you don't need any real-world data or real-world testing. Of course you do. Simulations do not replace real world data. But I do believe it is possible to generate new scenarios in simulations.

Those simulations are limited to the creativity of the mind to come up with situations that cars might see. What the mind is creating is relatively less creative than actual new real world edge cases. If perception was so easy to model using simulations, then camera perception would be already solved.

This is not some specific AV driving phenomenon, this is common in machine learning.

Data scientists would generally all choose a ton of real world data over simulated data, because the statistical distribution it samples from is from the actual statistical distribution the product will see (aka real world). The simulated data generates "new" cases that are really interpolated cases from previously seen cases. But if the previously seen cases aren't that big, the simulations aren't generating really unique edge / fat tail cases.

Like, this really isn't that debatable in machine learning. People don't go around collecting a few real cases and simulating a lot more unless they have to.

 

[diplomat33]

Those simulations are limited to the creativity of the mind to come up with situations that cars might see. What the mind is creating is relatively less creative than actual new real world edge cases. If perception was so easy to model using simulations, then camera perception would be already solved.

This is not some specific AV driving phenomenon, this is common in machine learning.

Data scientists would generally all choose a ton of real world data over simulated data, because the statistical distribution it samples from is from the actual statistical distribution the product will see (aka real world). The simulated data generates "new" cases that are really interpolated cases from previously seen cases. But if the previously seen cases aren't that big, the simulations aren't generating really unique edge / fat tail cases.

Like, this really isn't that debatable in machine learning. People don't go around collecting a few real cases and simulating a lot more unless they have to.

Yes, everybody knows that machine learning requires a lot of diverse real world data. That is why real world data is critical. Nobody disputes that. Simulation data cannot replace real world data. I am merely suggesting that simulations can play a useful role in generating possible edge cases.

 

[ZeApelido]

I am merely suggesting that simulations can play a useful role in generating possible edge cases.

My post was showing real-world examples of things that likely have not been put into simulations and point was showing the advantage of real world data.

This is most definitely true.

Simulations may kinda sort of come up with some edge cases (but at the sacrifice of also producing other instances that are unrealistic). But not really. And they are biased by the creator.

Simulations do not create new edge cases. They fill in gaps.

 

[daniel]

Simulations are an excellent,safe, and cheap way to generate hypotheses for real-world testing. They're not a substitute for real-world testing, they are an adjunct to it. In every field of engineering simulations have slashed development costs. You still need to include real-world testing, and you need to validate your models and your final product with real-world testing.

 

[Conflagrare]

Using simulations to supplement testing is fine.

Using simulations to train AI is wrong. You would Just end up with an AI that is very good at video games.

 

[diplomat33]

Speaking of simulations, Waymo's CTO, Dmitri Dolgov has an interesting answer about the role of simulations versus real world testing:

"You asked about start-ups and companies just getting started. To get started, you don't need it (simulations). You can make a lot of progress by just driving around the physical world. If you don't have a simulator and you want to launch, should you pack it up and go home, yeah, I think so.

Really, you drive in either the physical world or in simulations for 2 reasons. One is for testing. The other one is for validation. If you are developing the system, especially in the early days, then driving for testing is sufficient. And you don't need to do much, in either simulated or the physical world. Once you get into validation, that's orders of magnitudes more. Whether you do it in simulations, but then you need to convince yourselves that the results you are getting in simulations are realistic and valid. But then you need to crown it with a lot of driving and a lot of empirical observed data that only comes from the physical world."

Source:

 

[Soda Popinski]

Using simulations to supplement testing is fine.

Using simulations to train AI is wrong. You would Just end up with an AI that is very good at video games.

Really depends on the type of AI you're training. If you want to get accurate distance estimates, then you handicap yourself if you use real-world camera only data, as you don't have ground truth distance to train against. So that means either simulations or camera + Lidar data. Nice thing about simulation are, you can rerun it in different lighting conditions or from slightly different positions, which you can't do with recorded data.

 

[powertoold]

Nice thing about simulation are, you can rerun it in different lighting conditions or from slightly different positions, which you can't do with recorded data.

Sure, but your light simulations are based on your own biases and heuristics lol.

 

[diplomat33]

This is big news!

"Waymo is now the exclusive global L4 partner for Volvo Car Group, a global leader in automotive safety, including its strategic affiliates Polestar and Lynk & Co. International. Through our strategic partnership, we will first work together to integrate the Waymo Driver into an all-new mobility-focused electric vehicle platform for ride-hailing services. "

Waypoint - The official Waymo blog: Partnering with Volvo Car Group to scale the Waymo Driver

The blog mentions ride-hailing but I think this deal does open the door to Volvo delivering L4 cars to consumers that use the Waymo software.

 

[daniel]

This is big news!

"Waymo is now the exclusive global L4 partner for Volvo Car Group, a global leader in automotive safety, including its strategic affiliates Polestar and Lynk & Co. International. Through our strategic partnership, we will first work together to integrate the Waymo Driver into an all-new mobility-focused electric vehicle platform for ride-hailing services. "

Waypoint - The official Waymo blog: Partnering with Volvo Car Group to scale the Waymo Driver

The blog mentions ride-hailing but I think this deal does open the door to Volvo delivering L4 cars to consumers that use the Waymo software.

As long as the car is electric, I'll buy it if it's the first to become available here. The consumer version, that is.

 

[Knightshade]

This is big news!

Naah.

"X and Y announce partnership to develop something someday" isn't big news.

There's been a ton of "announcements" around self driving that haven't worked out.

Audi abandons self-driving plans for current flagship

BMW, Mercedes-Benz end ‘long-term’ automated driving alliance, for now – TechCrunch.

Lyft’s autonomous vehicle partner Magna is done with self-driving tech

https://www.pymnts.com/news/partnerships-acquisitions/2019/volkswagen-aurora-self-driving-cars/


"X and Y announce they have an actual for-sale production that works" is big news.

 

[mikes_fsd]

Amazon to buy Zoox for $1.2 billion. Amazon to buy self-driving technology company Zoox

In 2018 during a cap raise they were valued at $3.2 billion.
Either Amazon is getting a killer deal, or Zoox's technology ain't worth as much as they pretend.
Probably somewhere in the middle.

 

[diplomat33]

Amazon to buy Zoox for $1.2 billion. Amazon to buy self-driving technology company Zoox

In 2018 during a cap raise they were valued at $3.2 billion.
Either Amazon is getting a killer deal, or Zoox's technology ain't worth as much as they pretend.
Probably somewhere in the middle.

It's a great move for Amazon IMO.

 

[Tikib2020]

With regard to the Volvo/Waymo deal I think both companies will be happy. From Waymo's point of view they might have wanted a bigger name than Volvo but at least they have a clear route to market now.

As for the Amazon/zoox deal, Amazon have picked up a company which were making SDC progress pretty quickly at a low price really good move for them. As well as delivery vehicles some of the navigation technology might be usable for vehicles/robots within their warehouses self driving forklift anyone?

 

[mikes_fsd]

I found this article interesting re Mercedes Benz partnership with Nvidia... Tesla Is Still the Car Company to Beat. Just Ask Mercedes.

This quote stood out in particular:

Morgan Stanley analyst Adam Jonas says Tesla is the only company “fully monetizing its autonomous driving assets at scale.” In other words, Tesla generates actual money from its internally developed self-driving solutions.