Waymo acquires Latent Logic to improve its simulation with imitation learning

[strangecosmos2]

Waymo on Twitter:

“Today, Latent Logic becomes part of Waymo. Their uniquely talented team, based in Oxford, UK, uses imitation learning to simulate realistic models of human behavior on the road—key to developing safe self-driving vehicles.”​

From TechCrunch:

“...Latent Logic could help Waymo make its simulation more realistic by using a form of machine learning called imitation learning.​

Imitation learning models human behavior of motorists, cyclists and pedestrians. The idea is that by modeling the mistakes and imperfect driving of humans, the simulation will become more realistic and theoretically improve Waymo’s behavior prediction and planning.”
​

One of Latent Logic’s research papers focused on how to create realistic, human-like driving using traffic cameras observing roundabouts.

Imitation learning isn’t a brand new idea for Waymo. In December 2018, Waymo released a paper on ChauffeurNet, a neural network trained via imitation learning to drive in urban settings.

In February 2019, Waymo’s head of research, Drago Anguelov, gave a guest lecture for Lex Fridman’s MIT course on deep learning for self-driving cars. Anguelov stressed the importance of imitation learning for prediction, simulation, and planning:

Several other self-driving companies are doing work on imitation learning. That includes Cruise, Aurora, Comma AI, Ghost, and Wayve.

Tesla is using imitation learning and has already deployed imitation learned elements in Autopilot:

In an interesting proof of concept for imitation learning, DeepMind recently showed that a purely imitation learned agent can achieve a Diamond ranking on the StarCraft II ladder, performing better than over 80% of human players. By contrast, an agent trained via pure reinforcement learning performed worse than over 99% of humans; it didn’t learn how to play the game properly. The combination of IL and RL yielded an agent better than over 99.8% of humans.

 

[diplomat33]

Waymo on Twitter:

“Today, Latent Logic becomes part of Waymo. Their uniquely talented team, based in Oxford, UK, uses imitation learning to simulate realistic models of human behavior on the road—key to developing safe self-driving vehicles.”​

From TechCrunch:

“...Latent Logic could help Waymo make its simulation more realistic by using a form of machine learning called imitation learning.​

Imitation learning models human behavior of motorists, cyclists and pedestrians. The idea is that by modeling the mistakes and imperfect driving of humans, the simulation will become more realistic and theoretically improve Waymo’s behavior prediction and planning.”
​

One of Latent Logic’s research papers focused on how to create realistic, human-like driving using traffic cameras observing roundabouts.

Imitation learning isn’t a brand new idea for Waymo. In December 2018, Waymo released a paper on ChauffeurNet, a neural network trained via imitation learning to drive in urban settings.

In February 2019, Waymo’s head of research, Drago Anguelov, gave a guest lecture for Lex Fridman’s MIT course on deep learning for self-driving cars. Anguelov stressed the importance of imitation learning for prediction, simulation, and planning:

Several other self-driving companies are doing work on imitation learning. That includes Cruise, Aurora, Comma AI, Ghost, and Wayve.

Tesla is using imitation learning and has already deployed imitation learned elements in Autopilot:

Thanks for sharing. I find these threads you start highly informative and educational.

 

[strangecosmos2]

Thanks. Some folks have been skeptical of imitation learning for self-driving cars, sometimes favouring reinforcement learning. But if you want to use reinforcement learning, you’re probably way better off bootstrapping it with imitation learning. Here’s what Alex Irpan (an engineer at Google Brain) says:

“The fact that imitation learning gives a good baseline seems important for bootstrapping learning. It’s true that AlphaZero was able to avoid this, but AlphaGo with imitation learning bootstrapping was developed first. There usually aren’t reasons to discard warm-starting from a good base policy, unless you’re deliberately doing it as a research challenge.”
​

So, imitation learning can’t hurt and will probably help. There are also three potential problems with using reinforcement learning without first bootstrapping with imitation learning:

1. Pure reinforcement learning has done super well with some tasks like Go and Dota 2, but completely failed at others like StarCraft.​

2. Moreover, one problem for robotics tasks like self-driving that isn’t a problem for digital games is the “reality gap” problem for driving simulations. Games are perfect simulators of their own physics and environments (tautologically).​

3. Thirdly, in competitive games like Go, Dota, and StarCraft, there is a clear, built-in reward signal: either you win or lose. That means self-play, in which a reinforcement learned agent plays against versions of itself, can be used effectively for some games, but it seems less clear how to apply self-play in a driving simulator setting. Driving doesn’t have built-in victory and defeat conditions; the reward signal would have to be hand-designed by human engineers.​

In a game like Go, Dota, or StarCraft, an “alien”, bizarre form of play is not a problem, as long as it’s effective at achieving the end goal of winning the game. It’s fascinating for human players to see new techniques they haven’t thought of. But in driving, behaviour that is human-like and, therefore, predictable to human drivers is actually part of the end goal. It’s hard for me to imagine how hand-designing a good reward signal for self-play doesn’t boil down to, essentially, either a) hand-designing a driving policy or b) imitation learning (e.g. inverse reinforcement learning or reward learning from demonstrations).​

 

[SandiaGrunt]

Thanks for sharing. I find these threads you start highly informative and educational.

He mostly just copies and pastes stuff he finds on other websites.

When he editorializes, he always gets a bunch of stuff wrong. Sometimes it’s subtle, but it’s often obvious to a knowledgeable reader. This is a problem. He spreads lots of misinformation mixed in with his news recaps. It can be difficult to tell which parts of his statements are facts, and which parts are his conjectures. This is a problem.

You would do better to just add “self-driving cars” as a category on Google News, and read the primary sources yourself.

 

[diplomat33]

He mostly just copies and pastes stuff he finds on other websites.

When he editorializes, he always gets a bunch of stuff wrong. Sometimes it’s subtle, but it’s often obvious to a knowledgeable reader. This is a problem. He spreads lots of misinformation mixed in with his news recaps. It can be difficult to tell which parts of his statements are facts, and which parts are his conjectures. This is a problem.

You would do better to just add “self-driving cars” as a category on Google News, and read the primary sources yourself.

I do read my own articles from primary sources. I am not focused too much on the editorial parts. I focus more on the stuff he copies and pastes from the web.

 

[SandiaGrunt]

I do read my own articles from primary sources. I am not focused too much on the editorial parts. I focus more on the stuff he copies and pastes from the web.

As I said, the problem is that he freely mixes facts with his own conjectures. It’s very difficult to tell them apart, and he makes no effort to distinguish them.

Further, I think he actually intends to present his conjectures as facts. He craves an audience more than anything in the world, and to be seen as an expert in a field in which he has no education or experience. This desire leads him to dress up his conjectures to appear as facts.

Even further, I fear that he may actually believe his conjectures are facts, which is a serious behavioral problem.

 

[diplomat33]

As I said, the problem is that he freely mixes facts with his own conjectures. It’s very difficult to tell them apart, and he makes no effort to distinguish them.

Further, I think he actually intends to present his conjectures as facts. He craves an audience more than anything in the world, and to be seen as an expert in a field in which he has no education or experience. This desire leads him to dress up his conjectures to appear as facts.

Even further, I fear that he may actually believe his conjectures are facts, which is a serious behavioral problem.

Since you are an autonomous driving expert, feel free to share your knowledge by starting your own thread on a topic or provide factual information to set the record straight if you believe that @Trent Eady has said something that is incorrect. Sharing ideas and information is what a forum should be about.

 

[SandiaGrunt]

Since you are an autonomous driving expert, feel free to share your knowledge by starting your own thread on a topic or provide factual information to set the record straight if you believe that @Trent Eady has said something that is incorrect. Sharing ideas and information is what a forum should be about.

I wish it were so simple. When I have challenged his conjectures here, on seekingalpha, etc. he just begs a moderator to protect him, to enable him, by having the challenging posts removed.

He does not want open discussion — he wants an adoring audience, and an environment he can control. This is why he made his own forum, and spent months trying to siphon people away from this forum.

This is clear from his “thesis” — it has not changed at all over the span of years, despite hundreds of people trying to correct him. He is still just saying the same things over and over, trying each time to impress new people with the same tired arguments.

He’s a crackpot, and this is all typical crackpot behavior.

 

[strangecosmos2]

provide factual information to set the record straight if you believe that @Trent Eady has said something that is incorrect. Sharing ideas and information is what a forum should be about.

I welcome (civil, respectful) critique of my ideas and I of course welcome factual corrections if I’ve gotten something wrong. I make mistakes. I’ve been wrong in the past. I’m not an expert on this topic (just an enthusiast like most folks here) and, well, even experts are fallible. Humans are fallible. There have been people on this forum, on Twitter, and elsewhere who have changed my mind about things in a way that’s polite and not personally attacking. I’m grateful to them for that.

 

[SandiaGrunt]

Alright @Trent Eady put your money where your mouth is. Good luck, champ.

Trent Eady’s Misuse of Baidu’s Research

 

[Doug_G]

@SandiaGrunt @Trent Eady

We expect participants in this forum to behave politely and refrain from personal attacks.

It is one thing to criticize someone's arguments or positions on topics. It is another thing entirely to go after them personally. That is against the terms of service of TMC.

-mod

 

[strangecosmos2]

I've responded to some critiques here.