Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register

MIT’s Deep Learning for Self-Driving Cars: 2019 edition!

This site may earn commission on affiliate links.
The first lecture of the 2019 edition of the course is now up!


Deep Learning for Self-Driving Cars is an MIT course that Lex Fridman has taught since 2017. Real MIT students take the course and get credit. But the lectures are posted for free on YouTube, and anyone can join the course’s Slack chat group and get help from other students.

The lectures for past years are posted on the course website: MIT Deep Learning
 
Last edited:
  • Informative
Reactions: voyager
Lex says that beyond perception, there is very little machine learning in autonomous cars today. But that is starting to change:
  • Tesla is reportedly using imitation learning for autonomous driving.
  • Mobileye is openly using reinforcement learning for autonomous driving.
  • Waymo says it may incorporate components of an imitation learning system into its autonomous driving software.
  • Anthony Levandowski — a famous self-driving car engineer who formerly worked at Waymo, Otto, and Uber ATG, and who competed in the DARPA Grand Challenge — recently announced a new startup called Pronto and said it will use “end-to-end deep learning”. That could mean imitation learning or reinforcement learning. Or both.
An honourable mention to Wayve, a small startup in the UK founded by Cambridge machine learning researchers. Wayve’s CEO says: “Rather than hand-engineering our solution with heavily rule-based systems, we aim to build data-driven machine learning at every layer of our system, which would learn from experience and not simply be given if-else statements.” Wayve’s website mentions reinforcement learning.

I feel pessimistic about the hand-coded, rule-based approach to driving. (That is, the non-perceptual, action-related parts of driving.) There are a few reasons:
  1. It hasn’t seemed to work all that great for much of anything — not computer vision, getting bipedal robots to walk or open doors, board games, video games... Is there an example of it achieving human-level performance on any complex task?
  2. A beaver can build a dam, but it has no idea how to tell you to build a dam. Similarly, many tasks that humans perform easily, effortlessly, mindlessly — we don’t actually know how we do them, and we don’t know how to tell a robot to do them. We might need a better scientific understanding of how humans drive before we can get robots to do it. The introspection of software engineers might not do the trick.
  3. If we are stuck with hand coding robots, I worry that engineers will continue to gradually pluck away at the problem, inching ahead year by year. Only making as much progress this year as they made last year. There seems to be a wide chasm between today’s robot drivers and human drivers. Crossing that chasm inch by inch seems like it would take quite a while. To get across that chasm in a few years, it seems like we need progress to move a lot faster.
In sum, I worry that hand coding will only make slow linear progress, and it may at some point hit a ceiling where engineers just don’t know how to solve the next problems.

By contrast, machine learning has shown us a few examples of fast exponential progress, where it went from subhuman performance to superhuman performance in a few years. ImageNet, AlphaGo, maybe Dota.

If all these companies try various machine learning approaches to driving for a few years, and they don’t get any traction... I will feel pretty pessimistic about self-driving cars. If that happens, I think I might feel that the problem can’t be solved with the current machine learning paradigm, and that hand coding is unlikely to solve it either. So self-driving cars would be indefinitely on hold. Instead of being an engineering problem, self-driving cars would become (in my eyes) a science problem.

That’s a bleak place to be. Scientific progress in AI has happened in fits and starts. Prior to 2012, there was a long period of stagnation.

So, as a fan of self-driving cars — or the idea of self-driving cars — I am watching imitation learning and reinforcement learning because I think one or both of those techniques could be the key to all of it.
 
Last edited:
25min into was pretty cute, I think Tesla have developed something similar for their classification workforce.

Yeah, I thought the segmentation annotating thing was really cool. Such a time saver vs. doing the whole thing manually from scratch.

The Information reported that Tesla is working with an outside company that specializes in data labelling for machine learning. There are a few such companies out there that specifically do labelling for autonomous vehicles, like Mighty AI.
 
Last edited:
Lex says that beyond perception, there is very little machine learning in autonomous cars today. But that is starting to change:
  • Tesla is reportedly using imitation learning for autonomous driving.
  • Mobileye is openly using reinforcement learning for autonomous driving.
  • Waymo says it may incorporate components of an imitation learning system into its autonomous driving software.
  • Anthony Levandowski — a famous self-driving car engineer who formerly worked at Waymo, Otto, and Uber ATG, and who competed in the DARPA Grand Challenge — recently announced a new startup called Pronto and said it will use “end-to-end deep learning”. That could mean imitation learning or reinforcement learning. Or both.
An honourable mention to Wayve, a small startup in the UK founded by Cambridge machine learning researchers. Wayve’s CEO says: “Rather than hand-engineering our solution with heavily rule-based systems, we aim to build data-driven machine learning at every layer of our system, which would learn from experience and not simply be given if-else statements.” Wayve’s website mentions reinforcement learning.

I feel pessimistic about the hand-coded, rule-based approach to driving. (That is, the non-perceptual, action-related parts of driving.) There are a few reasons:
  1. It hasn’t seemed to work all that great for much of anything — not computer vision, getting bipedal robots to walk or open doors, board games, video games... Is there an example of it achieving human-level performance on any complex task?
  2. A beaver can build a dam, but it has no idea how to tell you to build a dam. Similarly, many tasks that humans perform easily, effortlessly, mindlessly — we don’t actually know how we do them, and we don’t know how to tell a robot to do them. We might need a better scientific understanding of how humans drive before we can get robots to do it. The introspection of software engineers might not do the trick.
  3. If we are stuck with hand coding robots, I worry that engineers will continue to gradually pluck away at the problem, inching ahead year by year. Only making as much progress this year as they made last year. There seems to be a wide chasm between today’s robot drivers and human drivers. Crossing that chasm inch by inch seems like it would take quite a while. To get across that chasm in a few years, it seems like we need progress to move a lot faster.
In sum, I worry that hand coding will only make slow linear progress, and it may at some point hit a ceiling where engineers just don’t know how to solve the next problems.

By contrast, machine learning has shown us a few examples of fast exponential progress, where it went from subhuman performance to superhuman performance in a few years. ImageNet, AlphaGo, maybe Dota.

If all these companies try various machine learning approaches to driving for a few years, and they don’t get any traction... I will feel pretty pessimistic about self-driving cars. If that happens, I think I might feel that the problem can’t be solved with the current machine learning paradigm, and that hand coding is unlikely to solve it either. So self-driving cars would be indefinitely on hold. Instead of being an engineering problem, self-driving cars would become (in my eyes) a science problem.

That’s a bleak place to be. Scientific progress in AI has happened in fits and starts. Prior to 2012, there was a long period of stagnation.

So, as a fan of self-driving cars — or the idea of self-driving cars — I am watching imitation learning and reinforcement learning because I think one or both of those techniques could be the key to all of it.
Have you watched Drago’s lecture yet? He gives a very good reason for using a hybrid approach with ML and rules based.
 
Have you watched Drago’s lecture yet? He gives a very good reason for using a hybrid approach with ML and rules based.

What a yummy talk! I loved it!

Drago seems to think the same way as Karpathy. There is an ML portion of the software stack, and a non-ML portion. The goal is to expand the ML portion of the stack, so that over time it takes over more and more responsibilities from the non-ML portion. It is comforting that they seem to agree on that.

I loved the discussion of supervised imitation learning (ChauffeurNet) and inverse reinforcement learning (trajectory optimization agent). When Lex Fridman asked Pieter Abbeel, an imitation learning and reinforcement learning expert, about imitation learning for self-driving cars, he recommended inverse reinforcement learning over supervised imitation learning. It was really cool to see Drago give a comparison of both approaches.

I found it really interesting how he talked about populating simulations with a zoo (or menagerie) of agents. Including recorded “ghosts” that don’t react at all, agents that use a simple brake and swerve algorithm (makes me think of cars in GTA V), and multiple varieties of complex machine learned agents. A mix of different kinds of agents will increase simulation diversity, and that could make simulation training generalize better to the real world. This is similar to the idea of simulation randomization, which can help compensate for the “reality gap” between simulation and reality, except that the differences between e.g. supervised imitation learned agents and inverse reinforcement learned agents isn’t random.

The discussion of training examples and the long tail of driving scenarios seems the most relevant to Tesla. Suppose there’s a scenario that occurs every 10 million miles. If Waymo’s fleet drives 10 million miles per year, it will encounter 1 training example per year. If Tesla’s fleet drives 10 billion miles per year, it will encounter 1,000 training examples per year. A scenario that occurs every 100 million miles: 1 per decade for Waymo, 100 per year for Tesla. Tesla can train its neural networks on a much longer tail of scenarios than Waymo. Where Waymo has to use hand-coded algorithms due to a lack of training examples, Tesla can use a machine learning approach.

Drago seemed to say that the main (only?) constraint to applying machine learning in the areas where Waymo currently uses hand-coded algorithms is the difficulty of collecting training data. If this is simply a question of cars on the road, then this supports the thesis that Tesla has an advantage in applying machine learning to autonomous driving.

Alphabet probably has an advantage with regard to neural network architectures, since Google and DeepMind seem pretty good at designing them. But if you don’t have the data to train your neural networks on, then it doesn’t matter.
 
Last edited:
  • Like
  • Informative
Reactions: AndyH and kbM3
What a yummy talk! I loved it!

Drago seems to think the same way as Karpathy. There is an ML portion of the software stack, and a non-ML portion. The goal is to expand the ML portion of the stack, so that over time it takes over more and more responsibilities from the non-ML portion. It is comforting that they seem to agree on that.

I loved the discussion of supervised imitation learning (ChauffeurNet) and inverse reinforcement learning (trajectory optimization agent). When Lex Fridman asked Pieter Abbeel, an imitation learning and reinforcement learning expert, about imitation learning for self-driving cars, he recommended inverse reinforcement learning over supervised imitation learning. It was really cool to see Drago give a comparison of both approaches.

I found it really interesting how he talked about populating simulations with a zoo (or menagerie) of agents. Including recorded “ghosts” that don’t react at all, agents that use a simple brake and swerve algorithm (makes me think of cars in GTA V), and multiple varieties of complex machine learned agents. A mix of different kinds of agents will increase simulation diversity, and that could make simulation training generalize better to the real world. This is similar to the idea of simulation randomization, which can help compensate for the “reality gap” between simulation and reality, except that the differences between e.g. supervised imitation learned agents and inverse reinforcement learned agents isn’t random.

The discussion of training examples and the long tail of driving scenarios seems the most relevant to Tesla. Suppose there’s a scenario that occurs every 10 million miles. If Waymo’s fleet drives 10 million miles per year, it will encounter 1 training example per year. If Tesla’s fleet drives 10 billion miles per year, it will encounter 1,000 training examples per year. A scenario that occurs every 100 million miles: 1 per decade for Waymo, 100 per year for Tesla. Tesla can train its neural networks on a much longer tail of scenarios than Waymo. Where Waymo has to use hand-coded algorithms due to a lack of training examples, Tesla can use a machine learning approach.

Drago seemed to say that the main (only?) constraint to applying machine learning in the areas where Waymo currently uses hand-coded algorithms is the difficulty of collecting training data. If this is simply a question of cars on the road, then this supports the thesis that Tesla has an advantage in applying machine learning to autonomous driving.

Alphabet probably has an advantage with regard to neural network architectures, since Google and DeepMind seem pretty good at designing them. But if you don’t have the data to train your neural networks on, then it doesn’t matter.

One thing I haven’t seen discussed is the difficulty of classifying an obstacle in the road along a spectrum of how safe to hit (a plastic bag) vs. how unsafe (a boulder). It seems like there are so many 1000’s of possible objects (the long tail), I don’t see how this can be trained. It is a not too uncommon problem that our human intuition solves fairly easily. I’m guessing they’ll just classify the most likely and swerve or brake if it’s not classified?

Have you heard any ideas how autonomy can handle this?
 
  • Helpful
Reactions: strangecosmos
I’m not sure if neural networks can reliably detect objects they can’t classify. I know there’s been work on novelty detection, but I don’t know if it’s a production-ready technique. Lidar is helpful for this because it can detect novel solid objects. So can radar, but not as well.

I think whether this is a big problem or a small problem depends how diverse the debris on the road actually is. I wonder if this is the sort of thing where 90% of the debris on the road is the same 20 object types — e.g. plastic bags, plastic bottles, cans, newspapers, tires, lumber, cement blocks — and 99% are the same 200 objects types. And 99.9% are the same 2,000 types.

Autonomous cars might just be worse than humans when it comes to hitting incredibly rare objects. Objects that fall into the 0.1% of road debris. But if they are better at avoiding collisions with incredibly common objects — vehicles, pedestrians, cyclists, animals, etc. — then the overall collision rate could be much lower. That might just be the trade-off. Better at avoiding people, cars, and animals, worse at avoiding runaway watermelon carts and dropped arcade machines.

I’m trying to think what’s the weirdest object I’ve ever encountered on the road. Maybe it was the fluff from the fur of some animal. But that was harmless to drive over. How common is it to encounter debris that needs to be avoided? And how often does this debris fall outside the same few dozen categories?
 
Last edited:
  • Like
Reactions: kbM3
I’m not sure if neural networks can reliably detect objects they can’t classify. I know there’s been work on novelty detection, but I don’t know if it’s a production-ready technique. Lidar is helpful for this because it can detect novel solid objects. So can radar, but not as well.

I think whether this is a big problem or a small problem depends how diverse the debris on the road actually is. I wonder if this is the sort of thing where 90% of the debris on the road is the same 20 object types — e.g. plastic bags, plastic bottles, cans, newspapers, tires, lumber, cement blocks — and 99% are the same 200 objects types. And 99.9% are the same 2,000 types.

Autonomous cars might just be worse than humans when it comes to hitting incredibly rare objects. Objects that fall into the 0.1% of road debris. But if they are better at avoiding collisions with incredibly common objects — vehicles, pedestrians, cyclists, animals, etc. — then the overall collision rate could be much lower. That might just be the trade-off. Better at avoiding people, cars, and animals, worse at avoiding runaway watermelon carts and dropped arcade machines.

I’m trying to think what’s the weirdest object I’ve ever encountered on the road. Maybe it was the fluff from the fur of some animal. But that was harmless to drive over. How common is it to encounter debris that needs to be avoided? And how often does this debris fall outside the same few dozen categories?

What about if a section of the route ahead fails the "pavement" match confidence instead? Size determines drive over vs around/stop.

'I don't know what it is, but it's something and it don't look drivable '
Or, for space freighters:
'That's no moon'
 
I’m not sure if neural networks can reliably detect objects they can’t classify. I know there’s been work on novelty detection, but I don’t know if it’s a production-ready technique. Lidar is helpful for this because it can detect novel solid objects. So can radar, but not as well.

Every object is classified, some as 0% cat, some as 100% cat. If the class is “obstacle” then asphalt might score 0%, a plastic bag 20%, concrete block 100%. It’s all up to labelers so decide what is obstacle and get enough data set. Then the network will do its magic. After that it is up to the software to decide at what threshhold they should take what action. Maybe just use imitation learning instead, brake for the ones where it think humans would brake and keep driving for the objects it thinks a human would drive through. Just use the obstacle class for deciding which data to use to train brake or not on.