Do you mean Tesla’s FSD product, or the general concept of self-driving cars?
There is a lot that Tesla’s FSD product doesn’t do now. It is quite limited today, and it will get less limited over time.
In principle, I don’t see why self-driving cars won’t be able to do complex negotiations with other vehicles, like the ones you mentioned. This might require replacing the hand-coded approach to driving policy and path planning (i.e. the actions that the car takes) — which has been the norm for the past 15 years — with an
imitation learning and/or reinforcement learning approach. It might take years to get it right. Maybe many years. It might require innovations in the imitation learning and/or reinforcement learning state of the art. But, in principle, I don’t see why these will always be things self-driving cars can’t do — even assuming no major, fundamental breakthroughs in artificial intelligence.
This is just a demo and might have been cherry-picked out of a hundred failed attempts, but it’s still interesting:
ytCropper | Prof. Amnon Shashua at 2018 Intel Capital Global Summit
At least 1 successful attempt gives us more information than 0 successful attempts.
People disagree on whether what Tesla’s FSD product can do will ultimately converge on what self-driving cars in general can do. I think they will converge.
Why do people disagree? One big reason is lidar. But as best I can tell, cameras can do everything lidar can do, and lidar can’t do everything cameras can do.
Mobileye and
Anthony Levandowski (once an important engineer at Waymo) agree lidar isn’t necessary.
Another big reason is that people think that while perception can only be solved by neural networks, action can be solved by hand coding. Whoever has the best programmers, or whoever has the codebase that has been worked on the longest — that’s who is in the lead. I’m skeptical that hand coding can solve complex multi-agent interactions (like negotiating with other cars) or similarly hard real world problems. I’m not aware of any example where it has in the past. It also seems a lot like perception— humans see, but we can’t write down a computer program that makes a computer see like we do. It’s too subtle and complicated and on the level of implicit knowledge. Humans drive, but that doesn’t mean we can write down how we do it. (Can you describe exactly how to ride a bike, so that a robot could follow your instructions?)
A third big reason is that people think whoever makes the best neural networks will have the best self-driving car. But this ignores the importance of training data. Say we put on a contest between some grad students and the best machine learning people in the world. They are competing to correctly classify images from the ImageNet test set. The grad students get 100% of the ImageNet training dataset. The best machine learning people only get 1% of the images from each category. The grad students would win, hands down.
HW2 Teslas drive over 12 million miles per day. Waymos drive about 12 million miles per year. When it comes to training data that doesn’t need to be labelled by humans, and especially data that (unlike raw sensor data) is easy to store and transmit a lot of, Tesla has a 100x advantage. What the exact consequences will be for neural network performance, no one can say— this has never been tried before. Machine learning isn’t yet a design science where we can predict the performance of a system based on its components. The only strong theoretical precept we have is that more data is better. Sometimes it’s only marginally better, and sometimes it’s radically better.
Tesla also has the ability to recruit machine learning people who aren’t just grad students. It’s one of the
most desirable companies in the world for people who work in tech. Although he didn’t stay long, the fact that Tesla could pull Chris Lattner from Apple shows its allure. So at Tesla it’s not just grad students with an 100x data advantage vs. elite machine learning people. It’s elite machine learning people with an 100x data advantage vs. other elite machine learning people.
A lot of the world’s best machine learning people publish their research in academic papers, and sometimes even put the code on GitHub. Not just people in academia, but also people at OpenAI, DeepMind, Google, and Facebook. A lot of ideas are shared freely. Tesla doesn’t need all of the best people working under its roof to use some of their ideas.
