Karpathy is actually making an extremely important point here. Andrej and Elon’s shared views on this topic are a key driver of why Tesla’s Robotaxi AI strategy is so different from the competition.
Andrej and Elon acknowledge just how difficult it is to solve driving with AI and how much of a head-start human drivers have. In contrast Waymo and everyone else going with their Data Light, Hardware Heavy strategy are instead trivialising just how much learning it takes to become competent at driving and how much data it will take to catch up with humans.
Views on exactly how human and animal learning work still vary greatly, but i’d say most common in the AI community is Yan Lecun’s view: “If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning.”
Where broadly:
- Unsupervised Learning - Learning patterns and correlations between actions and consequences as you begin to interact with the world.
- Supervised Learning - Children asking questions and getting answers about the world.
- Reinforcement learning - Rewards for good behaviour or correct answers.
However this is only part of the story and potentially only a relatively small part.
A large amount of the behaviour of animals is not actually learned during the animal’s life but is driven by behaviour algorithms present at birth. For example animals are born with innate abilities such as Spiders ability to hunt from birth, or mice's inherited burrowing techniques. Many animals are also born with extremely effective unsupervised learning algorithms that allow them to learn specific tasks very quickly with only a few unsupervised examples.
There is obvious evolutionary pressure for animals to be born with abilities that aid survival or an ability to learn to recognise food and predators quickly.
These innate behaviours and learning abilities are learned via hundreds of millions of years of training via natural selection. Natural selection is effectively a reinforcement learning algorithm which is rewarded when an animal produces offspring. However, learning via natural selection is very inefficient and is limited by 1) Very little useful information is transmitted from animals life - we only know whether or not the animal survived long enough to produce offspring, 2) Very little information can be stored in the genome.
Reinforcement learning via natural selection takes place via genes that encode particular circuits and connections of neurons corresponding to particular innate behaviours plus architectures for extremely effective learning algorithms to allow continued learning through the animals life. However the human genome only contains 1 GB of data and there is not enough storage capacity for the exact weights and wiring of a brain’s neurons to be specified in the genome. Instead the genome has to specify a set of rules for how to wire the brain as it develops. This is possibly one of the key reasons humans have developed relatively general intelligence. Given the limited amount of information that can be stored in the genome, there is pressure to develop very general patterns and algorithms which can be used for multiple applications.
https://www.nature.com/articles/s41467-019-11786-6.pdf This is called the “genomic bottleneck” and potentially the very small size of the human genome has been a key driver of human intelligence - relative to for example the lungfish with a 40x larger genome and much less pressure to develop generalised algorithms.
To some extent this learning via natural selection can be thought of analogous to a more powerful form of pre-training or transfer learning used in many machine learning applications today.
It is important to note though that natural selection develops strong neural network architectures and wiring rules rather than optimising neural network weights.
There is some very interesting recent work on “weight agnostic” artificial neural networks, showing that if you carefully select the network architecture for a particular task, the neural network can actually perform some reinforcement learning tasks with fully randomised weights.
https://arxiv.org/pdf/1906.04358.pdf This shows just how powerful the network architecture can be itself even before you start training on data.
So back to Robotiaxis - when we are trying to teach a car how to drive, in reality we are competing with a human who already has 100s of millions of years of data and reinforcement learning via natural selection and 20+ years of Unsupervised Learning, Supervised Learning and Reinforcement Learning via interaction with the world and teachers.
This is why a Robotaxi will need 10s of billions of miles of real world driving experience vs a human who can learn to drive with 1,000 miles of real world driving lessons. And this is why Waymo and every other potential Robotaxi competitor is wrong when they assume they can learn enough just from a few hundred extremely expensive test vehicles and 10 million+ miles of real world
So Tesla's Robotaxi strategy is built from the assumptions:
1. We cannot solve Robotaxis without 10 billion+ miles of real world experience.
2. We cannot get 10 billion+ miles of real world experience without a hardware suite affordable to install in a normal consumer owned car.
3. We cannot get Lidar this cheap in a reasonable timeframe and without the economies of scale of first having a functioning Robotaxi business.
4. Therefore we cannot use Lidar.
5. Hence we have to solve distance and velocity estimates using machine learning with Cameras and Radar data. If we can do this, Lidar has no extra value anyway as its capabilities will only be a subset of what we can already do with Machine Vision.
...)
