Real World AI = AI that is acting in the physical world, interacting with humans and nature. In contrast with games, simulation etc where the world is very controlled, less complex and it costs less to make mistakes.
Tesla has solved how to solve real world AI problems. To do this you need
1. A method of growing a dataset with useful edge cases to converge on high performance (software 2.0 stack, data engine, labelers etc)
2. Hardware to train the neural network (Dojo)
3. Hardware to run the neural network( HW3/HW4 etc)
What Tesla has done is set up a complete system for this, developing the entire software 2.0 stack.
Why are Tesla not ready with FSD yet? Because it turned out that solving FSD was a lot harder than we intially thought.
Google had driven 300 000 autonomous miles around San Francisco in 2012. They have had a lot of skilled engineer, led by the ”Probabilistic Robotics” author Sebastian Thrun, having had plenty of hardware infrastructure and grown the team a lot since then. Yet for 9 years of intense work by skilled engineers they are still not ready. Because it was a lot harder than they thought.
In comes Tesla. They had the best autopilot on the streets, but in 2016 they split from Mobileye. The split was really messy, they had to reimplement entire Mobileye intellectual property from scratch, which set them back, but they did this very quickly. But already in 2016 they had a FSD car up and driving for the demo ”Paint it black”. Clearly they had a pretty great team to be able to do this. Since then it has been mostly delays and minor releases, because it was harder than they thought.
Here comes Elon. Like Google and like Tesla he is really frustrated that progress is not happening as fast as he wants. He tries to understand the problem. He kicks out Chris Lattner, the top lead from Apple who made LLVM and Swift, clearly a really good project leader, because he realizes that it’s not enough to have a team of great developers, like Google and Tesla had. There needs to be a totally different approach, it’s needed to solve ”Real World AI”. What is needed is three components, the entire data engine software stack, see Karpathy software 2.0, dedicated hardware, see Jim Killer in Lex Fridman both for training(Dojo) and for inference(HW4) see Pete Bannon in Autonomy day.
With this approach, there is a clear path with constant improvements towards the end goal. The only way that we know that progress can be made on what the elite teams at Waymo and Tesla has struggled so much with over the last five years.
It might seem that Tesla are constantly missing goals and that no progress is happening. But finally all the pieces are almost into place and we will start seeing the march towards 99.9999%. And consider, if the elite teams at Tesla and Waymo(who managed to do so much from 2009-2016 and have since then gotten so much more resources) have struggled with making progress, how hard will it be for VW, Toyota etc to make progress?
Relevant videos: