Can someone please explain in simple words, what is the function of Dojo? How does Dojo help solve FSD? I keep hearing 'labelling', but I don't quite understand what that means. How are others solving this problem?
This is how I understood :
............. video ...................... labels
Cars ----------> Dojo -----------> Cars <cars can drive better now>
I'm going to give this a shot. And perhaps point out the foibles of Neural Networks in the process.
So: A Neural Net. Imagine a thousand neurons, hooked up to a bunch of nodes, maybe fifty or so. Each node has weights assigned to the 20 or so neurons attached to it; multiple a weight times the analog input on each of those twenty or so neurons; if one goes above a threshold, then
that node will fire, stimulating all the neurons leaving that node.
So, of the twenty or so nodes in this example, have all their twenty outputs go to a summary node.
Now - attach pixels from multiplicity of pictures with a giraffe in the picture. Giraffes standing, running from lions, glimpsed in the distance, right in front of one, inbetween trees, and so on. Have the final, output node, be "Giraffe or No Giraffe".
In addition, zillions of pictures with no giraffe in evidence.
Now, run through all those pictures. Each time one has a giraffe in evidence
and the giraffe bit is set, do some complicated math and increase the weights on the neurons and intermediate neurons that seemed to have something to do with the correct detection. Each time one has a giraffe in evidence but no giraffe is detected, reduce weights with math; i.e., wrong answer time.
Each time one shows a picture with no giraffe and a giraffe is detected, do the wrong answer algorithm. Each time one shows a picture with no giraffe and no giraffe is detected, correct answer algorithm.
Did I mention feedback? There's feedback.
This kind of a system gets really, really good at detecting giraffes. People who have done this kind of work discover that, when shown a picture that doesn't nominally have a giraffe in it, and the giraffe detector goes off, if they take a careful look - and yep, there's a giraffe, 90% hidden by leaves.
Now, make it more complicated: Lots of outputs, lots of intermediate nodes, lots of cross-connections. The outputs are every animal in the savanna. And, weights and all.. It works. In fact, it kind of works, and works well: Think about how well
people pick up creatures in the savanna - like that lion trying to hide in the grass. (And those ancestors of ours whose NN's didn't work up to snuff ended up winning Darwin awards and aren't around any more to complain.)
Now, for the nasty part. As one might expect for a system that's designed to work on a hair trigger and works on variable weights, to boot, training up such a NN means that if one isn't careful, one might train such a NN up for the
wrong thing. Suppose that one is attempting to train a NN to play poker. All well and good; but if one puts a deck of marked cards into play and lets the NN lean from lots and lots of pictures of people playing poker, one might end up, instead of a NN that can catch the "Tells" that professional poker players look for, a NN might get really, really good at figuring out how to play when it
knows what the cards are in the other players hands! It might do pretty well in that kind of environment: And fail hugely when it has to play with unmarked decks!
So, what's the Dojo compute platform doing? Near as I can tell from the AI day, it's a platform that generates an environment that's absolutely, positively realistic, right down to the hair on the back of the dog. It generates this environment from a combination of real pictures, maps, labels on other moving things, rainstorms, you name it. Have a semi go around the bend, in a realistic way. Or a Cybertruck drive through the intersection. Or have a family running down the freeway. Why do this? Because it's like buzzards on a tree waiting for something to die: You know that there's a particular corner case, because you either thought of one or saw one that was close: Now you want to run a zillion iterations against your NN for every possible version of this corner case, in all weathers, with all vehicles, in all possible road intersections and curves.
And it won't just be
this corner case. It'll be the whole panoply of corner cases, regular cases, medium cases, and so on. Millions or even billions of iterations, training up the NN processor: Image recognition, object detection, 3-d physics, and so on. It's a heck of a lot faster to do this than to try and get people and video cameras to do it for you. By orders of magnitude.
And
that's what the Dojo is doing. It's a real Dojo - a place where one can practice, and practice, and practice, over and over, so, when one is finally let out into the real world, one hopes one has the wherewithal to succeed.
In this environment, the Dojo internal display to the NN has to be balls-on accurate; if not, one is going to run into the marked cards problem and end up training it for something that's an artifact of the simulation itself.
And there you are.
