Project Dojo

[diplomat33]

Musk tweets cryptic reference to Dojo:

 

[boonedocks]

Musk tweets cryptic reference to Dojo:

View attachment 437895

Musk is using Jedi mind tricks

Definition of WILL

: to cause or change by an act of will

 

[EVNow]

I think this is what George Hotz talked about in the AI podcast. He talked about driving without first recognizing the lanes. That is what the nvidea blog talks about too.

End-to-End Deep Learning for Self-Driving Cars

The system is trained to automatically learn the internal representations of necessary processing steps, such as detecting useful road features, with only the human steering angle as the training signal. We never explicitly trained it to detect, for example, the outline of roads. In contrast to methods using explicit decomposition of the problem, such as lane marking detection, path planning, and control, our end-to-end system optimizes all processing steps simultaneously.
​

One way to think about Musk's derided comment about robotaxis next year is he thinks this end-to-end training of NN to drive the car will be complete by next year. We have no idea how far they are with respect to DoJo - and how long it will take to train the NN end to end to be as good or better than humans.

It makes complete sense that Musk would try something like this - and believe it will work and make "outlandish" FSD predictions.

 

[diplomat33]

I think this is what George Hotz talked about in the AI podcast. He talked about driving without first recognizing the lanes. That is what the nvidea blog talks about too.

End-to-End Deep Learning for Self-Driving Cars

The system is trained to automatically learn the internal representations of necessary processing steps, such as detecting useful road features, with only the human steering angle as the training signal. We never explicitly trained it to detect, for example, the outline of roads. In contrast to methods using explicit decomposition of the problem, such as lane marking detection, path planning, and control, our end-to-end system optimizes all processing steps simultaneously.
​

One way to think about Musk's derided comment about robotaxis next year is he thinks this end-to-end training of NN to drive the car will be complete by next year. We have no idea how far they are with respect to DoJo - and how long it will take to train the NN end to end to be as good or better than humans.

It makes complete sense that Musk would try something like this - and believe it will work and make "outlandish" FSD predictions.

What if Musk is wrong about the timeline but right about the process? Personally, I think that is likely. I could see Elon being late on his deadlines (he's already missed quite a few) but end-to-end deep learning actually does pan out in the end and actually does eventually deliver Full Self-Driving.

 

[Eno Deb]

I think this is what George Hotz talked about in the AI podcast. He talked about driving without first recognizing the lanes. That is what the nvidea blog talks about too.

End-to-End Deep Learning for Self-Driving Cars

The idea is of course very attractive (since it prevents losses of information that happen when your model reduces the output to a specific representation in an interim step), but there are still fundamental issues to be solved. Note that the article is 3 years old, and nobody (including Nvidia) is close to this.

 

[EVNow]

What if Musk is wrong about the timeline but right about the process? Personally, I think that is likely. I could see Elon being late on his deadlines (he's already missed quite a few) but end-to-end deep learning actually does pan out in the end and actually does eventually deliver Full Self-Driving.

Possible - and it definitely makes sense to try it out. But clearly, Tesla is also trying the alternate "conventional" approach of doing some stuff in NN and rest in Heuristics. Karpathy said during the autonomy day something along the lines of - somethings are better done using Heuristics. So, I don't know what is the primary play and what is the hedge.

But I think it is quite clear that conventional approach will not yield robotaxi in 2020. Just looking at the laborious work involved getting each task in NN to work (from Karpathy's multi-task talk) shows it s not a quick solution. So, only *possible* way to get to L5 in just a year is if DoJo suddenly works and becomes very good soon after.

AlphaGo and other projects took a long time to become barely able to play, but from that to beating the best humans took a short time. Many people have speculated from this (and some other NN) example that an end-to-end NN will take a long time to get to be able to barely drive - but from there to becoming the best driver in the world would only take a short time i.e. it improves exponentially, something Musk is fond of talking about all the time.

 

[theBurtReynold]

My hypothesis is that Dojo will roar into life only once Karpathy a team are “feature complete” with respect to all the things FSD should be able to do. Here’s what I think is up:

1. Dojo is a powerful compute resource for processing lots of video (and, likely, driver steering commands);
2. In the same manner that the “engine” collects / is fed images, Dojo will collect / be fed video (all 8 cameras) and steering commands;
3. Dojo will use the feature-complete NN to identify objects and build the “360-degree model of the world” for every video sequence;
4. For each frame, therefore, it’ll know the list of inputs (aka: objects + their relative positions and relative motion vectors) and the steering output (aka: what did the human do?);
5. End-to-end learning will then be performed — “the engine” can request similar initial object and vector conditions from events across the Fleet, the Fleet will upload qualifying video clips, and Dojo will infer the optimal steering outputs;
6. Rinse and repeat at scale

In this manner, Dojo wouldn’t train the vision NN, it would develop the steering system — with the “master” (human beings) training the computer — BUT the vision NN needs to be very mature until they’re ready to open the taps.

Also, this approach would differ from the Nvidia one people have linked to, because the Nvidia approach is direct pixel-to-steering. The approach I outlined has an intermediate step (object detection), so Dojo would be more like objects (and vectors)-to-steering.

Does this make sense to anyone who knows ML, etc.?

 

[DrDabbles]

My hypothesis is that Dojo will roar into life only once Karpathy a team are “feature complete” with respect to all the things FSD should be able to do. Here’s what I think is up:

1. Dojo is a powerful compute resource for processing lots of video (and, likely, driver steering commands);
2. In the same manner that the “engine” collects / is fed images, Dojo will collect / be fed video (all 8 cameras) and steering commands;
3. Dojo will use the feature-complete NN to identify objects and build the “360-degree model of the world” for every video sequence;
4. For each frame, therefore, it’ll know the list of inputs (aka: objects + their relative positions and relative motion vectors) and the steering output (aka: what did the human do?);
5. End-to-end learning will then be performed — “the engine” can request similar initial object and vector conditions from events across the Fleet, the Fleet will upload qualifying video clips, and Dojo will infer the optimal steering outputs;
6. Rinse and repeat at scale

In this manner, Dojo wouldn’t train the vision NN, it would develop the steering system — with the “master” (human beings) training the computer — BUT the vision NN needs to be very mature until they’re ready to open the taps.

Also, this approach would differ from the Nvidia one people have linked to, because the Nvidia approach is direct pixel-to-steering. The approach I outlined has an intermediate step (object detection), so Dojo would be more like objects (and vectors)-to-steering.

Does this make sense to anyone who knows ML, etc.?

I think people don't understand what Dojo is. It's self supervised machine learning. It's a hedge that self supervised machine learning can be achieved practically and with meaningful results. That's very much an area of debate among researchers. But it doesn't require the vehicles have a "feature complete" FSD system, it requires a breakthrough in the industry.

 

[electronblue]

There is a risk of gathering around one more hopeful buzzword.

It was once called the ”FSD codebase”.

Then ”HW3 hardware”.

Now ”Dojo”.

That magic thing that will turn the bleak reality we are witnessing into instant utopia.

The hope.

This is reminiscent of the pattern P85D HP went through. First a hopeful software update or two, then the Ludicrous hardware... and so it went until everyone — even Tesla — finally admitted defeat.

Let’s face it. There is a higher than zero chance there is no magic trick, no revolutionary move... just a few steps forward and a few steps back and so the dance goes at a snail’s pace. There seems to be an evolution, but a revolution is hardly guaranteed.

The open question is: Did Tesla lie at Autonomy Investor Day about realistic current status and trajectory when they said they are expecting Level 5 feature complete at the end of year — was that thinking contigent on an unrealistic Hail Mary pass?

It is very possible there is no Hail Mary pass.

 

[diplomat33]

There is a risk of gathering around one more hopeful buzzword.

It was once called the ”FSD codebase”.

Then ”HW3 hardware”.

Now ”Dojo”.

That magic thing that will turn the bleak reality we are witnessing into instant utopia.

The hope.

This is reminiscent of the pattern P85D HP went through. First a hopeful software update or two, then the Ludicrous hardware... and so it went until everyone — even Tesla — finally admitted defeat.

Let’s face it. There is a higher than zero chance there is no magic trick, no revolutionary move... just a few steps forward and a few steps back and so the dance goes at a snail’s pace. There seems to be an evolution, but a revolution is hardly guaranteed.

The open question is: Did Tesla lie at Autonomy Investor Day about realistic current status and trajectory when they said they are expecting Level 5 feature complete at the end of year — or was that thinking contigent on an unrealistic Hail Mary pass?

It is very possible there is no Hail Mary pass.

You are being too pessimistic as always. What bleak reality? And why are you bringing about the idea that Tesla lied during Autonomy Day? What is your evidence that they lied?

 

[electronblue]

There is a risk of gathering around one more hopeful buzzword.

It was once called the ”FSD codebase”.

Then ”HW3 hardware”.

Now ”Dojo”.

That magic thing that will turn the bleak reality we are witnessing into instant utopia.

The hope.

Now I remember the last buzzword I forgot.

AKnet.

That too was a thing that will suddenly make everything better — at least until the cars started dancing.

 

[electronblue]

You are being too pessimistic as always. What bleak reality?

Bleak reality compared to the promise of where we’d be and what the progress would be. There have been many buzzwords that have been thought to be the answer that will finally reveal the glorious future. None have materialized, instead it has been slow progress, evolution on evolution and no revolutions.

And why are you bringing about the idea that Tesla lied during Autonomy Day? What is your evidence that they lied?

Oh I have none, nor am I claiming they did. It is an open question that will become evident as their progress and claims from the event are eventually matched with outcomes and/or discovery. I hope they were being truthful but I have no evidence they were either. But to claim they were on track for Level 5 no geofence feature complete in 2019 for example is a big claim so hopefully that was realistic.

 

[diplomat33]

Bleak reality compared to the promise of where we’d be and what the progress would be. There have been many buzzwords that have been thought ot be the answer that will finally reveal the glorious future. None have materialized, instead it has been slow progress, evolution on evolution and no revolutions.

Sure, AP/FSD progress has been slow but it has been steady progress IMO. And V10 is in early access now which will be another step closer to FSD.

Oh I have none, nor am I claiming they did. It is an open question that will become evident as their progress and claims from the event are eventually matched with outcomes and/or discovery. I hope they were being truthful but I have no evidence they were either. But to claim they were on track for Level 5 no geofence feature complete in 2019 for example is a big claim so hopefully that was realistic.

I think Tesla is being truthful about what you call "L5 feature complete nogeofence" but I don't it means what you think it means.

 

[electronblue]

As yes, V10, another buzzword maybe to wait for?

Look. There have been so many times someone has said just wait for this or that and finally AP2+ will unveil its potential. So far it has been very slow and unsteady progress, delay after delay.

I’m definitely not saying there isn’t progress. That’s not my point. My point is be wary of the hopeful-sounding buzzwords and belief that relief is around the corner. It probably isn’t. Just more very slow progress with the occasional steps back.

Hey would love to be wrong as an AP2 car owner, but this is how I see it personally.

 

[DrDabbles]

There is a risk of gathering around one more hopeful buzzword.

It was once called the ”FSD codebase”.

Then ”HW3 hardware”.

Now ”Dojo”.

That magic thing that will turn the bleak reality we are witnessing into instant utopia.

The actual issue here is how machine learning works. Searching the jungle for the highest hill. Every time we reach a new summit, we find a new higher peak. That's the problem with claiming you've found the highest hill- there's always another waiting when you get there.

This is why it's still a research problem and not an implementation problem.

You are being too pessimistic as always. What bleak reality? And why are you bringing about the idea that Tesla lied during Autonomy Day? What is your evidence that they lied?

The bleak reality that we're decades away from having anything resembling reasonable AI, and that we may never have a system complex enough to actually handle level 5 driving. While you see pessimism, I see realism. I think you're vastly overoptimistic and it's likely because you don't understand the problem at hand.

 

[diplomat33]

The bleak reality that we're decades away from having anything resembling reasonable AI, and that we may never have a system complex enough to actually handle level 5 driving. While you see pessimism, I see realism. I think you're vastly overoptimistic and it's likely because you don't understand the problem at hand.

I don't claim to be an AI or autonomous driving expert. But based on what I've read and watched on the subject from what Waymo, MobilEye, Nvidia, etc are working on, I highly doubt that L5 is still decades away. It will happen sooner than "decades". You underestimate how fast AI progress is going.

 

[DrDabbles]

I don't claim to be an AI or autonomous driving expert. But based on what I've read and watched on the subject from what Waymo, MobilEye, Nvidia, etc are working on, I highly doubt that L5 is still decades away. It will happen sooner than "decades". You underestimate how fast AI progress is going.

MobilEye is the one company _not_ using machine learling the way Tesla and the others are. This only proves my point more. You don't have to be an expert, but you don't seem to understand what's going on either. AI is _in no way_ as advanced as you seem to believe it is. I don't underestimate how fast it's progressing, you're simply believing marketing hype and I'm listening to researchers in the actual field.

 

[mongo]

Warning: details completely fabricated since NDAs are a thing

Dojo gets the full video and radar feeds from the Tesla test fleet. It also watches YouTube dashcam videos to learn how to avoid humans.*
In parallel, it runs a simulated city with 3,500 instances of the current NN where any instsnce can go rouge at random and purposely drive poorly at various levels from mild deviation to wrong way on the expressway to aiming for other vehicles. Also has a lot of rule abiding NPCs.

The SW has the ability to change the NN layer sizes based on coefficient usage.

*which is why there are no Tesla in Russia, AP refuses to operate there.

Again, all made up.

 

[EVNow]

The open question is: Did Tesla lie at Autonomy Investor Day about realistic current status and trajectory when they said they are expecting Level 5 feature complete at the end of year — was that thinking contigent on an unrealistic Hail Mary pass?

That has nothing to do with the topic of the thread. You continue to be a concern / short troll. I wish this sub-forum was moderated to take care of people like you.

If you are not interested in this topic, why even bother to reply ? Why bring up FUD ? Why resort to slander ?

ps : I'm requesting a moderator for this sub-forum ...

Can we have somone moderate Autonomy forum ?

 

[EVNow]

MobilEye is the one company _not_ using machine learling the way Tesla and the others are. This only proves my point more. You don't have to be an expert, but you don't seem to understand what's going on either. AI is _in no way_ as advanced as you seem to believe it is. I don't underestimate how fast it's progressing, you're simply believing marketing hype and I'm listening to researchers in the actual field.

I suggest you listen to the AI podcast by Lex Fridman's interview with Yann. Hopefully you will consider Yann, recipient of the Turing Award for his work on deep learning someone who understands what's going on.

Yann LeCun: Deep Learning, Convolutional Neural Networks, and Self-Supervised Learning | MIT | Artificial Intelligence Podcast