Project Dojo - the SaaS Product?

[Buckminster]

Sounds like Elon is going to re-building "open AI" from the ground up. Perhaps this time make sure it is OPEN and unbiased, and maintain control of the investment this time.

https://twitter.com/x/status/1630610998460555264

I wonder if that is what Elon mean when he said he will reveal something during Investor Day for all of humanity? Good use for dojo?

Edit: Youtube on this topic

 

[mspohr]

As I hope we have learned from the Twitter saga, "unbiased" very much depends on your point of view. Musk has his own view of "unbiased" which some people view as very biased.
Beware of people claiming to give you the unbiased "truth". It's usually very much just their opinion.

 

[Buckminster]

https://twitter.com/x/status/1631033881079197696

 

[Buckminster]

Thanks @wipster. That did help me find the quote:

Here it is in the video starting at 3:54:55:

Elon says:


I'd like to know exactly what Elon meant by increasing NN training capability by an order of magnitude. It seems "order of magnitude" is a term thrown around a lot. But if that's truly what is happening then we are all going to be very rich rather soon.

Well, AI hardware is heating up. Due to Nvidia clients making ASICS, they are now adding more and more specific AI accelerators into their training hardware in which under specific work load they are claiming 7x-9x faster training vs their last generation A100. So even if Tesla buys the same amount and clone their A100 cluster, it should in theory be an order of magnitude faster. However those are claims so who knows in the real world.

 

[Buckminster]

https://twitter.com/x/status/1636105708558221312

 

[Buckminster]

Huge news - Elon didn't disagree:

https://twitter.com/x/status/1644659294153977858

 

[Buckminster]

https://twitter.com/x/status/1647324600462888960

 

[Buckminster]

That really depends on Dojo and how that's going because the back end hardware that runs the brain can be incredibly valued when coupled with the code. Tesla has been quiet on it for a while and we don't know how much of FSD has been switched over to Dojo.

If Dojo reaches it's potential and the hardware becomes a viable hardware alternate to say Nvidia to be sold to other companies. If Tesla was able to get to general AI computing for say Optimus then the combination of the code + the specialized hardware to run the code becomes insanely valuable.

Will be a couple years to really see how it all shakes out.

 

[Buckminster]

What manipulation? Do you mean people selling their shares because Tesla's prospects look worse? There has been some pretty bad news lately:

- Odds of DOJO working going down by the way Elon described the project in the Q1 call.
- BYD saying full autonomy won't work and that they're giving up on it.
- Demand falling far short of supply - large price drops not working as well as hoped.
- Earnings cratering.
- Energy going from bigger than auto with 50% margins to bigger that auto batteries with margins at 20%. That's like a 85% drop in 2030 profitability.
- No update on Bot during the Q1 call.
- Elon getting deeper and deeper into Twitter.

I thought that the news flow was great in the first quarter with FSD progress, Mexico, Reno, Energy, Lithium refining etc. But, the news has been pretty dreary more recently.

What?
We are talking about the call where they mention selling access to DOJO similar to AWS, right?

I could be reading too much into Elon's comment but it seemed to me he is now a little less optimistic about DOJO working: from the Q1 call "I'm not sure I fully understand your question, but I'd look at Dojo as like kind of a long-shot bet, but if it's a long-shot bet that pays off, it will pay off in a very, very big way." IIRC his most recent previous comment on DOJO working was that it had a 50/50 chance. Long-shot is below 50/50. The longer we go without success in this project the longer the odds get against it working. We have very capable, motivated competition in this field not some bloated legacy car company.

I think I see the confusion, Dojo was mentioned twice, in two contexts. One being internal training and the other being external money maker:

Intro comments which were positive toward it:


And as part of an analyst question. Also positive, but hedging the probability of it being hundreds of billions in revenue.

 

[Buckminster]

https://twitter.com/x/status/1654568279426605056

 

[Buckminster]

https://twitter.com/x/status/1654892810590650376

Translation on 1. - Dojo is gonna kill it!

 

[Buckminster]

Papafox recommended watching this. You won't learn very much new but it is important to reinforce your thesis. Nice bits on:
Optimus 41 mins
Dojo as a service 52 mins
TruthAI 62 mins

 

[Buckminster]

Today it clicked for me. It's so obvious what Elon intends to do. Earlier today he posted:

https://twitter.com/x/status/1661928353006493696

What does this mean?

For the last few months he has been saying how hard it is to get hold of nVidia A100s for his different companies, talking about back alley deal, how it's harder to get than drugs. Not unlike how hard it was for him to buy a Russian rocket. Well how does Elon react to other people not wanting to do business with him?

Here are 2 pictures to ponder:
View attachment 941195
(look at the data center vs gaming lines)

At AI day he said that Dojo would be 6x cheaper than A100.

Then think about these numbers:
View attachment 941196
Datacenter is like 66% of nVidia earnings and growing 3x faster(ie 3x higher PE) than gaming over the last 3 years, YoY it was +14% vs -38%... Obv datacenter is a huge part of nVidia's market cap.


Does Tesla and nVidia's relative market cap make any sense? Should Tesla be spending hundred of millions in back alleys trying to get hold of those A100? X.ai?

So how far along is Dojo? 2 years ago at AI day it could train GPT.

AI day 2022 they were talking about how they optimize it for parallelization and were overcoming piezo-electric bugs when they pushed them to the limit. They were gonna build there first exapod in Q1 2023 that would 2.5x their labelling capacity...

Half a year ago they intended to 10x their capacity over the next year and 100x the next year. And they already had one of the biggest nVidia clusters in the world. This was before chatGPT and nVidia taking of like a rocket and the world going crazy for compute. And by now they likely have had their first exapod up and running for a few months with 6 more to build in Palo Alto.

I have a strong feeling that Elon thinks it's crazy that nVidia has a higher market cap, when Tesla's product is 6x better, and that he intends to take over the datacenter market. The tweet was him hinting that Tesla is coming for their lunch. And usually when he says things like that he means it...

Will Tesla sell the chips? Maybe, if they cannot build the clusters fast enough themselves. If nVidia can get that high market cap just selling them, that is at least a proof of concept for that strategy. Maybe doing a a competitor to AWS/Azure/GCP is better, maybe not, but at least they have one proven strategy as an option.

So what does this mean? For the next for quarters I expect extra capital expenditures and then in a few more quarters expect Tesla to slowly eat nVidia's datacenter market cap. Because it doesn't make sense that nVidia are overcharing everyone for a worse product, having a higher market when Tesla could do it better. For a product that scales very fast and has a demand that is expected to grow every year. And it will be the life blood of Tesla and X.ai over the next years, so why not secure it even more?

 

[Buckminster]

https://twitter.com/x/status/1662758457781657600

 

[Buckminster]

Penultimate section on Nvidia Grace Hopper. Elon's friends seem to be unaware of Dojo.

 

[Buckminster]

https://twitter.com/x/status/1665799225341296642

 

[Buckminster]

I must be missing something. I keep reading that Nvidia make chips, but google tells me that Nvidia chips are made by TSMC. So their lead is in chip design? I also read that chip design is, these days, performed by AI.

So where is Nvidia’s moat? I can’t see it. Either Tesla or Apple or any other player capable of circuit board assembly and big enough to place a fat order with TSMC can leap frog Nvidia, no? Dojo, comes to mind.

To lead in AI and stay there requires a data flywheel that nobody else has, like FSD and soon, Optimus. I feel like I must be wrong on this, otherwise the whole market is barking up the wrong tree.

They make more than chips. They make the driver software that runs on those chips, which it tightly tied into their low level AI programming language, CUDA, which they also own. And then their boards use a proprietary high speed network called NVLink along with proprietary switches for same. A huge amount of the AI software ecosystem is essentially tied to NVidia this way.

Nvidia is also now starting to roll out entire rack systems for AI.

They have a formidable moat. Not insurmountable, but it’ll take years for anyone to even compete with them at a serious level.

Tesla’s Dojo is the only system I’ve seen which comes close. But Tesla hasn’t written all the software stack pieces needed to sell Dojo as a standalone system. It would take Tesla a year to do that if they wanted to, and then many years to muscle NVidia in the marketplace. Not sure Tesla even wants to do that. I suspect Tesla will just use Dojo and other AI systems in house for FSD and Teslabot, and then years down the road, make it available to others if they have a technological lead, which isn’t assured since NVidia isn’t standing still.

Thanks for setting me straight on Nvidia. Great answer.

On Optimus, I will still be watching out for that viral video. The market is forward looking - only needs clear proof of humanoid robot concept and it gets priced in. Actual profits can come later.

CUDA is what now? Last time I checked, years ago, it was a GPU vector math lib. CUDA Toolkit - Free Tools and Training looks like it still is. Supports c/c++/Fortran/Python

Two ex Tesla leads are the competition to Dojo and nVidia.

We have Jim Killer doing the hardware:

And Chris Lattner doing the Software:

Give it a few more years and the space should be very interesting. Google are not standing still and are investing pretty heavily in TPU and GCP. Amazon have been doing their own chips for AWS also and Pytorch was developed by Meta(who probably didn't enjoy WWDC very much). And China are not super happy about US doing chip warfare on them, history will tell if that was a good decision or not. Kai Fu Lee had an interesting book on the topic:

Did you watch the video before replying to the post?


nVidia is a HARDWARE manufacturer. Cathy Wood makes the point that HARDWARE manufacturers of renown were each eventually dwarfed by the purveyors of the SOFTWARE that utilized their hardware.

Those who developed the applications for the bleeding edge hardware were many times more successful than the ones who originated the hardware itself.

The video wasn't about moats at all. It exemplified how companies like Microsoft, Apple, Google, and it looks like Tesla will too, achieved markedly higher growth rates and profits by creating useful applications for those various processors over the years.

In Tesla's case they have also developed their own hardware when their applications exceeded the scope of what the market offered.



As for TSMC, they build chips to order, not circuit boards. What they build is owned by whoever did the chip design (not circuit board design) and payed to have them build it. I don't believe that the goal of Tesla is to release Dojo chips assembled onto boards by Tesla in order to compete with nVidia. Tesla is more interested in developing the AI than they are the hardware, for all the reasons Cathie and SMR pointed out in the video.

And yes, Tesla has designed their own chip and had a chip shop build it to their spec. Because they also have the other pieces of the AI puzzle (robotic auto fleet, Optimus, Dojo, huge database, etc.) ARK indicates they are best positioned to lead AI development and deployment in a variety of use cases with staggeringly enormous potential for investors.

 

[Buckminster]

I would say the company that comes close to what Nvidia is doing is actually AMD as their MI300 platform is a one stop shop all in one x86 training computer on one chip. It uses Rocm which is supported natively in Pytorch, however most developers would rather just use CUDA as it's the most consistent to their expectations. Dojo is very specific to one type of training with a Tesla compiler most developers wouldn't touch with a 10ft pole(they wouldn't touch AMD's great hardware with a 5 ft pole is how much of a monopoly Nvidia has on ML). So when Tesla said they are going to become AWS, my take is that they can provide you a service in which you tell Tesla what you want trained and provide the data then Tesla engineers will do the training for them.

 

[Buckminster]

Sounds like the sort of service Dojo will provide:

https://twitter.com/x/status/1668665102902657024

 

[Buckminster]

https://twitter.com/x/status/1671239079394934785