I guess it depends on what you mean by "custom hardware and software".
This is not a completely apropos counterexample, but I was involved in the Supercomputer industry back in the 1980s. There was the "big iron", dominated by companies like Cray, CDC, IBM, Fujitsu, Amdahl. Then there were a number of "mini-supercomputers", startups using different architectures and specialized hardware. Most of them you'll never have heard of: Elxsi, CHOPP, Pyramid, nCUBE, and a few others. What killed them? "
The attack of the killer micros". Basically, machines with lots of commodity microprocessors. (I was involved in a project at IBM Research that replaced a vector mainframe with a rack full of Riscsystem 6000s. Got into a lot of political trouble. Deep Blue was a rack full of RS6000s too, which is how I got to meet Gary Kasparov. Long story.)
Companies like Intel (especially), Motorola, National Semiconductor, at the time could throw more money and effort into making better microprocessors than these other companies could improve their specialized hardware. At any given instant, their lead was only 18 months over a machine with a bunch of micros at 1/10th the price (Sequent, Sun, others.)
Fast forward to a decade ago, same thing happened with general purpose micros versus dedicated GPUs, we still need a CPU but it controls an array of GPUs. Fast forward to now, and we see dedicated (but soon to be commodity) NN machines taking part of the market that was dominated by GPUs and vector instruction sets.
I see us at a fork in the road. Who will be the big player in NN processor chips? Who will be the Intel, or NVidea,
Bitmain, for NNs? There are a bunch of incumbents who have recognized the space being important (Intel, NVidea). There are a bunch of new startups (
LMGTFY). Then there's Tesla. If it was any other company/entrepreneur I'd say that Tesla was not going to succeed in this area, because they would be the supercomputer manufacturers facing the killer micros, and one of the startups would win. But they're not, they are (in this area anyway) a well-funded startup with some of the best talent and a captive market. I think they will lead, not get crushed. I certainly hope so. BUT, to lead, they have to make the chips available to others too; their captive market isn't big enough, and I don't just mean other car manufacturers. They have to open to other entire industries. In other words, they succeed if their hardware is no longer "custom", rather "commodity".
Corollary: Intel and NVidea will not win this race. Incumbents never do, too much baggage. If it isn't Tesla it'll be a different startup. Hmmm, must start looking around and following some of those startups.