shrineofchance can you summarize what they are talking about and how it relates to Tesla? I got as far as this before giving up:
We care a lot about being able to generate content with neural networks, and being able to specify and control that content through natural language is super appealing, because natural language is this amazing natural label space that humans talk to each other all the time... (Episode #2, 0:45)
“...imo much performance of neural nets across the industry is becoming upper bounded by large-scale data to pre-train/finetune from, not algorithms (they are largely known/published and often available as open source) or compute (available in cloud).”
That doesn't mean much. "Performance of neural nets" as compared to what? Yes we know if you have a database of everything then the neural net will be able to identify everything. We will never have a database of everything. Also this doesn't speak to complex programming needed to drive in adverse situations that neural nets can't handle. The type of situations they may come up rarely that are difficult to have a database entry for.
If ever FSD becomes common , then people will come up with creatives ways to spoof the cars into doing something they shouldn't do. In the end, if there is (remote) human driver assistance, that will take care of many of these odd situations.