For very large neural networks (in the billions of parameters), the weights do end up approximating something like a database. That's how LLMs end up remembering specific facts about topics.
In terms of file size for those weights, we can look at Llama v2. Their 70 billion parameter model has 16 bit floating point precision, so it works out to about 130 GB.
I believe HW3 has 64 GB of flash storage, and even if it could hold 64 GB of weights, it wouldn't be able to process them due to 8 GB of memory per chip.
But this NN size constraint is a part of what induces generalization. The network isn't capable of solving the task it's being trained to do by memorizing specific answers, so it minimizes the loss by learning generic answers. And some of these generic learnings may help v12 avoid collisions in the first place.