You are losing me here, sorry. I was not there, but have never heard anyone say a tesla representative said to them "You dont need that keycard, the phone works 100% of the time so no need to carry it".
You can fit it in your wallet / purse in the same place you carry a drivers license. Even if you dont carry credit cards or they are all digital, you still have to have a physical drivers license, and wherever you put that, you can put the keycard, regardless of "what tesla said".
Where you lose me, frankly, is, the lack of personal accountability. You chose to not take the keycard, regardless of what someone else said. The attempt to evade your own responsibility here is where I start to lose some sympathy for your situation.