I love the phone-as-key convenience, but agree it is not nearly as seamless as my old Cadillac's reliability of linking driver profile to smart entry fobs. The Caddy never mixed them up even when both of us had our fobs and switched driver/passenger duties at a gas station stop. I think that car had sensors at each door and trunk.
The Tesla phone key just as often selects the correct profile as not when we're using the car together. Maybe a future version support UWB will have better time-of-flight ranging measurements for each phone.
It has occurred to me that it might be less trouble to use the RFID keycard, since I keep it in my phone folio wallet anyway as the outermost card. As such, I can hold my phone up to the B pillar to lock/unlock the car. That always selects the correct profile. I'd be less likely to leave the phone on the wireless charging mat as well. Hmm.