Isn't there the same problem if you leave a fob in the car
I've left my fob in the Prius and tried to lock it. The car can tell that the fob is inside the car rather than outside. It beeps at me and does not lock. If you ignore the beeping, you can walk away with the car unlocked and the fob inside, ready to drive.
But this very seldom happens because I almost always have the keyfob in my pocket. It's small enough, and its only use is to open, start, or lock the car, and I don't need to take it out of my pocket to do any of that. The phone is bulky in my pocket, and on a road trip it will be plugged in to the car's charger.