Well, that's exactly what you're doing; waking up the app
. Swipe the app away, and you'd need to open it up again before being able to unlock the car... which is exactly what I do until I need to use the car. Remember the 'phone as key' requires both the app, and bluetooth, active ('key' has a separate BT connection than the phone). Maybe there's a way to keep an app active when phone is locked, which would solve that issue, but I haven't looked into that.
And yes, the solution to that misery is to drop another $150 into Tesla's coffers for the darn fob which should be standard indeed.