This can happen for a couple of reasons, but it's usually on the phone side. By the way, this:
"But unless I'm right at the nose of the car..."
has nothing to do with it. The car and phone are not sending signals directly to each other, so the closeness of the two of those objects is irrelevant. The phone is connecting to the cellular towers, which are then sending the signal over the internet to the servers at Tesla in California, which then authorizes the app on your phone, and then sends a signal back out over the internet to the cellular receiver in the car. So even if you are standing right next to your car, it is still going hundreds of miles round trip to talk to Tesla's systems and then back to the car. So next to the car or on the other side of the world doesn't make any differerence, as long as your phone and the car both have strong internet connections.
But, as to making it work, if it just spins like that, I've seen it happen sometimes if the app was just put into the background, and then it can't quite bring everything back up when you switch to it, like it won't renew the login the make it current or something. Force close the app, and then start it up again, and probably about 98-99% of the time, that will make it connect properly again. But since it is doing a specific connection over the internet, there have been rare times over the years that there is some widespread outage of cellular networks or Tesla's servers were down for some reason where it just can't connect for half a day or so. My wife always uses the app to drive, since she doesn't carry a purse or wallet--just a few cards--but she always has her phone. She got caught with that one day, where she was going to leave work early to go to an appointment, but AT&T happened to be having big problems with their cell network in several cities, so no go. I had to run down to give her the key fob to be able to leave. Fortunate we do work at the same place then.