you can definitely drive without the fob. I tested this by accident recently. I had the fob in my pocket, sitting in the passenger seat. Wife was going to be driving, so i was showing her the controls, letting her get a feel for the regen braking, etc... (was her first time driving the car).
Then i got out (fob still in my pocket - oops! had forgotten about it), and drove our other car following her to where we were going. I realized about 10 minutes into the drive that i still had the fob. She never noticed, and there wasn't anything unusual about her ability to drive, even without the fob.
It does seem somewhat odd to me that you can do that.
Tempus