I had pretty much the same impression. Drove both (separate days) but they gave me the LR AWD until the next day. Acceleration is still very fast, with recent power updates suggest the car is probably closer to 4.1 0-60. The weak point of handling for the LR is the wheels and tires which (as many others have stated) are more efficiency and comfort-oriented than handling. Ideally, I would like to test the LR with the 19 sport option or with some aftermarket wheels/tires to get a closer comparison.
I ended up ordering the M3P despite reason saying the LR was "just about as good," and with the difference in the price, you can upgrade however you see fit. However, I have done the springs/coil overs and aftermarket wheels/tires in the past and would rather keep things OEM. If you like to tinker and plan to change out wheels/tires/brakes/etc, or just want a fast, comfortable car the LR is a great option. Stock for stock I preferred the performance looks and feel over the LR.
Good luck with the decision.