Yep, if you use a multimeter with one probe in the ground hole, you'll only see 120v to any other pin. You'll only see 240v with respect to each other, not ground.
I'd stick with 6 guage wire for the wall charger and a 50 amp breaker. You're not losing out on anything, the extra 8 amps...