For the sake of clear communication, I need to correct sone terminology:
The charger is built into the car.
The item in the wall is the Wall Connector. It is a higher threshold GFCI, contactor, and electronics to tell the car how much current it can pull and control the contactor, plus check for faults.
The car's charger interfaces to the 400V pack, but the Wall Connector only provides split phase 240V.
The ground connection to the car exists whenever the car is plugged in, it is not switched when charging is started or stopped.
If the ground at the Wall Connector is actually connected to neutral at an intermediate sub panel, then you could get an offset between the car and the floor (note this would be ore apparent if the ground is wet). This can also occur if everything is correctly wired due to voltage gradients in the earth itself (the reason there is a bonding ring around poools and hot tubs).
Why would this potential be more apparent when charging? Hard to say, usually neutral shifts occur with bad connections and 120V loads.
Edit: Not to imply it is a ground issue. Other items could cause a slight AC leakage.
@MC Lee
Do you notice any lights in the house that change brightness when you charge?
Where was the car when this occurred? Was it wet out?