I've always just tapped the magnet and the door released - the button on the charge cable never works for me... My method isn't even listed in the manual.
pg 41 Opening the Charge Port:
With Model X unlocked, or a recognized key nearby, press and release the button on a Tesla charge cable to...