There's really no danger. The plug is absolutely not live when it's not connected to the car...that's what the EVSE is all about. If you also turn off the circuit breaker, then you don't even have to worry about trusting the EVSE.
You can try a small stick with a dot of superglue on it, or a pin or needle that could be pushed into the plastic. Use two so that you can pull up on opposite sides.