Assuming the GFCI breaker isn't defective, it tripping means that there is a current imbalance between the input and the output conductors.
So, I would check a few things:
1) Make sure that the GFCI breaker has its neutral pigtail directly tied to the neutral bar in the panel
2) Make sure that there is nothing connected to the neutral output of the 50A GFCI breaker (or, only a neutral run directly to the NEMA 14-50 outlet that your HPWC connector is connected to. The H{WC doesn't use the neutral, but it is important that nothing else is connected to it)
3) Make sure that the L1 and L2 (usually red and black) conductors go directly to the HPWC and are not connected to anything else.
4) Make sure that the neutral and ground conductors are NOT connected after the GFCI breaker anywhere. One way to test this is to disconnect them at the panel and ring them out with a meter. They should have no connection.
5) Try a different HPWC... it is possible that the unit you have is leaking some current to ground that it should not be (One way to test this: a clamp meter over the ground wire coming back from your HPWC. It should read 0A always, including when charging).