Yes, you really should because you should have something pretty solid to enforce the right amp limit. So my second question comes to HOW are you lowering the amp limit? That setting where you can lower the amps from the touch screen in the car (or mobile app) is really not very reliable, so you shouldn't count on that. Reconfiguring a wall connector to a 50A circuit is much better, but you should still probably switch the breaker too.
If you can find a device that really lets you SET it for a 44A limit, then yes, that would be an appropriate thing, but it is extremely rare to find a car charging device that has a setting at that amount. And no, just turning it down to that level on the car's screen doesn't cut it. The cars can forget that setting and default back to max for a few reasons that aren't uncommon (GPS glitching, firmware updates, etc.).
Trying to fudge your way around these things just isn't a good idea. Fix the circuit to a 50A breaker to be correct all the way through.