As mentioned above - charging slows as the battery gets full. Otherwise the battery's lifespan would be reduced.
Here is a more detailed graph:
https://teslamotorsclub.com/tmc/attachments/b440c7d0-86c7-4853-beb4-4a8f9790981a-png.277091/
Its for a model 3 - but the results will be very similar for your X (and mine).
The graph plots battery charge (in %) and also charging power (kW - how quickly power is being transferred to your battery). You can see on the left that the charging power quickly ramps to 120kW, or about 500km of range per hour. Then as the battery fills up the rate reduces. As you have found, those last few percent really take a long time.
From an time-efficiency standpoint - charge the car as little as possible to do all the driving you need (such as getting to the next supercharger). That way all your charging is in the fast region (when the battery is not full). Filling it to 100% every time will be very slow, and likely not unnecessary if you don't need those extra few kilometres.
In other words - to make the best use of your time - it is quicker to charge the battery when it is lower (say 20% charge to 70%) than keeping it full (say 50% charge to 100%). You get the same amount of driving (50%), but spend less time at the pump.
Also, batteries get the most wear and tear at the two extremes - 0% and 100%. From a longevity standpoint - its better to keep the car below 80% or 90%. I plan to drive my X into the ground, and so I only charge it over 80% on long trips - to baby the battery.