You don't notice it with ICEs because a) most ICEs don't give you as much feedback on your mileage as Tesla does and b) it gets buried in with all the other inefficiencies.
When you have a car capable of 90% energy efficiency, everything that hurts efficiency is going to hurt more than a car that is only capable of 20% efficiency. A loss of 20% on top of 90% drops you to 72%. A loss of 20% when you're starting at 20% drops you to 16%.