There is a lot more than outside vs inside temperature. How much IR radiation hits the driver makes a very significant difference what feels like a comfortable temperature. There are different sensors in most cars today (that have climate control) that try to consider it and adjust the heating and cooling. Fan speed is also not as simple as 'faster when the delta is larger'. A comfortable temperature will feel too cold if the air is moving too much. So when heating the cabin, running to fan higher will have a counter effect. That's why the Model S climate control will use the lower vents (where the legs are) when heating the car and the upper ones when cooling the car. Less airfolow in the face will make it feel warmer. More airflow in the face makes it feel cooler.
There is also no way the car can know in what state of emotion you are. If you're late and nervous, you will feel much hotter. If you're relaxed the exact same temperature will feel just fine.
I think the climate control works just fine. I always leave it in Auto and just adjust the temperature up or down depending how I feel. I really don't pay any attemtion what the number is. It's irrelevant. If I'm warm, I dial it down. If I'm cold, I'll dial it up. Don't get hung up on the numbers.