At the high end of the S range you're paying (substantially) for performance. Supercar performance, mind you.
Supercar performance, yes, but without a suspension to match. In fact, the top-of-the-line P100D has the same suspension as the bottom line 75. It's quite lacking in terms of handling. The discontinued P85+ had a sports suspension and you could feel it, wow is that an amazing car to drive. Not so with the current generation. They drive and feel sloppy, tons of body sway, and generally feel like a big, heavy town car. Lots of reports of steering slop as well in the current generation of cars. Not good.
At least with a BMW M3, M5, etc., you get a car with a real sports suspension and all the goodness that goes with it. Tesla gives you great acceleration, but mediocre handling.
Those who are saying the Model S/X are luxury cars don't know luxury. The Model S/X have interiors on par with Volkswagen's mainstream, mid-priced cars. There is nothing luxurious in a Model S or X, not even by a remote stretch of the imagination. I'd say the interior is at or below the quality of my VW Jetta.
Tesla has some serious upgrading to do before anyone can really compare the Model S and X to real performance and/or luxury cars from the likes of BMW and Mercedes. There is currently no comparison, Tesla fails on all accounts except for the drivetrain. Look at a Mercedes S Class or a BMW 7 Series. The Model S, in the same price range, is an embarrassment if you ignore that it's an EV. For those who don't care about the drivetrain (there are many), Tesla is simply not a viable option.