Battery swapping would not only solve "fast charging", it would also solve any "degradation" doubts that people have (this assuming that you don't need to come back to get your own battery back, that the swapped battery is just as good as it was when the car was bought).
Until they find out that the battery pack that they received only has ~180 miles range left rather than ~250. Sure, that won't happen on day-one, but it will happen.