Pied-à-terre is a thing in places like NYC, London, etc. Certainly you are right that there are differences and different cities have different features which attract different folks with different ideas.

But I think vacation home can include not just a season spot near a beach or lake or mountain - people buy vacation homes in Mallorca, or Venice, or Provence, or wherever too.

I am not really opposed to them or anything like that, I just wanted to point out there are some harmful effects for some places. The travel industry and the ease of travel is probably more of a concern, because instead of having a family buy a vacation property on a beach somewhere, you've got an entire industry turning that vacation spot into a zoo and for the number of people that do live in that vacation area it hollows it out a bit of any sort of meaning. The whole thing becomes a simulation of a vacation spot.