many of us consider southern califoria part of the southwest, but not so much the central and northern parts. similarly with texas, the southwestern parts, less so with the rest.
that's because that's what borders mexico, and we have a politico-cultural fascination with that border in particular.
To me, "southwest" is sand and cactuses. San Francisco isn't the southwest because it doesn't have that sort of climate. Also because 37°N doesn't seem meaningfully "south" relative to the continental US as a whole.
I don't think it has much to do with Mexico. Nevada (well, Las Vegas anyway) seems the "southwest" to me, and Nevada has no border with Mexico.
sure, that might be the picture that pops into our heads, but the reason we think about the southwest as a thing at all is not because we all love the desert.
I'd say we think of the "Midwest" because it's flat and the "South West" because it's sandy and hot. These clearly are geographic regions, like Pacific Northwest, (which has evergreen forests and mountains). If your point is these regions would have different cardinal names if borders looked different, that might be true, but that's not exactly profound and it's certainly not a matter of a "politico-cultural fascination with that border in particular."
that's because that's what borders mexico, and we have a politico-cultural fascination with that border in particular.