That describes any hard commodity, no matter how common. Is water a "natural monopoly"?

The pipes that service a city are.

Yes.

It is a utility.

No, that's water pipes to your home. You can buy water in many, many, many other ways.

I assumed you meant water piped to a home as broadband is cabled to a home.

I can't go out and buy a bottle of WiFi to use how I see fit.

Sure, but you can buy alternative methods of accessing the internet to keep this metaphor going. They're expensive and aren't as convenient but that's true of having to go to the store to buy bottles of water as well.

How about plutonium?

Plutonium is not traded on the free market.