From my point of view, full-access pipes are the problem. Asymmetric connections, bandwidth caps, incoming port filtering, etc are all de rigueur for large fixed line ISPs these days. This is especially true in North America. The mobile connectivity picture here is even worse.
Demand can be generated, if you do it correctly. The hardware is less of a challenge if you do the distributed computing on several layers at once. Apple has shown us the way on UI/UX, as has science fiction literature for decades now.
We'll never get completely away from client-server (especially with mobile computing), and that needs to be part of the design. But we have seen the power of peer to peer networks, and even some kind of proxy network between the p2p base and mobile clients could be done in a non-evil way.
Cheap hardware and cheap bandwidth are easy to get... in datacenters. And if you have reasonable scale you can amortize away the cost of high availability, too. The cloud isn't the problem here, it's SaaS. If you have a personal cloud server, you can not upgrade all you want.
That's an interesting angle that I don't know enough about. For example, do the laws governing SaaS data provision to law enforcement also apply to data centers? If they do, then you're giving up some privacy from the gov't for the convenience of outsourcing server management.
I guess what's interesting about your idea is that it presents control as a continuum as opposed to a binary. There are varying degrees of control/convenience possible, but most of the current options are heavily weighted away from user control.
Also, I don't think this thread is really about the cheese-moving/silent-upgrade phenomenon so much as data and identity ownership.
We're not getting away from SaaS without an underlying infrastructure for something along the lines of Software as a Peer to Peer Network. PaaS and IaaS have the potential to exhibit some of the same problems as SaaS. Look at the AWS/Wikileaks thing. There may not be privacy issues in the same way, but Daddy can still take the car keys away.
Also, bandwidth is only as good as the size of the total pipe between communicating entities. The cheap bandwidth has to go all the way to the end-user or it doesn't help too much. Think about backing up personal data. The initial upload takes forever in cloud backup systems.
Demand can be generated, if you do it correctly. The hardware is less of a challenge if you do the distributed computing on several layers at once. Apple has shown us the way on UI/UX, as has science fiction literature for decades now.
We'll never get completely away from client-server (especially with mobile computing), and that needs to be part of the design. But we have seen the power of peer to peer networks, and even some kind of proxy network between the p2p base and mobile clients could be done in a non-evil way.