I see where you're coming from, but consider the cost of the replacement process itself: it can be chaotic jumping from one set of crap organizational skills to another. Point being, it's not so much that the person is very valuable themselves, but more that higher position = bigger disruption.
In fact there might be a slightly counterintuitive relation here: assuming good intent on all sides, transition from a leader with good organizational skills would be less disruptive.
You're not wrong, and I think that your last point is actually very true—and that this is part of what lets the leaders with crap organizational skills get away with it. It's not immediately obvious that their terrible leadership is ruining the organization.
What's needed, IMO, is a cultural shift away from the idea that we need a singular, extremely powerful CEO/president/chairman, and toward more group-oriented decision-making processes and devolution of both responsibility and power.
We abandoned absolute monarchies for a lot of very good reasons, and one of them is that having one guy who can just walk into any room in the kingdom/company and exercise the power of (in this case, financial, but that often becomes very literal) life and death over anyone else in there is a bad thing.
In fact there might be a slightly counterintuitive relation here: assuming good intent on all sides, transition from a leader with good organizational skills would be less disruptive.