That's how parties work, of necessity. They are all uneasy alliances of people who can barely tolerate each other. People find the one that supports their most important issues and hopefully few things they really detest. Then they have to pay at least lip service to all of it. By getting everyone else's support, at least one or two of your favorite issues get worked on.
In doing so you need to find a way to live with the cognitive dissonance. The best way is to truly buy into all of it, as hard as you can. That wins and keeps on winning. Or you can try to mitigate things to your conscience, but that leads to a lot of halfhearted efforts and poor turnout.
Having seen student politics with and without parties - my student union had them, my engineering society banned them - I'm convinced that it's not bad voters that ruin democracies but political parties. Parties need to simplify their messages to get buy in, and promote a 'team first' over 'issues first' mentality in their members. They're anathema to principles of honest debate and compromise.
The large scale something is, the more a political party matters. At a school level you can be closely informed about all of the issues and know all of the players. You can barely do that at the level of city politics. State and federal politics simply doesn't allow it.
Large scale democracies only work when people are willing to live together. If you play democracy as a winner-take-all game, it's going to fail sooner or later.
I'm not convinced that anything works at the national scale, at least not over the long term. I suspect that the US, as one of the oldest-and-largest democracies, is demonstrating a path that others will eventually follow.
The problem is that the side that organizes always wins over the side that does not. And it's very difficult to ban political organization (which is ultimately what parties are) in a way that is actually enforceable.
The American founding fathers were mostly of the opinion that political parties are bad and should be avoided if republic is to stand. Yet they found themselves organizing into parties before the ink was dry.
So the best we can do in practice is engineer the political system such that the damage from party groupthink is minimal.
This is all true, but I think the issue is upstream of where you're pointing - democracy declines when the goal becomes to win rather than to serve the constituents. Parties are a way to win and they also reinforce the idea that winning is the goal.
I'm not sure as to solutions but I don't think they're impossible - something like an inoculation of the entire political class against the memeset that prioritizes winning over serving the constituents. Then if an unofficial party tries to seize power in a system that officially disallows them, the majority is already primed to respond in an organized way.
There are democracies with proportional representation out there. Those have their own problems in forming coalitions, but the parties themselves are much closer aligned with their base.
It comes at the cost of locality, but that's far less important today than it had been in the past. Nobody knows their congressman anyway.
I'd really like to give PR systems a try, if for no other reason than to do a reset on the current coalitions. I fear that they will eventually settle down into a pair of coalitions very similar to the current parties, but that leaves us no worse off.
It need not; you can have more proportional representative in a district based system (and still also have vote-for-person), using multimember districts with a system like Single-Transferrable Vote.
You can also get finer grained proportionality with Mixed Member Proportional which combines a district-based system (either single-member or a multimember proportional system described above) with top-up representation from party lists.
MMP would require Constitutional change in the US; but multimember districts with STV (in states with more than one seat, as well as increasing the size of the House so more states would have more than one seat) can be done by Congress without Constitutional amendment.
It's not an either-or. In mixed-member proportional system, you still get a representative specifically for your district who can thus argue for its interests. But you also get some people elected on party lists so that the representation as a whole remains proportional to party vote. New Zealand is a good example of the system in practical use.
In doing so you need to find a way to live with the cognitive dissonance. The best way is to truly buy into all of it, as hard as you can. That wins and keeps on winning. Or you can try to mitigate things to your conscience, but that leads to a lot of halfhearted efforts and poor turnout.