Good architecture is a windmill. To do that right, you have to know how your requirements will change and grow over time, and put effort into being flexible and extensible in those feature areas.
But seeing into the future is risky/imperfect. So code bases are full of clever features and libriaries that are used exactly once. And where rapid growth happened but the code wasn't prepared, sketchy chains of conditions and runon code to paper over the gaps.
I have to disagree. It doesn't take prescience to build solid architectures. It takes skills.
Prescience (understanding your future requirements) can help dictate what your priorities are as a business, what features and flexibility to deliver today, but it has little to do with architectural soundness, which is simply a property of a system (independent of and not relative to the future).
This is my point -- to argue that a architectural sound choice is too expensive today is right only if the right skills aren't present. This is a problem for our industry to solve.
There are so many decisions to make, and they depend upon knowing where you're going. The 'soundness' of the system is a cool property, but it doesn't help make your code nimbler unless the right flexibility is in place. What other purpose is there to architecture?
Architectural flexibility and architectural soundness are orthogonal properties. An inflexible system that is architecturally sound may obligate you to build more than its more flexible counterpart when novel use cases come around but it allows you to build more when these use cases come. An architecturally unsound system makes it prohibitively expensive to solve future use cases.
So the purpose of architecture, I'd say, is to first ensure soundness (which should cost no more than building an unsound system so long as you have the right skill set) and only then strike the right level of flexibility in the architecture (another, separate skill set that is valuable but less of a current industry problem, imo).
You can evolve the flexibility of a sound architecture but you can't do much with an unsound one. Inflexiblity in a system is generally a tractable "problem". Unsoundness is not (unwinding coupling, eg, is rarely tractable.)
But seeing into the future is risky/imperfect. So code bases are full of clever features and libriaries that are used exactly once. And where rapid growth happened but the code wasn't prepared, sketchy chains of conditions and runon code to paper over the gaps.