I disagree wrt software engineering. I have been a dev for 10 years, so still new. But. I don't think the problem can be solved with better techniques. I think the problem is humans just aren't smart enough to do the work. There is a fundamental limit to how much you can compress certain things. It's called the algorithmic complexity of the thing. Some things are just complicated. Some things are just large and hairy which ever way you turn them, whichever basis you construct. This is why we are spinning our wheels, why the next big architectural technique never lives up to the hype. There comes a point where the complexity of the thing you are trying to construct exceeds the capacity of any network of human beings to construct. I think there is a way to go with tooling, freeing people from the overhead of mundane work, which fractures people thinking time and reduces the complexity of the things that they can hold in their head at any one point in time. But the fundamental limit remains. I think ML is going to be the next big tool set. It will allow us to add layers of perception over the code and allow us to perceive the code and problems in different ways, freeing us to think at a higher level. But the fundamental limit remains. What we need is the ability to not just apply the human mind, with it's 20watts of power, but to open up a multi megawatt power station on the problems. We need genuine AI and I think this should be our main focus, not pissing around with little problems around the edge, not building the next phone app. Every other programmer and scientist in the entire fucking world should be working on AI.
> I think the problem is humans just aren't smart enough to do the work.
I think this is true if you define "the work" as building on top of the infrastructure we have today. I believe we're capable of building conceptually clean, non-ball-of-mud architectures, but the need to interoperate with piles and piles of legacy systems forces compromises into the design. Just look at a typical web application stack; you've got layers and layers of cruft, and nobody is able to pull off a bold move that tears layers off; the best we can do is add more layers on the top.
The amount I have to think about is considerably less. All the complexity just melts into Functions and Objects. There's similar stuff happening in React with inline styles. Mixins, variables, custom-properties, state-dependence, automatic-prefixes, and more are available without language extension when styles are expressed as Objects of css properties. Whether or not they're satisfactory, there are occasional efforts to derive more functionality from fewer abstractions.
To put words in your mouth, what you're saying is basically that we've reached the pinnacle of theory and practice in software engineering, aside from hiring AIs to do the job for us. That, to me, doesn't seem to be a tenable position. We haven't even reached the limit of application for known best practices, for example. And to say that we've reached the limit of understanding after only a few decades of practice seems equally unlikely.
Software is a complex subject, but is it so much different than chemistry, physics, mathematics? Each of which took hundreds of years to progress through multiple stages of advancement. Is functional flavored OOP with bolted on TDD the grand unified theory of programming? That seems unlikely to me. I suspect there are further conceptual breakthroughs on the horizon. And there is still a tremendous amount of improvement available just in getting everyone up to the level of adhering to known best-practices.
I think the problem is humans just aren't smart enough to do the work.
Every other programmer and scientist in the entire fucking world should be working on AI.
Or we can work on techniques to improve our ability. Our collective IQ improved a lot when we dropped roman numerals in favor of arabic/hindi numerals.
In programming we have know of better techniques for a long time[1]. Sadly, as a community we just haven't put understanding as a priority. We follow "Move fast and break things." instead of "Elegance is not a dispensable luxury but a factor that decides between success and failure.".
