The faster-than-exponential growth phenomenon (I prefer to avoid the term "singularity", which makes no physical or mathematical sense) goes back billions of years. It took 2.5 billion years for life to get to sexual reproduction, another 700 million to the Cambrian explosion, and so on with each phase getting shorter and the growth rate getting much faster. In human history, we've seen typical economic growth go from less than 0.001% per year (paleolithic) to 0.1% per year (urbanization, agrarian life) to 1% (early to mid-industrial) to ~6% per year in the 1960s. (We've slowed down to about 4-5%, globally, and the developed world is stagnant. That's another topic.) I don't know what will happen in the next 10 years regarding Moore's Law, but I don't see any reason to doubt that the faster-than-exponential growth can carry on for a while. It would surprise me not to see 10% world economic growth by the end of my life.
The OP does a great job of explaining why Silicon Valley won't be involved in much of a meaningful way. I think that the biggest problem is that the balance of power between connections guys and talent has fallen into a state of irreversible moral calamity. In their time, Steve Jobs and Steve Wozniak were approximate equals. The business partner wasn't innately taken to be superior to the engineering partner. That changed somewhere between 1995 and 2005. Now, the connections guys are the only people who really matter and engineers (even up to the CTO level) are largely viewed as interchangeable. And they probably are interchangeable given that these businesses are all built to be dumped on a buyer inside of 3 years, and the consequences of mediocre engineering generally don't have business-macroscopic effects (beyond "throw more money at it" problems) until 5 or 6 years have passed.
I don't know where, when, or how the positive-sum mentality of the old Silicon Valley will reconstitute itself. I do think that it will be at least 500 miles away from the current one, because the current tech hub has "Future Detroit, But With Less Architectural Character" written all over it.
I believe that two things will happen in the 21st century that will have outsized impacts on the economy, both of which are software related.
One is that we'll finally get a leg up on software development. This is more likely to be the result of lots of incremental improvements. We've made huge strides since the '60s, we have tons and tons of tools that we've built and use, but still at the end of the day software development is a crude endeavor. More often than not projects end up with "big ball of mud" architectures. And we lack the fundamental models and terminology to even talk about software design and architecture at a reasonable level most of the time. We have all these tools like TDD, static analysis, and so on, all of which is more or less bolted on to our other tools. And I can't help but be reminded of both the pre-structured programming era and the pre-OOP era, when there was a transition from a kludgy mess of useful components bolted on to existing paradigms that congealed into a cohesive design that became a universal standard. In, say, 30 years programmers will not only have better tools they'll have better techniques, better standards, and better models. They'd be able to look at the software projects of today and go "oh, well, here you have a classic example of X common architecture design anti-pattern, which you can fix using techniques U, V, W, and Z" and so forth. I think that alone will unleash a tremendous amount of potential in the use of computing systems and result in an inflection point in the effectiveness of software development projects.
The other is fully automated and configurable manufacturing. We have almost all of the necessary components in place for that today, but nobody's put them all together yet, but it'll be transformative. Imagine being able to upload a handful of files to some service somewhere and then those files would be used to produce PCBs; mechanical components and structures built from various materials (plastics, metals, composites, etc.) using 3D printing, injection molding, CNC milling, etc; and then having all of that assembled into a final device then shipped off to you. Imagine how that changes the economy we have today, how much it could accelerate innovation, how it could result in un-serviced economic niches finding satisfaction, and so on. Think about how many thousands of kickstarter projects would translate into simply designing something then making use of such a service. Also imagine how much things change if you can have a completely automated factory pumping out parts and goods 24/7. Imagine if you could bootstrap an industrial economy anywhere on Earth, or off, with a few shipping containers of machine tools set up the right way. And then you get into idea like self-replicating factories. Think about how all of this changes the economy into something that we would scarcely recognize today? What if manufacturing an automobile in 2100 was economically equivalent to manufacturing a diecast hot wheels toy today?
I disagree wrt software engineering. I have been a dev for 10 years, so still new. But. I don't think the problem can be solved with better techniques. I think the problem is humans just aren't smart enough to do the work. There is a fundamental limit to how much you can compress certain things. It's called the algorithmic complexity of the thing. Some things are just complicated. Some things are just large and hairy which ever way you turn them, whichever basis you construct. This is why we are spinning our wheels, why the next big architectural technique never lives up to the hype. There comes a point where the complexity of the thing you are trying to construct exceeds the capacity of any network of human beings to construct. I think there is a way to go with tooling, freeing people from the overhead of mundane work, which fractures people thinking time and reduces the complexity of the things that they can hold in their head at any one point in time. But the fundamental limit remains. I think ML is going to be the next big tool set. It will allow us to add layers of perception over the code and allow us to perceive the code and problems in different ways, freeing us to think at a higher level. But the fundamental limit remains. What we need is the ability to not just apply the human mind, with it's 20watts of power, but to open up a multi megawatt power station on the problems. We need genuine AI and I think this should be our main focus, not pissing around with little problems around the edge, not building the next phone app. Every other programmer and scientist in the entire fucking world should be working on AI.
> I think the problem is humans just aren't smart enough to do the work.
I think this is true if you define "the work" as building on top of the infrastructure we have today. I believe we're capable of building conceptually clean, non-ball-of-mud architectures, but the need to interoperate with piles and piles of legacy systems forces compromises into the design. Just look at a typical web application stack; you've got layers and layers of cruft, and nobody is able to pull off a bold move that tears layers off; the best we can do is add more layers on the top.
The amount I have to think about is considerably less. All the complexity just melts into Functions and Objects. There's similar stuff happening in React with inline styles. Mixins, variables, custom-properties, state-dependence, automatic-prefixes, and more are available without language extension when styles are expressed as Objects of css properties. Whether or not they're satisfactory, there are occasional efforts to derive more functionality from fewer abstractions.
To put words in your mouth, what you're saying is basically that we've reached the pinnacle of theory and practice in software engineering, aside from hiring AIs to do the job for us. That, to me, doesn't seem to be a tenable position. We haven't even reached the limit of application for known best practices, for example. And to say that we've reached the limit of understanding after only a few decades of practice seems equally unlikely.
Software is a complex subject, but is it so much different than chemistry, physics, mathematics? Each of which took hundreds of years to progress through multiple stages of advancement. Is functional flavored OOP with bolted on TDD the grand unified theory of programming? That seems unlikely to me. I suspect there are further conceptual breakthroughs on the horizon. And there is still a tremendous amount of improvement available just in getting everyone up to the level of adhering to known best-practices.
I think the problem is humans just aren't smart enough to do the work.
Every other programmer and scientist in the entire fucking world should be working on AI.
Or we can work on techniques to improve our ability. Our collective IQ improved a lot when we dropped roman numerals in favor of arabic/hindi numerals.
In programming we have know of better techniques for a long time[1]. Sadly, as a community we just haven't put understanding as a priority. We follow "Move fast and break things." instead of "Elegance is not a dispensable luxury but a factor that decides between success and failure.".
The OP does a great job of explaining why Silicon Valley won't be involved in much of a meaningful way. I think that the biggest problem is that the balance of power between connections guys and talent has fallen into a state of irreversible moral calamity. In their time, Steve Jobs and Steve Wozniak were approximate equals. The business partner wasn't innately taken to be superior to the engineering partner. That changed somewhere between 1995 and 2005. Now, the connections guys are the only people who really matter and engineers (even up to the CTO level) are largely viewed as interchangeable. And they probably are interchangeable given that these businesses are all built to be dumped on a buyer inside of 3 years, and the consequences of mediocre engineering generally don't have business-macroscopic effects (beyond "throw more money at it" problems) until 5 or 6 years have passed.
I don't know where, when, or how the positive-sum mentality of the old Silicon Valley will reconstitute itself. I do think that it will be at least 500 miles away from the current one, because the current tech hub has "Future Detroit, But With Less Architectural Character" written all over it.