> People with the monolith mindset will not be able to do it.
That's like saying "people with the phlostigon mindset will not be able to electrically insulate your house." Of course they won't, but why are you letting them touch your house in the first place? Designing components for the possibility that they'll be used in a distributed context is just proper software engineering—it's just thinking quantitatively about cohesion and coupling, really—and is to be expected of a proper software engineer.
...but that sounds crazy, right? I mean, we both know how hard it is to hire a proper distributed systems engineer. That's the implicit complaint you're making here.
Well, but wait—why is it hard? I don't think it's an intrinsic thing. Distributed systems aren't any more complicated than calculus or graph theory. It's not like we couldn't make the proper design of them a required course in colleges.
Thus, I would say it's hard precisely because people (including curriculum planners) think that there are only the two alternatives: either up-front-designing a monolith with low operational costs and fast iteration, or up-front-designing a formation of distributed components with high operational costs and slow iteration. The second alternative is obviously bad anywhere but at scale, and we need very few engineers that can operate "at scale", so it ends up not being seen as a required skill.
But, obviously, those aren't the only two options! If you have the engineers with the skill to design distributed systems, and then have them start off your software design as a "distributed system all in one process", then you get the best of both worlds: low operational overhead, fast iteration from the start (redrawing the service boundaries when you're still at the experimentation phase will be painless as long as you use a framework/language/platform that abstracts away inter-service RPC), and a design that organically "anneals into" a proper set of services, instead of a big ball of circularly-dependent mud.
But, again, you need actually-competent engineers.
...but that sounds crazy, right? I mean, we both know how hard it is to hire a proper distributed systems engineer. That's the implicit complaint you're making here.
Well, but wait—why is it hard? I don't think it's an intrinsic thing. Distributed systems aren't any more complicated than calculus or graph theory. It's not like we couldn't make the proper design of them a required course in colleges.
Distributed systems have more possible failure modes, far broader latency spectra, and a greater degree of simultaniety / nondeterminism, as compared to non-distributed systems.
Distributed systems are fundamentally more complex than purely local systems, in all of correctness, fault-tolerance, and performance.
.
I can build distributed systems, but that doesn't mean I do build them if not absolutely necessary. Because it's a pain in the ass, and I can get far more useful work done if I don't bother with it.
That's like saying "people with the phlostigon mindset will not be able to electrically insulate your house." Of course they won't, but why are you letting them touch your house in the first place? Designing components for the possibility that they'll be used in a distributed context is just proper software engineering—it's just thinking quantitatively about cohesion and coupling, really—and is to be expected of a proper software engineer.
...but that sounds crazy, right? I mean, we both know how hard it is to hire a proper distributed systems engineer. That's the implicit complaint you're making here.
Well, but wait—why is it hard? I don't think it's an intrinsic thing. Distributed systems aren't any more complicated than calculus or graph theory. It's not like we couldn't make the proper design of them a required course in colleges.
Thus, I would say it's hard precisely because people (including curriculum planners) think that there are only the two alternatives: either up-front-designing a monolith with low operational costs and fast iteration, or up-front-designing a formation of distributed components with high operational costs and slow iteration. The second alternative is obviously bad anywhere but at scale, and we need very few engineers that can operate "at scale", so it ends up not being seen as a required skill.
But, obviously, those aren't the only two options! If you have the engineers with the skill to design distributed systems, and then have them start off your software design as a "distributed system all in one process", then you get the best of both worlds: low operational overhead, fast iteration from the start (redrawing the service boundaries when you're still at the experimentation phase will be painless as long as you use a framework/language/platform that abstracts away inter-service RPC), and a design that organically "anneals into" a proper set of services, instead of a big ball of circularly-dependent mud.
But, again, you need actually-competent engineers.