My dream tutorial shows how to move from what I have to the new thing, because that teaches me what new ways of thinking I need to adopt.
Like right now at work we have a small team making a website with docker-compose to specify the dev environment, pip-tools to pin python dependencies, and saltstack to deploy new state to bare metal in production.
How exactly, command by command, would nix solve the same problems? What would change from my current solution? How would I think differently? How would my life get better? What parts wouldn't be solved and I'd have to keep some of the old stuff around?
Those are the tutorials that convince me to switch, because they get me over the hump of not getting the new paradigm, and being afraid of the downsides I don't know about yet. It's tricky because it's pretty specific to how someone's currently solving problems, but maybe knowing what I'd optimally want can help with figuring out something doable.
(This blog post didn't work for me because it was too vague. I already have old boring solutions for their python pain points, like pip-tools, so I can't take their assessment of how much better things can be on faith, and they don't show me the details of the new thing.)
> How exactly, command by command, would nix solve the same problems?
This might be problematic, because to understand these commands, you would have to understand Nix. It's not just a tool that replaces your existing tool, it's a total workflow change, and without understanding what's happening under the hood, you can't really appreciate how much it makes your life easier. I could show you a `nix-build -A deploy.somemachine`, but without understanding why it's called like that, and what exactly it does, and how I build most of that myself using nixpkgs tooling, you wouldn't really 'get' it, because you probably wouldn't understand how is that better than doing an `ansible-playbook -i foo.yml`, and I wouldn't be able to explain without explaining Nix/nixpkgs from scratch.
Nix, compared to other DevOps tools, is alien technology, and requires unlearning a lot of things, learning a whole new language and ecosystem, beginning to think in terms of that ecosystem and general programmability and functional programming, and then building on top of this to solve your problems. Nix doesn't immediately bring solutions to your problems, it brings you a new way of thinking about these problems and solving them in ways you didn't even consider before.
I know this is all handwavy, and that is unfortunate, but in my opinion unavoidable when dealing with radically different tooling. From my experience, the current 'classic' devops tooling (Docker, Terraform, ...) has reached a local maximum of usability. Getting out of that maximum requires a vastly different approach, and thus requires a much longer learning experience than what most people are used to. If you keep switching to different tools that behave how you expected them to, you're really just using the same thing as before, but with a different syntax.
>This might be problematic, because to understand these commands, you would have to understand Nix. It's not just a tool that replaces your existing tool, it's a total workflow change, and without understanding what's happening under the hood, you can't really appreciate how much it makes your life easier.
That's the entire point of a tutorial like GP describes. Yes, Nix works on a higher level of abstraction than existing tools, but it solves the same problem, and comparing it to "alien technology" doesn't help.
Nix is simply:
* a large corpus of work (nixpkgs; which contains build instructions for source code that can combine all the way to an operating system or a service image)
* organized in a particular way (using the Nix language; used to declare the expected results of all the different build strategies out there)
* and coupled to the tools that let you turn a description of a process into the result of that process (nix-build, etc).
This is abstract but hopefully less handwavy? The concrete details of the Nix paradigm really lie in the syntax itself so what you're saying about "same tools with different syntax" doesn't make much sense to me, as every programming tool is defined by its syntax.
The on-ramps to becoming fluent with the Nix language are, admittedly, subpar. I got nearly nothing out of "nix-pills" and cringe every time someone recommends that tutorial which takes 10 chapters to build your first package in the most roundabout way possible.
What Nix needs is a single page specifying what goes in a Nix derivation. Like the Compose YML reference but simpler. Also the nixpkgs manual is quite central but impenetrable for casual reading. Also how about someone writing a tutorial for generating a static HTML site with Nix, then deploying it, then incrementally adding dynamic backend services to it?
Right, I do not disagree with a tutorial that outlines Nix in this way. Indeed, this is how I teach Nix to others, generally by starting with the repl and then adding things on top like derivations, callpackage, nixpkgs and stdenv, trivial nixpkgs builders, and finally NixOS.
I'm arguing only the one point that I cited - that Nix's advantages can be presented as a step-by-step 'write these commands' getting started guide for migrating, I don't know, some ansible based production into NixOS. Following some steps blindly and expecting immediate results will only result in frustration and disappointment.
The 'alien technology' is in my experience quite helpful in getting expectations straight when teaching people Nix. Driving the point that this is not like switching from npm to yarn, but that you will have to spend a serious chunk of time learning a whole new thing.
> The concrete details of the Nix paradigm really lie in the syntax itself so what you're saying about "same tools with different syntax" doesn't make much sense to me, as every programming tool is defined by its syntax.
There is more to programming languages than their syntax. But maybe I couldn't have been more clear. What I meant, is when you're comparing Prolog, Ruby and Python, Ruby and Python basically look like the same language. In the same vein, when you're comparing nix{,pkgs,os}, Ansible and Puppet, Ansible and Puppet basically look like the same tool. Switching from Ansible to Puppet is relatively easy, while switching from Ansible to Nix requires switching your mindset and approach considerably.
> What Nix needs is a single page specifying what goes in a Nix derivation. [...] [A] tutorial for generating a static HTML site with Nix, then deploying it, then incrementally adding dynamic backend services to it?
All of these are good ideas in my opinion. Nix needs better documentation, but the answer isn't more "build production on Nix in 5 minutes" guides, but more "go on an adventure to understand Nix thoroughly and make using it obvious" guides, IMO.
So nix is kind of weird thing, because parts of it you can use to replace part of components but hen you can create a whole solution (and ultimately that works better). In that case it's like asking to write a tutorial for python.
To illustrate what I mean here is my history with nix (simplified, but it's meant to show the evolution):
So I started using nix as a reproductive dev environment. Initially using shell.nix to have all my tooling. This is great if you also have direnv installed, which basically automatically makes tools related to the project to appear.
Anyway I started then using various tools that would translate python project to nix initially was also a big fan of setup.py/cfg and pip-tools. I wasn't happy with either of them until I found poetry2nix. Poetry is also quite decent so now that's my favorite way to build a project.
So what poetry2nix does is translates on the fly pyproject.toml and poetry.lock file to nix expression. With it you can build project in nix you can also create a shell that has python with all packages available as described by poetry. So that's now quite decent reproducible environment.
I realized that to make the dev environment nice I'm adding a lot of nix boilerplate to every project, also as I'm learning new things and improving it it is getting hard to synchronize that boilerplate across other projects.
So I created a new repo where I put everything there, I also learned how to use modules system (which they created for NixOS) ultimately I made repo that you reference in default.nix and shell.nix in my project it loads project.nix file which describes information about my project. Allows me to set various aspect of the application, like what additional packages should be installed in dev shell (maybe I need a local postgresql installation). How the project will be deployed (if it is meant to run as AWS lambda, a serverless (npm application) is added and package.json/package-lock.json files are sourced. If it is a docker asks what binary should be executed in the container. Whether to compile using minified python (with missing some of core libraries) etc.
That ended to be some Nix code, but simplified things greatly, and makes things usable by people not familiar with Nix.
We use gitlab for CI/CD so I created a special gitlab-runner for nix builds. It actually is already built in in NixOS and only needs to be enabled, and they also show configuration in example. Basically how it works is that it stills spins a docker container for the build, but exposes /nix/store (read only) and the unix socket to trigger build. I also created an S3 bucket and configured it for storing caches of builds.
Having this runner lets me utilize one of major benefits of Nix which is caching (if all inputs (source files, architecture, interpreters etc) don't change then output will be the same). This is great, because if I only change source file, and don't change dependencies, the pipeline will only rebuild my app. If I work on a branch and branch passes all tests, I merge it and merge is just fast forward, then build job in master branch just takes couple seconds, because nothing change. If the branch wasn't rebased and actual merge was done and some files are now different the nix will rebuild things that were modified.
Deployment of my python app as a lambda ends up being just:
nix-shell --run "sls deploy"
nix-shell takes care that the dev environment is created and inside of it invokes "sls deploy" command. This similarly utilizes caching (that persists between builds) so it is faster than what we normally used like loading a docker container and then running it.
This is quite decent and I'm still testing it on a single app, but it basically cut deployment time from 10 minutes to 5 minutes. And I know I can cut it further (serverless doesn't know about nix, but it understands poetry, so once again again it downloads all dependencies it needs, extracts them and bundles them into the lambda. I plan to populate its cache from the packages that nix stores when is running the build).
There is also possibility that I'm aware of, but haven't explored yet, but seems interesting. It might require some work. For example you could configure your local computer to use the machine that is used for CI/CD build to help with local builds. Similarly you could expose the cache as well to prevent unnecessary rebuilds. You probably shouldn't do that if the user isn't trustworthy, otherwise you might need to create some additional safety layer.
Now in your scenario you have bare bone servers to deploy. If those would run NixOS I believe the deployment step would be simply to deliver new configuration.nix referencing your application to them and also have those machines configured to use the same cache as you had with CI/CD so they won't have to rebuild everything again. There are several tools that make the process easier, for people who use public cloud there's terraform support, NixOps (last time I used it was the weakest part of Nix, but maybe it improved, it was great tool when I wanted to test everything a la hashicorp's vagrant)
I don't think it fits the tutorials you're creating on nix.dev, but I would love a deep dive how to use module system that nixpkgs provides. What I mean is how to use it to create my own solution. For example if I would create own configuration file like flake.nix or configuration.nix how could I process it. I think I figured it out, but I'm still not certain if I'm using it correctly.
