Having used nix for two years now for both work and personal purposes, I agree. Though I have also come to think nix is doing things in much better ways than AppImage, Docker, Snap, Flatpack, and others.

I won't focus on the good parts, but focus on your comment.

Nix has three things that I think are confusing and took me an embarrassing long time to grok, and I find other people confused by as well.

1. Where does the semicolon go? Reading code makes it feel arbitrary, but really the only time a semicolon is used is to terminate an assignment (=). If there are no equal signs, then you don't need any semicolons.

2. The heredoc strings ('' '') are in fact strings. Nixpkgs passes these strings to a shell quite frequently and their docs also make it feel like these statements are shell scripts and not strings. The problem here is that to pull a variable from nix you use ${VAR} and to use an environment variable you use $VAR. When you look at these heredocs it looks like a pure bash script, but it doesn't quickly jump out at you that ${VAR} are being templated in by nix. It also isn't obvious that $VAR is not being templated in by nix.

3. The nix language is very very tiny, and almost everything comes from nixpkgs which is basically stdlib as well as the package repository. Almost everything, including callPackage is in nixpkgs. The documentation is fairly decent, but when trying to figure out how do something, you are basically hunting in that code base.

Bonus 4. In nix it is really easy to build single language applications. However if you need to mix two, finding good docs or examples is really hard. IE, if you want to build the javascript front end of your web app as well as the python backend. These combinations need at least two derivations, and how to make it happen is bespoke every time.

Regular Nix user here, I agree with your points regarding the language's peculiarities, and adding on a few points:

> 1. Where does the semicolon go? Reading code makes it feel arbitrary, but really the only time a semicolon is used is to terminate an assignment (=). If there are no equal signs, then you don't need any semicolons.

The semicolon is also required when using with expressions (i.e. augments the environment of an expression like let, but confusingly doesn't shadow let bindings)

  nix-repl> with { a = 3; }; a
  3
  nix-repl> let a = 5; in with { a = 3; }; a    
  5 # ??????

> it it doesn't quickly jump out at you that ${VAR} are being templated in by nix

One situation I've run into is how to quote ${ in a multiline string, and the solution is to write ''${ :

  nix-repl> '' ''${ ''
  "${ "

> The documentation is fairly decent, but when trying to figure out how do something, you are basically hunting in that code base.

This is very much true, essentially Nix and its ecosystem is a big instance of Hyrum's Law[0] whereby people go with what they actually observe rather than what should be according to the spec (if it exists).

[0] https://www.hyrumslaw.com/

`with` should be considered harmful.

Well, you recognized the gotcha, but there are people who don't, they inserted a bug into nixpkgs, then someone else fixed it not realizing its genesis. There is a lot of spaghetti code in nixpkgs caused by misunderstanding of `with`

> Bonus 4. In nix it is really easy to build single language applications. However if you need to mix two, finding good docs or examples is really hard. IE, if you want to build the javascript front end of your web app as well as the python backend. These combinations need at least two derivations, and how to make it happen is bespoke every time.

For anything that doesn't fit a common nixpkgs function (like `buildPythonPackage`, etc.) then I always start with this:

    nixpkgs.runCommand "my-application"
      {
        dependencies go here
      }
      ''
        shell commands go here
      ''

This is certainly "bespoke every time", but it's literally just a shell script (with explicit dependencies). This approach doesn't need much documentation either: we just copy-paste whatever commands are in the normal, non-Nix documentation for whatever we're doing. We'll probably need to add a few dependencies to the `buildInputs`, and possibly disable the sandbox to allow network access (which we can factor out later, if we like).

Thanks for the example - I’ve been looking for how to make mix packaging easy.

How do you add files to this build? For example, if I have this program.rb that depends on ruby:

    puts “Hello World”

And then you add a file called package.nix

    let pkgs = import <nixpkgs> {}; in
    pkgs.runCommand "my-app"
      {
        buildInputs = [pkgs.ruby];
      }
      ''
      pwd && ls;
      ruby ./program.rb
      ''

How do you get your files in there - like “ADD” in a Dockerfile?

You can splice file paths directly into the script, like this:

    let pkgs = import <nixpkgs> {}; in
    pkgs.runCommand "my-app"
      {
        buildInputs = [pkgs.ruby];
      }
      ''
        pwd && ls
        ruby ${./program.rb}
      ''

Or you could add it as an env var:

    let pkgs = import <nixpkgs> {}; in
    pkgs.runCommand "my-app"
      {
        buildInputs = [pkgs.ruby];
        program = ./program.rb;
      }
      ''
        pwd && ls;
        ruby "$program"
      ''

example in the manual: https://nixos.org/manual/nixpkgs/stable/#chap-trivial-builde... shows how it creates a file `$out/message`.

If you want to write to nix store with stdenv use $out.

That is a gross oversimplification.

An javascript front end can have thousands of dependencies, and the nix build environment does not have network access unless it's computing a fixed output.

To do that manually would be a nightmare, so you must automated either with tools you write or tools someone else has written.

Then you still have to figure out how to combine it all together.

Also on allowing network access in the sandbox requires elevated privileges which means you cannot do that method on environments you don't have full control. And it would create builds that are likely not reproducible.

> That is a gross oversimplification

It's really not. I've used this successfully for Javascript projects (running npm and yarn), Python projects (running pip install), Scala projects (running sbt), Haskell projects (running cabal), Racket projects (running raco), Go projects (running go get), etc.

It's pretty much like writing a Makefile, except we have to specify the buildInputs, and there are occasional annoyances like tools trying to write to $HOME (requiring an `export HOME="$PWD"` line at the top of the script).

> the nix build environment does not have network access unless it's computing a fixed output

Yes it does, if we avoid the sandbox. This used to be done by specifying a `__noChroot = true;` attribute; I don't know if that's changed in more recent Nix versions.

> Also on allowing network access in the sandbox requires elevated privileges

Sandboxing is a sensible default, but for practicality I usually set it to "relaxed" on NixOS and disable it entirely on macOS (since that often uses stuff from the host system).

> which means you cannot do that method on environments you don't have full control.

Could you give an example of where that would ever be an issue? The only thing I can think of is hydra.nixos.org, but there's no way in hell such a derivation would be acceptable for nixpkgs, so that's pretty irrelevant (similar to how it's really easy to make a .deb package from a directory and a control file; but there's no way in hell it complies with the Debian project's packaging guidelines)

> And it would create builds that are likely not reproducible.

Yes, that's why I mentioned we might want to factor out the input-fetching at some point.

> Then you still have to figure out how to combine it all together.

That has nothing to do with Nix. As long as a project has a build script, or a Makefile, or a list of commands in a README, or whatever, then we can plug it into that template and see if it works. Are you complaining that the Nix package manager doesn't have specific documentation for how to build every random application ever written?

>> which means you cannot do that method on environments you don't have full control.

> Could you give an example of where that would ever be an issue? The only thing I can think of is hydra.nixos.org, but there's no way in hell such a derivation would be acceptable for nixpkgs, so that's pretty irrelevant (similar to how it's really easy to make a .deb package from a directory and a control file; but there's no way in hell it complies with the Debian project's packaging guidelines)

In any production environment, using any external build service like nixbuild.net, self-host hydra. Basically anything beyond personal use.

The post about all this is from Channable, Channable runs nix built code in production. Their developers (likely) cannot disable the sandbox and ship that code.

You even say that there is "no way in hell such a derivation would be acceptable for nixpkgs"

If you cannot share it, it's only good for you and you alone.

> production environment

> in production

Lol. If someone doesn't want to be executing random unverified binaries fetched from arbitrary online locations, then they shouldn't write such things in their build scripts.

That has nothing to do with Nix. The same goes for Makefile, or whatever.

Regarding point 2, yes it's definitely a pain to mentally switch back-and-forth when writing shell code in a Nix string.

Three things I find immensely helpful:

- Syntax highlighting (I use Emacs, which has a Nix mode)

- Syntax checking (again I use Emacs for this)

- Writing complicated scripts in a separate file. This makes it harder to splice in Nix values (there are ways to substitute in text, but I find them more trouble than they're worth). Setting some extra env vars in the build environment is usually the easiest approach.

> the only time a semicolon is used is to terminate an assignment (=)

I find it easier to think of "attribute sets" like `{ foo = bar; }` more like a JSON objects (e.g. `{ "foo": bar }`) rather than "assignment".

Nix does have a `let` syntax, which probably looks more like assignment, but I've never used it in the 7 years I've been using Nix. I find `with myAttrs; myExpr` to be far better (except for the WontFix issue that `with` doesn't shadow function arguments).

Off the top of my head, the following expressions also require `;`:

    with myAttrs; myBody
    assert myCondition; myBody

The way I remember this is that every Nix expression returns a value; yet `with myAttrs` doesn't have a value, and neither does `assert myCondition`. Hence they have a separate return value "stuck on" via the `;`

> I find `with myAttrs; myExpr`

To make it clearer, he is saying that he does

with { foo = "bar"; }; <some other block>

instead of

let foo = "bar"; in <some other block>

The differences here so other people can follow along.

1. with will not shadow variables defined higher up. This can be either desirable or very confusing.

2. let is recursive by default, meaning you can reference other variables in that block. This behavior can be mimicked with `with rec {}`

Yep. The advantage of `with` is that we're dealing with first-class values (attribute sets, like `{ foo = "bar"; }`), rather than an implicit environment/context. My rationale is at https://github.com/NixOS/nix/issues/1361#issuecomment-390420...

The thing about shadowing is that `with` will not shadow anything that's in the implicit context, which includes things defined by `let` (hence why I avoid it) and function arguments (unavoidable). We can shadow other `with` bindings as much as we like, e.g.

    with { foo = 1; };
    with { foo = 2; };
    with { foo = 3; };
    with { bar = with { foo = 4; }; foo; };
    [ foo bar ]

This evaluates to [ 3 4 ], since later bindings to `foo` will shadow/override earlier ones.

This doesn't work for bindings that are in the implicit environment/context, for example:

    with {
      f = foo: with { foo = 42; }; foo;
    };
    f 123

This evaluates to 123, since the binding `with { foo = 42; }; ...` doesn't shadow/override the argument `foo`.

The rationale for this behaviour is to avoid changes made to an attrset from overriding explicit arguments. For example:

    foo:
      with bar;
      x + foo

This is presumably meant to behave like `foo: bar.x + foo`, but a future change to `bar` might include a `bar.foo` attribute, in which case we don't want the above to change behaviour to `foo: bar.x + bar.foo`.

I understand this rationale, especially when using huge, constantly-changing attrsets like `with nixpkgs; ...`, but it's still annoying ;)

I am of a similar opinion, but I think it's largely due to my typical use case.

If you use Nix in a "drive-by" kind of way, it's very hard to make any inroads into the language. The documentation and tutorials are not really intended for a casual user - it is expected you will sit down with the Nix site like a good book, and go at it from start to finish. If (like me) you instead touch Nix little and often, doing small invocations or on-the-fly editing, the docs are much less useful.

There's also no way to understand it in terms of a few simple concepts for beginners. Do you want an override, an overlay, a flake, or something else? Does specifying an option in multiple places combine them, or does one spec override the other? It feels as though Nix is a technology of special cases, and the docs don't make it easy to understand what to do in each case when all you know is the kind of change you want to make.

The worst culprit of this is NixOS services, which each declare an ad-hoc API that I normally find myself digging into the service declaration file in the Nix repo to try and understand.

> If you use Nix in a "drive-by" kind of way, it's very hard to make any inroads into the language. The documentation and tutorials are not really intended for a casual user - it is expected you will sit down with the Nix site like a good book, and go at it from start to finish. If (like me) you instead touch Nix little and often, doing small invocations or on-the-fly editing, the docs are much less useful.

This is super serious, major, I hope the devs are paying attention.

Most users of anything are drive-by, rather casual users. Think of any big tech out there: they became popular precisely because they were easy to get started (at least back when they were launched/became popular): C, Unix, Bash, Perl, PHP, MySQL, Javascript, Java, Ruby, Go, etc. Almost everything that's big was usable by people don't read an entire book before they can make "Hello world".

> Most users of anything are drive-by, rather casual users.

On top of this, Nix could be extremely well suited to casual users. It inherently avoids most of the problems that plague casual users of traditional package management tools.

Unfortunately, most of the more serious Nix contributores I have interacted with are in too deep to see the usability problems. Even worse, usability issues get trivialized, and users who ask for basic improvements get talked down and told they just aren't getting the magic of Nix.

For example, the topic of versions has been argued about for over five years: https://github.com/NixOS/nixpkgs/issues/9682

To agree with you:

I've used nix for a few projects over the past couple of years, always because I wouldn't have done the project without it (or I would have had to learn a series of separate solutions). I have never learnt nix, except to get it to do what I want.

When I started, it was full on "what is the magical incantation to make it do what I want?" mode. Now I am usually capable of saying "to do X I need to do Y; what is the documentation for Y?", and when I find the documentation for Y it usually conforms to my presuppositions and merely adds detail and color.

This is a really weird feeling. The idea that Nix/nixpkgs makes sense is so shocking to the core of my being. But it is still difficult even to find the documentation I want. But nix does give me another feeling that I like: confidence that I am building a system.

There are some serious gotchas like nix-env in this read I would like to delete. But, that said, I don't think targeting "drive by usesr" is worth it.

The benefits of Nix is that it dramatically organizes the absolute disgusting mess that is how most people manage their computer. I don't want to drive people trying things out away, but Nix will never be worth it unless you embrace it, and half-ways usage will deliver all the downsides without enough of the upsides.

Not every value function is monotonic like that, I am not going to pretend it's otherwise.

The goal is to help people fully embrace it, not to allow them to comfortably come to a stand-still half way.

I've used NixOS at home since like 2016. While I haven't used it professionally or for "serious" enthusiast deployments, I don't think I can claim to be a drive-by user. I still insist on using nix-env because it's what all the first-party manuals tell me to do and I don't want to get into a situation where I am experiencing huge pain and the first-party people all tell me "well, that's on you for doing things in a different way".

At least my impression from a while ago is that the actual documentation Nix has seems to be aimed at "drive-by" or at least casusal/entry-level users, and to get the deeper insights that make the whole thing enjoyable, you need to give up on the documentation and just absorb all the details from github discussions, through extensive trial&error, etc.

Yeah people say the docs are unapproachable, but I agree the docs are too much "here is how to do a bunch of random things" and not enough systematically describing what Nix is.

Reference first, then tutorials.

That's not how humans work.

Good tutorials first, reference second.

Organizing that mess should make every user a casual user, serious it not?

hmm? Genuinely unsure what you mean.

The word serious was meant to be should. My phone keyboard picked the wrong word, and I waited too long to edit.

By "services" do you mean the ones you find on options page[1]? If yes, I never needed to dive into nixpkgs repo since that link provides every option + their expected input types.

[1]: https://search.nixos.org/options

I still don't understand the language syntax after using it for several years, I just make it work through trial an error. The documentation is pretty unhelpful too. It's ultra ambiguous.

Recently my laptop died, and I just needed another one going in a hurry and I tried Xubuntu because it's the only distribution which had Wifi drivers that worked for a 12 year old Mac I had lying around without any trickery.

It's funny because Nix worked fine, but my Ubuntu installation just stays out of my way and lets me work too. I started to make me question whether all the added Nix complexity really added much value. Would investing in learning Nix really pay off over spending my time learning a new programming language with real world application?

I also think there is something to be said for the fact that, unless you team mates are using Nix,good luck using any of the standard tooling or scripts others will use. One of my favorites are shell scripts that start with `#!/bin/bash` never working and always having to patch them to use `#!/usr/bin/env bash` I understand the latter is the correct way to do things in 2021, but we all know it's not going to happen :)

I switched to NixOS after tripping over my laptop's power chord during an update, which broke Linux in such horrifying ways that I didn't dare attempt recovery.

I agree that I'm not sure the extra complexity is worth it for day to day use, but just the ability to avoid catastrophic failure and near instant setup on new machines is worth it I think.

Many times repairing those kinds of situations in Linux is as simple as:

- boot linux livecd

- mount your broken filesystem

- bind mount the important bits from the live kernel (/proc)

- chroot

Like this:

    cd /

    mount -t ext2 /dev/sda1 /mnt #Here's your broken install

    mount -t proc proc /mnt/proc

    mount -t sysfs sys /mnt/sys

    mount -o bind /dev /mnt/dev



   chroot /mnt /bin/bash #boom you are in.

Now do whatever you can to repair the system. depends on what broke.

usually apt stuff for me(pulled power during a package upgrade)?

>Now do whatever you can to repair the system. depends on what broke.

What needs to be done to fix the system may not be obvious and the fix may not be simple. Furthermore, the trial and error for fixing may leave a trail of new issues.

And that is why I use NixOS. I have to wrestle with it to get something working, but once I'm done I can guarantee it will keep working until the heat death of universe. Compared to that, Ubuntu and others let me get off the land fast but I never know when I'll crash or if I'll able to fly back from it.

I just use ZFS filesystem w/ snapshot before update

Or you could, you know, use any kind of backup/restore software, spend like 5 minutes to learn how to use it, and use those many hours invested in learning Nix to watch football and drink beer :-)

I feel like restoring a backup is a lot more disruptive than the update just not happening and you being back where you started automatically.

For a catastrophic event that maybe happens once per decade?

https://xkcd.com/1205/

Actually, I found the language to be pretty similar to Haskell, if you do know that. You don’t have explicit functions, you use let in, and everything is an exception. It is even lazy evaluating.

The lack of lazy evaluating is an issue as the Nixpkg grows. I don't believe anyone has a solution to this yet.

Could you elaborate what you mean by this? Nix has lazy evaluation so as Nixpkgs grows it doesn't perceptibly change how long package installations, building, etc. take.

When I run `nixos-rebuild switch --verbose`, it does seem to go through the entirety of nixpkgs. Especially noticeable on a RPi with its slow I/O.

It's not true that it pulls in everything. It's just that a nixos installation pulls in a lot of transitive code =). It actually pulls all of this in lazily and lazily evaluates it; but it's just a lot of code and nix evaluator isn't that fast.

That’s just evaluation. The next bigger change in nix itself will likely be the ability to better cache evaluations, significantly speeding up this.

I’m not sure what you mean by lack of lazy evaluation. It is lazy as far as I know.

Also, check out flakes, which somewhat steers away from the central repo design.

I agree. I’ve spent person-weeks looking into Nix, and I still can’t get it to work beyond the most basic of use cases. A few weeks ago I spent an entire weekend on the Nix discord channel trying to get help to configure VSCode for Rust development, and we collectively couldn’t figure it out.

Nix is a great idea, and it’s goals are ambitious, but I haven’t found it to be usable yet. Almost any time I try to do anything I get mired in a tar pit and I basically end up having to give up. I’ve learned lots of tools and technologies in my career, but Nix remains elusive.

I did a month, diving directly into the deep end with flakes and all. I don't know, it really is hard in the beginning. Like, really hard. But eventually I got myself a setup I could use in my two laptops and workstation. A setup, that sets my home directory, all my programs and my custom desktop just the way I want. Everything is in the github repo, and installing with the flake will give me the exact experience I have in my other machines.

I tend to use lots of custom tools and commands, that are really painful to install and setup for a new machine. With NixOS all of it is just one command away.

But, I agree, it is REALLY HARD in the beginning to grasp things.

Here's my configs if you want to see how I approached my own setup: https://github.com/pimeys/nixos

"It's hard to grasp things" is only one part of the problem. I mentioned that there's no good way to understand nixpkgs except to manually traverse the file tree. Grokking the directory scheme will help a bit, but there's no amount of Nix Nirvana that will make it as easy as hovering over the parameter in VS Code.

In other cases, things are just broken--my nix installed rustup is failing to build packages this morning with a clang linker error (can't find iconv) and Googling the error message + 'rust' indicates that this problem is only encountered in Nix environments. Similarly, many of the other problems are that it requires you to repackage other libraries from the ground up. For example, most Python packages are available in Pypi such that users can just `pip install` them. Nixpkgs has a `pypiPackage` function that allows us to import a package from Pypi, but if that package has C dependencies (and many do, because Python's performance sucks and Python's performance sucks because everything depends on C packages precluding optimizations to the interpreter, but that's a gripe for another day) then Nix requires you to package those C dependencies in Nix first, and because every C package has its own bespoke build system with implicit dependencies, you must be expert in each individual C package in the dependency tree. Again, there's no amount of Nix Nirvana that will save you from this.

In many other cases (e.g., my 'packaging vs code with my rust plugins' example from my earlier post) we have to know the specific conventions for specific nixpkgs functions which will vary from others, and even that might not be enough. These conventions can't be derived from any general understanding or intuition of Nix (i.e., "Nix Nirvana") although it certainly doesn't hurt.

Further, NixOS is easy mode. It's the first class citizen in the Nix ecosystem. Other platforms don't work so well. In my vs code + rust plugins example, there were several posts documenting the process on NixOS, but for whatever reason Nix on other platforms (e.g., macOS) requires different configuration which is undocumented and even proficient Nix users couldn't figure out the issue.

But yeah, to your point, it is really hard to grasp as well, which exacerbates all of the above problems.

I get your problems, and I agree with you this is not always great.

I'm mostly writing Rust, so as long as I have `libssl` available, I don't really need to tweak my system that much to be able to work. I definitely would reconsider, if my work would involve using Python...

Although, I'm kind of interested about the challenge of understanding the packaging of these dependencies you're talking about. Making the whole compilation setup nice and clean with nix. But, I don't know, I get it it's not for everybody. And it also might be these lockdowns that put me to this mode of doing stuff such as tweaking NixOS.

Then, again, computers are my hobby. And new challenge is always accepted!

Do you have some examples of that? Maybe a link to discord discussion?

I think not only documentation but a complete UX overhaul.

The examples you find for creating a shell.nix usually work individually, but if you need a combination of two nix files there seem to be too many ways to do things, because I find it almost always impossible to combine two examples into one.

Also for searching packages I always end up looking on the website. Using the built-in search you have to do some weird incantations that aren't easy to remember and you still don't get the full list of packages that are related.

It also doesn't help that the package names are random. There's no standard format it seems: gcc 9 is "gcc9" while clang 9 is "clang_9". Why not have packages be <namespace>.<package>.<major[.minor[.build]]> or any other standard convention.

Then you could also say you want a certain version of a package instead of having a name for each. Or a minimum version.

Maybe we need a wrapper around nix that exposes a user friendly interface. I really like the basic idea.

I wrote a language tutorial for only the language a while back, and have gotten the feedback that it has helped a lot of people - maybe it'd clear something up for you: https://github.com/tazjin/nix-1p

What took me a long time to understand is that there are different types of nix expressions (e.g. nix modules). For a long time I thought they are all the same. I hope nix flakes fixes the situation when they are released/stable.

Did you follow the language tutorial in the manual or something else?

I would have followed the official manual. I see that you are putting together some tutorials, so I will definitely check those out.