/usr/bin/env is required by POSIX, but bash's location already varies across distros and operating systems. NixOS's choices there are fundamentally correct.
Since shebangs have no native way to look up binaries from your PATH scripts should be including the full path to the binary from /nix in the shebang (and they should stop opening PRs asking people to switch to /usr/bin/env for them)
>but bash's location already varies across distros
It almost always exists in both /bin/bash and /usr/bin/bash via a symlink.
>NixOS's choices there are fundamentally correct
I suspect the choice has more to do with that they don't want to do the work needed to remove /usr/bin/env.
And one of its main uses in practice today is to perform lookup via PATH in shebangs, since it has to do that kind of lookup to be able to launch programs with modified environments as given in those examples anyhow. It obviously can't perform that common function if applications using it can't find it, so it's in an exceptional position relative to other utilities.
NixOS just makes the correct tradeoff here for `env` to be able to perform one of its main functions by putting it in a conventional location.
> Since shebangs have no native way to look up binaries from your PATH scripts should be including the full path to the binary from /nix in the shebang (and they should stop opening PRs asking people to switch to /usr/bin/env for them)
Nixpkgs already has facilities to handle this for scripts that are packaged via Nix. Those shebangs matter only when (a) some impurity is actually desired or (b) the scripts are used/managed out-of-band.
The issue also doesn't only affect NixOS.
> It almost always exists in both /bin/bash and /usr/bin/bash via a symlink.
What if I don't want to use the system bash (ever) because it's old? If someone isn't shipping their own bash or targeting a specific version that they know comes with a target platform very deliberately, why should they point to the system bash?
>NixOS just makes the correct tradeoff here for `env` to be able to perform one of its main functions by putting it in a conventional location.
At that point you might as well do the same for sh and bash as it would also save a lot of unnecessary headache. Shebangs are not a necessary part of a distro. You can always call the interpreter directly and pass in the script.
I personally find the idea of disrupting the normal FHS for purity to be a cool idea, but if you are going to ignore that goal and compromise to increase compatibility with existing scripts I feel you might as well compromise that purity more to support bash as that will solve the users of the OS even more problems.
>What if I don't want to use the system bash (ever) because it's old?
When you are sharing a bash script with others you typically aren't going to want to use bleeding edge features. I don't see that as a big problem. For your own scripts you can always use the full path to whatever version you want.
> No, it isn't. The required directories are /, /dev, /tmp. The reqired files are /dev/null, /dev/tty, and /dev/console.
To finish splitting the hair: the env utility is specified by POSIX (it also specifies sh, and not bash), but it doesn't specify the path of any standard utility.
> Since shebangs have no native way to look up binaries from your PATH scripts should be including the full path to the binary from /nix in the shebang
Weird hill. Shebangs are a "native" way to tell the ~system what interpreter to use, and optionally what arguments to pass to it. Can you clarify how this argument doesn't self-reduce to absurdity? Why should the distinction you're drawing apply to the "non-native" behavior of env and not to the non-native behavior of every other external interpreter and executable?
> (and they should stop opening PRs asking people to switch to /usr/bin/env for them)
Again--weird hill. I won't pretend the Nix community isn't a big source of these requests, but your sample may be skewed if you think Nix is the only source of these requests.
For example, a narrow GH search (("/usr/bin/env bash" AND "/bin/bash" AND nix) -org:NixOS -org:nix-community) focused just on bash only gives me 429 issues and 283 PRs that smell nix-motivated (and some are references to "*NIX"). A similar search for macOS (("/usr/bin/env bash" AND "/bin/bash" AND macOS AND NOT nix AND NOT nixos) -org:NixOS -org:nix-community) turns up 1.8k issues and 1k PRs.
> I suspect
No real need to speculate--https://github.com/NixOS/nixpkgs/commit/7281a851b3594 makes it somewhat clear that this is a convenience for end users that may want to use (or write) scripts ~can run on other NixOS and other distros.
(There are definitely people in the community who'd be happy to remove both /bin/sh and /usr/bin/env. Both of these are even nullable options, though I don't think I've heard of them being used outside of the envfs module in nixpkgs and the FHS module in https://github.com/balsoft/nixos-fhs-compat.)
> they don't want to do the work needed to remove /usr/bin/env.
Remove it from where? These are already extensively patched out in nixpkgs. If that's the work you're talking about, it gets done all the time.
>Can you clarify how this argument doesn't self-reduce to absurdity?
I think you misunderstood me. My comment about being native is that Linux does not look at your path when handling the shebang. Since POSIX doesn't standardize where env is there is no way to know how to launch env ahead of time. If there is no way to write a generic shebang you might as well just accept that you will need to change it.
>but your sample may be skewed if you think Nix is the only source of these requests.
Fair point. I am also biased from PRs also trying to insert flake.nix into projects. My issue is that nix is doing something nonstandard and it feels like projects are having to accommodate nix's oddities instead of nix being the one to accommodate what already exists.
>makes it somewhat clear that this is a convenience for end users that may want to use (or write) scriptsz
Thanks for sharing this. I always assumed they had it from the beginning.
>Remove it from where
Not all instances of it in build scripts are properly rewritten. Also there would be work in creating a migration plan to not breaking people who currently rely on its existence.
> I think you misunderstood me. My comment about being native is that Linux does not look at your path when handling the shebang.
I understood your comment. Maybe we're just doing a prescriptivist/descriptivist thing.
The shebang is a good example of a Unixy affordance with creative applications, and I asked for clarification because I (a descriptivist :) don't see why any specific functionality enabled by this affordance is more or less valid.
> Since POSIX doesn't standardize where env is there is no way to know how to launch env ahead of time.
In a fundamental sense, it's impossible to be certain that code written to run on one system will run on any ~other (including the one it was written on, since it may have mutated it in incompatible ways as soon as execution starts). We can try to trust POSIX, but most distros enable users to mutate the system in ways that even it can't protect us from.
> If there is no way to write a generic shebang you might as well just accept that you will need to change it.
I'm of (at least) two minds about this. It's obviously correct that there's no way to write a generic shebang that will work everywhere. But I think it's also obviously a poor use of human time and attention, at scale, for us to all individually waste time ~localizing every third-party script for every system we ever run it on (perhaps repeatedly, as the system changes). The impossibility of a 100% solution isn't a good reason to shun 90% solutions.
(I say this as someone who develops a tool, https://github.com/abathur/resholve, for ~localizing bash/shell scripts--shebang included. In some sense my ego could benefit from end users needing a tool for this, but the task is inherently fraught on stateful systems and is probably best addressed at the package-manager level.)
> Fair point. I am also biased from PRs also trying to insert flake.nix into projects.
Just as fair. They mean well (and generally just want to make the project easier to distribute or contribute to), but they can be a bit ~deaf to reasonable concerns about adding to the maintenance burden.
> My issue is that nix is doing something nonstandard and it feels like projects are having to accommodate nix's oddities instead of nix being the one to accommodate what already exists.
Also fair, though I hope you'll keep your eyes open for cases where people trying to package a project with Nix are shaking loose portability/reproducibility/underspecification problems that are already latent in the project and are worth fixing even if you ultimately reject the .nix file. (This won't be everything. There are certainly cases where Nix is the only one needing the patch and should do the work to keep up with it.)
Anecdotal, but it's reasonably common to run into projects with hardcoded paths that leave the build/test processes working cleanly on exactly one person's mutable system without anything anywhere saying what that system is or how they bootstrap it. I've seen this include ~snapshotted test outputs that will only pass on the authors system because it's affected by files or even dotfiles that may be in the project directory but not committed, or higher up in the directory tree on the system.
This kind of stuff is just as likely to trip up non-Nix contributors. It may also be a big hurdle to the broader OSS community being able to pick up the torch for them if they get burned out, fall ill, change careers, etc.
(Sometimes maintainers are standoffish about fixing these, especially if they distribute their own binaries or directly push artifacts out to package repositories and so on. They may not understand why we want to build from source or run their test suite or build scripts or build the manual. I'm sympathetic on some level, but I must admit I'm flummoxed when people seem actively incurious about reports that suggest they might have trouble building the project if they lost access to their device).
> Not all instances of it in build scripts are properly rewritten.
Any example? There are definitely asterisks (https://github.com/NixOS/nixpkgs/blob/a4d0fe7270cc03eeb1aba4...), and the default behavior here is one driver for people going out to ask for `/usr/bin/env` (because those can get autopatched). But we do still have the tools to patch them. It isn't the end of the world if you tell us to kick rocks.
> Also there would be work in creating a migration plan to not breaking people who currently rely on its existence.
Yes (though, perhaps ironically, the closest I have to a pitch for Nix is what an incredible lever nix/nixpkgs/NixOS are for experimenting with the feasibility of ideas like this at scale).
It was too long ago since I contributed to nixpgs or used nix for me to remember any specific examples. It may have just been with something I was personally packaging.
