> What is the story in nix for packaging untagged branches of software
Most Nix packages define their sources using `src`. You can pass in whatever you want. E.g. fetch a tarball from GitHub with `fetchUrl`, fetch from a git repository, or point to a local directory.
> or reasoning about "snapshots" where pools of unreleased repos/packages are able to be treated as a single versionable unit?
This is basically what the Nixpkgs collection is minus the "unreleased" part. You can extend Nixpkgs with the packages that you care about using overlays. You have control over the stuff that you put in the overlay, so you can put in unreleased software as well like I described above.
> - I have an existing system for managing packaging metadata which I don't want to migrate from. How much trouble will I get into if I want to generate the metadata on the fly each time (as I currently do for my debs)?
This is difficult to answer without knowing more details.
- You can generate the Nix source files based on the metadata like the sibling comment pointed out.
- Or you could read the metadata with `builtins.fromJSON` and generate Nix derivations programmatically from within Nix.
> - What is the apt/nix interop story?
If you don't want to bother packaging certain applications using Nix (or work towards that gradually), you can. Nothing prevents you from referencing things outside of `/nix/store` in a Nix package. You can launch stuff from `/usr/bin` from a Nix binary if you are so inclined. Libraries are going to be tricky though.
As for the other way around, I'm aware of two options:
- At Channable, we would package the Nix closure of a package in a `.deb`. This works, but you run into trouble when multiple debs need the same store path. You can work around this by packaging the Nix closure under e.g. `/var/lib/<your_package>/nix/store` and bind mounting to `/nix/store` before launching.
- Eventually we realized we didn't want the `/nix/store` stuff in our `.deb` packages. IIRC we created a `postinst` script which ran `nix-env` to realize a store path from our cache + create a GC root so it doesn't get garbage collected.
Some of that stuff may have changed since I left.
Feel free to contact me if you'd like to discuss further. Info is in my profile.
The federated nature of the ecosystem has led to a culture where it's very normal to be building dozens of these at once, in the same workspace together, often from multiple repos (the repo above has four in it). So there are several build tools which automate the work of examining a source workspace and building all the packages within it in the correct topological order, in parallel, while respecting build_depend tags. The newest of these tools (colcon) has actually made the package.xml optional in many cases, as it can examine CMakelists, setup.py, BUILD, etc, and discover for itself what the dependencies are.
Anyway, so the existing ROS/nix efforts (1) seem to basically consume all of this package/distribution metadata at once and generate a giant parallel structure of nix definitions (eg https://github.com/lopsided98/nix-ros-overlay/blob/master/di...), which I fear would be completely opaque to my users. And any system which required a flag day where everyone leaves behind these existing workflows would be an immediate non-starter.
I think the ideal scenario (and what it would look like if I built this myself based on debs) would be that you could source the "base" workspace as usual (enter the nix-shell?), and check out source, build groups of packages as usual with colcon, but there'd be an extra plugin/verb/flag for it, which would make it build each package as a nix package instead of into the usual installspace. The verb would generate the nix definitions on the fly, and probably handle the invocation and build-parallelism side of it as well.
Most Nix packages define their sources using `src`. You can pass in whatever you want. E.g. fetch a tarball from GitHub with `fetchUrl`, fetch from a git repository, or point to a local directory.
> or reasoning about "snapshots" where pools of unreleased repos/packages are able to be treated as a single versionable unit?
This is basically what the Nixpkgs collection is minus the "unreleased" part. You can extend Nixpkgs with the packages that you care about using overlays. You have control over the stuff that you put in the overlay, so you can put in unreleased software as well like I described above.
> - I have an existing system for managing packaging metadata which I don't want to migrate from. How much trouble will I get into if I want to generate the metadata on the fly each time (as I currently do for my debs)?
This is difficult to answer without knowing more details.
- You can generate the Nix source files based on the metadata like the sibling comment pointed out.
- Or you could read the metadata with `builtins.fromJSON` and generate Nix derivations programmatically from within Nix.
> - What is the apt/nix interop story?
If you don't want to bother packaging certain applications using Nix (or work towards that gradually), you can. Nothing prevents you from referencing things outside of `/nix/store` in a Nix package. You can launch stuff from `/usr/bin` from a Nix binary if you are so inclined. Libraries are going to be tricky though.
As for the other way around, I'm aware of two options:
- At Channable, we would package the Nix closure of a package in a `.deb`. This works, but you run into trouble when multiple debs need the same store path. You can work around this by packaging the Nix closure under e.g. `/var/lib/<your_package>/nix/store` and bind mounting to `/nix/store` before launching.
- Eventually we realized we didn't want the `/nix/store` stuff in our `.deb` packages. IIRC we created a `postinst` script which ran `nix-env` to realize a store path from our cache + create a GC root so it doesn't get garbage collected.
Some of that stuff may have changed since I left.
Feel free to contact me if you'd like to discuss further. Info is in my profile.