Simple and convenient dependency pinning for Nix
npins is a simple tool for handling different types of dependencies in a Nix project. It is inspired by and comparable to Niv.
- Track git branches
- Track git release tags
- Tags must roughly follow SemVer
- GitHub/GitLab releases are intentionally ignored
- For git repositories hosted on GitHub or GitLab,
fetchTarballis used instead offetchGit - Track Nix channels
- Unlike tracking a channel from its git branch, this gives you access to the
programs.sqlitedatabase - Can also track Nix channel artifacts like live isos
- Unlike tracking a channel from its git branch, this gives you access to the
- Track PyPi packages
npins should readily be available in all sufficiently new nixpkgs:
nix-shell -p npinsYou can easily get a nightly if you want to (requires newstyle Nix commands):
nix shell -f https://github.com/andir/npins/archive/master.tar.gzYou could also install it to your profile using nix-env (not recommended, but might be useful for bootstrapping):
nix-env -f https://github.com/andir/npins/archive/master.tar.gz -i$ npins init
[INFO ] Welcome to npins!
[INFO ] Writing default.nix
[INFO ] Writing initial sources.json with nixpkgs entry (need to fetch latest commit first)
[INFO ] Successfully written initial files to 'npins'.
$ tree
.
└── npins
├── default.nix
└── sources.json
1 directory, 2 files
$ npins show
nixpkgs: (Nix channel)
name: nixpkgs-unstable
url: https://releases.nixos.org/nixpkgs/nixpkgs-22.05pre378171.ff691ed9ba2/nixexprs.tar.xz
hash: 04xggrc0qz5sq39mxdhqh0d2mljg9wmmn8nbv71x3vblam1wyp9b
$ cat npins/sources.json
{
"pins": {
"nixpkgs": {
"type": "Channel",
"name": "nixpkgs-unstable",
"url": "https://releases.nixos.org/nixpkgs/nixpkgs-22.05pre378171.ff691ed9ba2/nixexprs.tar.xz",
"hash": "04xggrc0qz5sq39mxdhqh0d2mljg9wmmn8nbv71x3vblam1wyp9b"
}
},
"version": 2
}
In Nix, you may then use it like this:
let
sources = import ./npins;
pkgs = import sources.nixpkgs {};
in
…You may also use attributes from the JSON file, they are exposed 1:1. For example, sources.myPackage.version should work for many pin types (provided that that pin actually tracks some version). Note however that the available attribute may change over time; see npins upgrade below.
$ npins --help
Simple and convenient dependency pinning for Nix. All options are available in subcommands.
Usage: npins ([-d=FOLDER] | --lock-file=FILE) [-v] COMMAND ...
Available options:
-d, --directory=FOLDER Specifies base folder for sources.json and the boilerplate default.nix
[env:NPINS_DIRECTORY: N/A]
[default: npins]
--lock-file=FILE Specifies the lockfile and operates only on it (lockfile mode)
-v, --verbose Prints debug messages
-h, --help Prints help information
-V, --version Prints version information
Available commands:
init Intializes the npins directory. Running this multiple times will
restore/upgrade the `default.nix` and never touch your sources.json
add Adds a new pin entry.
show Lists the current pin entries
update Updates all or the given pins to the latest version
verify Verifies that all or the given pins still have correct hashes. This is
like `update --partial --dry-run` and then checking that the diff is
empty
upgrade Upgrade the sources.json and default.nix to the latest format version.
This may occasionally break Nix evaluation!
remove Remove pin entries
import-niv Try to import entries from Niv
import-flake Try to import entries from flake.lock
freeze Freezes a pin entry, preventing it from being changed during an update
unfreeze Thaws a pin entry, allowing it to be changed during an update like a
normal pin
get-path Evaluates the store path to a pin, fetching it if necessary. Don't
forget to add a GC rootIn order to start using npins to track any dependencies you need to first initialize the project:
npins initThis will create an npins folder with a default.nix and sources.json within. By default, the nixpkgs-unstable channel will be added as pin.
$ npins init --help
Intializes the npins directory. Running this multiple times will restore/upgrade the `default.nix`
and never touch your sources.json
Usage: npins init [--bare]
Available options:
--bare Don't add an initial `nixpkgs` entry
-h, --help Prints help informationYou can import your pins from Niv:
npins import-niv nix/sources.json
npins updateIn your Nix configuration, simply replace import ./nix/sources.nix with import ./npins — it should be a drop-in replacement.
Note that the import functionality is minimal and only preserves the necessary information to identify the dependency, but not the actual pinned values themselves. Therefore, migrating must always come with an update (unless you do it manually).
$ npins import-niv --help
Try to import entries from Niv
Usage: npins import-niv [-n=NAME] [FILE]
Available positional items:
Available options:
-n, --name=NAME Only import one entry from Niv
-h, --help Prints help informationSome common usage examples:
npins add channel nixos-21.11
npins add channel nixos-unstable latest-nixos-graphical-x86_64-linux.iso2
# Remove -b to fetch the latest release
npins add git https://gitlab.com/simple-nixos-mailserver/nixos-mailserver.git -b "nixos-21.11"
npins add github ytdl-org youtube-dl
npins add github ytdl-org youtube-dl -b master # Track nightly
npins add github ytdl-org youtube-dl -b master --at c7965b9fc2cae54f244f31f5373cb81a40e822ab # We want *that* commit
npins add gitlab simple-nixos-mailserver nixos-mailserver --at v2.3.0 # We want *that* tag (note: tag, not version)
npins add gitlab my-org my-private-repo --token H_BRqzV3NcaPvXcYs2Xf # Use a token to access a private repository
npins add pypi streamlit # Use latest version
npins add pypi streamlit --at 1.9.0 # We want *that* version
npins add pypi streamlit --upper-bound 2.0.0 # We only want 1.XDepending on what kind of dependency you are adding, different arguments must be provided. You always have the option to specify a version (or hash, depending on the type) you want to pin to. Otherwise, the latest available version will be fetched for you. Not all features are present on all pin types.
$ npins add --help
Adds a new pin entry.
Usage: npins add [--name=NAME] [--frozen] [-n] COMMAND ...
Available options:
--name=NAME Add the pin with a custom name. If a pin with that name already exists, it will
be overwritten
--frozen Add the pin as frozen, meaning that it will be ignored by `npins update` by
default.
-n, --dry-run Don't actually apply the changes
-h, --help Prints help information
Available commands:
channel Track a Nix channel
github Track a GitHub repository
forgejo Track a Forgejo repository
gitlab Track a GitLab repository
git Track a git repository
pypi Track a package on PyPi
container Track an OCI container
tarball Track a URL
url Track a URLThere are several options for tracking git branches, releases and tags:
$ npins add git --help
Track a git repository
Usage: npins add git [--at=<TAG OR REV>] [--submodules] (-b=BRANCH | [--pre-releases] [--upper-bound
=VERSION] [--release-prefix=VERSION]) [--forge=FORGE] URL
Available positional items:
URL The git remote URL. For example <https://github.com/andir/ate.git>
Available options:
--at=<TAG OR REV> Use a specific commit/release instead of the latest. This may be a tag
name, or a git revision when --branch is set.
--submodules Also fetch submodules
-b, --branch=BRANCH Track a branch instead of a release
--pre-releases Also track pre-releases. Conflicts with the --branch option.
--upper-bound=VERSION Bound the version resolution. For example, setting this to "2" will
restrict updates to 1.X versions.
--release-prefix=VERSION Optional prefix required for each release name / tag. For example,
setting this to "release/" will only consider those that start with that
string.
--forge=FORGE
[default: auto]
-h, --help Prints help informationNpins can track plain old links to URL resources. They will never update.
Alternatively, you can also add the --mutable flag to make them behave similarly to the
Lockable HTTP Tarball Protocol:
Npins will follow any redirects and then pin that url as the actual version, while keeping the original url as "update" url.
$ npins add tarball --help
Track a URL
Usage: npins add tarball [--mutable] URL
This can be either a static URL that never changes its contents or a "mutable" URL that redirects to
an immutable snapshot.
Available positional items:
URL Tarball URL
Available options:
--mutable Treat this URL as mutable, and assume it will redirect to an immutable version of
the content to be pinned. For example, a HEAD URL redirecting to the currently
latest commit
-h, --help Prints help information$ npins remove --help
Remove pin entries
Usage: npins remove NAMES...
Available positional items:
NAMES Names of the pins to remove
Available options:
-h, --help Prints help informationThis will print the currently pinned dependencies in a human readable format. The machine readable sources.json may be accessed directly, but make sure to always check the format version (see below).
$ npins show --help
Lists the current pin entries
Usage: npins show [-b] [-e] [NAMES]...
Available positional items:
NAMES Names of the pins to show
Available options:
-b, --plain Prints only pin names
-e, --exclude Prints all the pins not specified
-h, --help Prints help informationYou can decide to update only selected dependencies, or all at once. For some pin types, we distinguish between "find out the latest version" and "fetch the latest version". These can be controlled with the --full and --partial flags.
$ npins update --help
Updates all or the given pins to the latest version
Usage: npins update (-f | -p) [-n] [--frozen] [--max-concurrent-downloads=NUM] [NAMES]...
Available positional items:
NAMES Updates only the specified pins
Available options:
-f, --full Re-fetch hashes even if the version hasn't changed. Useful to make sure the
derivations are in the Nix store.
-p, --partial Don't update versions, only re-fetch hashes
-n, --dry-run Print the diff, but don't write back the changes
--frozen Allow updating frozen pins, which would otherwise be ignored
--max-concurrent-downloads=NUM Maximum number of simultaneous downloads
-h, --help Prints help informationTo ensure compatibility across releases, the npins/sources.json and npins/default.nix are versioned. Whenever the format changes (i.e. because new pin types are added), the version number is increased. Use npins upgrade to automatically apply the necessary changes to the sources.json and to replace the default.nix with one for the current version. No stability guarantees are made on the Nix side across versions.
$ npins upgrade --help
Upgrade the sources.json and default.nix to the latest format version. This may occasionally break
Nix evaluation!
Usage: npins upgrade
Available options:
-h, --help Prints help informationThere are two ways of specifying the access token (not deploy token!), either via an environment variable or via a parameter.
The access token needs at least the read_api and read_repository scopes and the Reporter role.
The read_api scope is not available for deploy tokens, hence they are not usable for npins.
Specifying the token via environment variable means that npins will use the token for adding/updating the pin but not write it to sources.json.
To update the repository in the future, the variable needs to be set again and nix needs to be configured accordingly to be able to fetch it (see the netrc-file option).
Environment example:
$ GITLAB_TOKEN=H_BRqzV3NcaPvXcYs2Xf npins add gitlab my-org my-private-repoWhen specifying the token via the --token parameter, the token is written to sources.json so future invocations of npins will use it as well.
The token is also embedded into the URL that nix downloads, so no further nix configuration is necessary.
As npins adds the token to your sources.json, this feature is not advised for publicly available repositories.
When a pin has specified a token, the GITLAB_TOKEN environment variable is ignored.
Parameter example:
$ npins add gitlab my-org my-private-repo --token H_BRqzV3NcaPvXcYs2XfWhile npins allows you to pin dependencies in reproducible fashion, it is often desirable to allow fast impure iterations during development.
Npins supports local overrides for this.
If your sources.json contains a source named abc, you can e.g. develop from /abc by exposing the environment variable NPINS_OVERRIDE_abc=/abc.
Please note, that only alphanumerical characters and _ are allow characters in overriden sources.
All other characters are converted to _.
Also check, that you are building impure, if you are wondering, why these overrides are maybe not becoming active.
By default, all pins are fetched through builtins fetchers.
These fetch at eval time and do not produce a derivation, like with IFD.
This is necessary for bootstrapping purposes (the first Nixpkgs can only be fetched through a builtins), but may be undesirable for other pins.
All pins optionally take a pkgs argument, which will use the Nixpkgs fetchers instead and produce a derivation.
let
sources = import ./npins;
pkgs = import sources.nixpkgs { };
in
sources.mySource { inherit pkgs; }The recommended way is to use our packaging in the repository by pinning npins itself with npins:
npins add github andir npins -b master
let
sources = import ./npins;
npinsSources = import (sources.npins + "/npins");
npinsPkgs = import npinsSources.nixpkgs { };
in
npinsPkgs.callPackage (sources.npins + "/npins.nix") {}Alternatively, a good old package override can do the same:
pkgs.npins.overrideAttrs (final: old: {
version = …;
src = (import ./npins).npins;
cargoHash = null;
cargoDeps = pkgs.rustPlatform.fetchCargoVendor {
src = final.src;
hash = …;
};
})Contributions to this project are welcome in the form of GitHub Issues or PRs. Please consider the following before creating PRs:
- This project has several commit hooks configured in the
shell.nix, make sure you have these enabled and they are passing - This readme is templated, edit README.md.in instead (the commit hook will take care of the rest)
- Consider discussing major features or changes in an issue first
Getting LLMs to produce any reasonable output requires expertise with the codebase, which drive-by contributors typically lack. At the same time, making easy contributions by hand is the only way to gather such expertise in the first place. Therefore, LLM contributions are poisoning our own supply of domain experts and our interest in reviewing LLM code is accordingly low.