Skip to content

Repository files navigation

nix-bulkfetch-url

Fetch URLs, print hashes. No Nix store, parallel.

screenshot

Features

  • Configurable worker pool (default: 16 concurrent downloads)
  • Retries with exponential backoff (3 attempts by default)
  • Fail-fast mode to bail on first error
  • Zero dependencies, single static binary

Install

Run directly without installing:

nix run github:z1-0/nix-bulkfetch-url

Or install permanently:

nix profile install github:z1-0/nix-bulkfetch-url

As a flake input

Add to your flake:

inputs.nix-bulkfetch-url.url = "github:z1-0/nix-bulkfetch-url";

NixOS system-wide (in configuration.nix):

environment.systemPackages = [
  inputs.nix-bulkfetch-url.packages.${pkgs.system}.default
];

Home Manager (in home.nix):

home.packages = [
  inputs.nix-bulkfetch-url.packages.${pkgs.system}.default
];

Dependencies

The binary itself has no Go dependencies. It is statically linked. Archive unpacking uses external tools:

Format Tool
.tar.gz, .tgz tar
.tar.xz, .txz tar
.tar.bz2, .tbz2 tar
.tar.zst, .tzst tar + zstd
.tar.lzma, .tlzma tar
.tar.lz, .tlz tar + lzip
.tar.lz4, .tlz4 tar + lz4
.tar.lzo, .tlzo tar + lzop
.tar.Z, .tZ tar + compress
.zip unzip

Usage

Feed URLs via stdin, one per line. Lines starting with # and empty lines are ignored.

echo "https://example.com/archive.tar.gz" | nix-bulkfetch-url

Flags

Flag Default Description
-j 16 Concurrent workers
--type sha256 Hash algorithm: md5, sha1, sha256, sha512, blake3
--format sri Output format: base16, base32, base64, sri
--unpack false Unpack archive and compute NAR hash
--json false Output as JSON
--timeout 300 Download timeout (seconds)
--fail-fast false Exit on first error

Exit codes

Code Meaning
0 All URLs succeeded
1 Some failed, some succeeded
2 Nothing succeeded (or no input)

Examples

Hash a single URL

echo "https://github.com/user/repo/archive/v1.0.0.tar.gz" | nix-bulkfetch-url

Hash multiple URLs from a file

cat urls.txt | nix-bulkfetch-url -j 8 --json

Unpack and compute NAR hash

Downloads each archive, extracts it, and computes a NAR hash on the unpacked directory. That is the same hash you would put in a Nix derivation's sha256 field.

cat urls.txt | nix-bulkfetch-url --unpack

Pipe URLs from a script

grep -oP 'https://[^\s"]+\.tar\.gz' packages.nix | nix-bulkfetch-url --type blake3

JSON output for scripting

cat urls.txt | nix-bulkfetch-url --json | jq '.[] | select(.error == null)'

How it works

  1. Reads URLs from stdin
  2. Downloads them concurrently using a pool of N workers
  3. For each URL: fetches to a temp directory, unpacks if --unpack is set, then hashes with nix-hash
  4. Prints one hash per line (or a JSON array with --json)

About

parallel of nix-prefetch-url

Topics

Resources

Stars

31 stars

Watchers

0 watching

Forks

Contributors

Languages