Skip to content

Repository files navigation

StarMetal - armored registry proxy

High-performance, self-hosted multi-language package registry and pull-through registry proxy for private software supply chains.

Starmetal gives teams one controlled path for package-manager traffic across ecosystems. It speaks native registry protocols, proxies upstream reads, stores artifacts behind a common service layer, verifies cached bytes with Blake3, and applies policy before dependencies reach clients.

StarMetal is experimental. All currently implemented registry read/proxy workflows are experimental core capabilities; native publishing is not supported yet, and local publishing remains experimental and disabled by default.

PyPI · npm · Cargo · Hex · Maven · RubyGems · NuGet · pub.dev · Go · Zig · Swift

CI Rust 2024 License: MIT

Install · Quick Start · Registry Support · Docker · Configuration · Deployment · Release · Architecture · ADRs


Why It Exists

Modern teams pull dependencies from several registries, each with different protocols, metadata formats, auth expectations, and client behavior. Starmetal puts those workflows behind one self-hosted service boundary so operators can centralize caching, integrity checks, policy, storage, and observability without asking developers to stop using native package-manager clients.

What It Does

Starmetal sits between package-manager clients and upstream registries:

Capability Current shape
Registry proxy Speaks native package-manager routes and rewrites upstream metadata for Starmetal URLs
Pull-through cache Fetches from upstream on miss, stores artifacts, and serves cache hits
Integrity Stores Blake3 sidecars and re-verifies cached artifacts before serving
Policy Blocks packages, licenses, and vulnerability severities through shared service checks
Protocol adapters Feature-gated routers for PyPI, npm, Cargo, Hex, Maven, RubyGems, NuGet, and pub.dev
Git-sourced adapters Feature-gated Go, Zig, and Swift adapters that read directly through a GitMirror port instead of the shared cache (experimental, disabled by default)
Storage OpenDAL-backed filesystem, S3, GCS, and memory backends
Operations CLI plus stdio MCP tools over the same local operations layer
Dependency updates Scans manifests and opens update PRs, reusing the proxy as the version datasource (experimental; Cargo + GitHub)
Content model Postgres-backed component/asset/blob store with blake3 cross-ecosystem dedup, reference-counted garbage collection, and retention policies (experimental, disabled by default)
Supply-chain scanning OSV vulnerability gate at publish and serve time, with serve-time quarantine and admin promote/reject (experimental, disabled by default)
Access control Authorizer port with a deny-by-default grant model migrated from the flat auth/admin/publishing tokens

Starmetal is built for private/internal deployments first. It is not yet a public internet-facing registry product, and every registry workflow should be treated as experimental until fresh live native-client E2E passes for that ecosystem.

Registry Support

Registry Route Read/proxy Local publish substrate Native publish support Evidence
PyPI /pypi Experimental Experimental, disabled by default Unsupported HTTP route conformance and Docker proxy E2E
npm /npm Experimental Experimental, disabled by default Unsupported HTTP/native Docker proxy E2E plus pnpm local publish-then-install
Cargo /cargo Experimental Experimental, disabled by default Unsupported HTTP route conformance and Docker proxy E2E
Hex /hex Experimental Experimental, disabled by default Unsupported HTTP/protobuf route conformance and Docker proxy E2E
Maven /maven Experimental Experimental, disabled by default Unsupported HTTP route conformance and Docker proxy E2E
RubyGems /rubygems Experimental Experimental, disabled by default Unsupported HTTP/native Docker proxy E2E
NuGet /nuget Experimental Experimental, disabled by default Unsupported HTTP/native Docker proxy E2E
pub.dev /pub Experimental Experimental, disabled by default Unsupported HTTP/native Docker proxy E2E
Go /go Experimental (git-sourced) Not applicable (read-only, git-sourced) Unsupported Live native-client E2E (go mod download)
Zig /zig Experimental (git-sourced) Not applicable (read-only, git-sourced) Unsupported Live native-client E2E (zig fetch)
Swift /swift Experimental (git-sourced) Not applicable (read-only, git-sourced) Unsupported Live native-client E2E (swift package resolve + build)

Go, Zig, and Swift are git-sourced (ADR-0023): rather than proxying a package-index protocol, their adapters mirror an upstream git repository through the starmetal-git crate's GitMirror port and translate tags/trees into each ecosystem's artifact shape. They bypass CachingPackageService entirely, are disabled by default, and are gated behind the go/zig/swift build features.

Planned registry work includes OCI/distribution, Composer, Conda, Debian/APT, and RPM/YUM. See ADR-0011 for experimental support criteria and promotion gates. Configured Ed25519 Starmetal DSSE sidecars are experimental. Protocol-native signing, PQ signing, and client-verified PQ support are planned only; ecdsa-p256-sha256, ml-dsa65, and the pq feature are reserved until matching client-visible verification is implemented and tested.

Install

Docker is the primary deployment path. Release builds also publish verified sm installers for npm and PyPI, Homebrew bottles through Goldziher/homebrew-tap, and GitHub release archives for direct download.

docker run --rm -p 8080:8080 -v starmetal-data:/var/lib/starmetal ghcr.io/goldziher/starmetal:latest
brew install Goldziher/tap/starmetal
npm install -g starmetal
pipx install starmetal

The npm package requires Node 22+. The PyPI package requires Python 3.10+. Both download the matching prebuilt binary from GitHub Releases and verify it against the release checksums file. The crates.io starmetal package currently holds the public namespace while the Rust crate publishing layout is finalized.

Quick Start

Requirements:

  • Rust edition 2024, Rust 1.85+
  • Task for the documented workflow commands
  • sccache, optional but used automatically by Taskfile cargo commands
# First-time setup: hooks, sccache check, generated AI config
task setup

# Build and run tests
task ci

# Install the local sm binary from this checkout
cargo install --path crates/starmetal-cli --bin sm

# Or install the prebuilt sm binary from the latest GitHub release via cargo-binstall
# (starmetal-cli is not on crates.io, so point binstall at the git repo)
cargo binstall --git https://github.com/Goldziher/starmetal starmetal-cli

# Start Starmetal with defaults on 127.0.0.1:8080
sm serve

# Write a starter config
sm config init

# Inspect registries without a config file
sm --no-config --storage-backend memory registry status

# Fetch one artifact through the cache path
sm package fetch pypi six 1.16.0 six-1.16.0.tar.gz

Run live native-client E2E before treating an experimental read workflow as ready:

task test:e2e:pypi
task test:e2e:npm
task test:e2e:cargo
task test:e2e:hex

task ci:live-e2e runs live schema freshness plus the PyPI, npm, Cargo, and Hex read gates. Maven, RubyGems, NuGet, and pub.dev have separate live tasks and must pass before their workflows are promoted.

Docker

Docker is the primary deployment path for private experimental installs. The image uses Chainguard builder and runtime bases, runs as non-root, and uses one image for both API and CLI operations. Its entrypoint is sm; its default command is serve, so no args starts the API server, and args after the image name run normal CLI or MCP commands.

Published images use GitHub Container Registry: ghcr.io/goldziher/starmetal.

docker build -t starmetal:local .
docker run --rm -p 8080:8080 -v starmetal-data:/var/lib/starmetal starmetal:local
docker run --rm starmetal:local config validate
docker run --rm starmetal:local mcp serve

Run the deterministic local proxy gate:

task docker:proxy:e2e

That gate builds the image, runs a local fixture upstream plus StarMetal on an isolated Docker network, exercises every implemented registry route with HTTP assertions, runs native client containers for PyPI, npm, Cargo, Maven, RubyGems, NuGet, and pub.dev, restarts StarMetal with the same OpenDAL filesystem volume, and repeats the checks with the fixture upstream stopped. It also runs pnpm read-through install, cached reinstall with a fresh pnpm store, and experimental local npm publish-then-install. The pnpm publish pass proves only the local npm publishing substrate; it is not a supported native publishing claim for npm or any other ecosystem. The native client pass disables read auth because package-manager support for Bearer read auth is uneven; the HTTP pass covers auth behavior.

Use a mounted config file for production settings, auth tokens, public_base_url, and S3/GCS OpenDAL options:

docker run --rm \
  -p 8080:8080 \
  -v ./starmetal.toml:/etc/starmetal/starmetal.toml:ro \
  -v starmetal-data:/var/lib/starmetal \
  starmetal:local

The default container config is docker/starmetal.toml.

Configuration

Starmetal defaults to loopback binding and filesystem storage. A minimal private deployment usually starts from:

[server]
bind = "127.0.0.1:8080"
public_base_url = "https://starmetal.internal.example.com"
cors_allowed_origins = []
max_upload_bytes = 536870912

[storage]
backend = "fs"
path = "/var/lib/starmetal"

[auth]
enabled = true
tokens = ["replace-with-a-secret-token"]

# [metadata]       # experimental Postgres content model: dedup, GC, retention
# enabled = true
# database_url = "postgresql://user:password@host:5432/starmetal"

# [supply_chain]   # experimental OSV vulnerability gate at ingest and serve
# enabled = true

Upstream URLs must be HTTPS and public by default. Local, private-network, or insecure upstreams require explicit allow_private_network and allow_insecure settings. See docs/configuration.md for every option and validation rule, and docs/deployment.md for private deployment guidance.

CLI and MCP

The CLI command is sm. Config lookup still supports STARMETAL_CONFIG and starmetal.toml for compatibility; the CLI and MCP server can run without a config file using built-in defaults plus explicit flags.

Common CLI operations:

sm config show
sm config validate
sm registry status
sm package list pypi
sm package versions npm is-odd
sm package metadata cargo once_cell 1.19.0
sm package fetch npm is-odd 3.0.1 is-odd-3.0.1.tgz --output ./is-odd.tgz
sm cache delete-artifact npm is-odd 3.0.1 is-odd-3.0.1.tgz --yes

Use --output json for machine-readable output. MCP runs over stdio:

sm mcp serve
sm mcp serve --allow-writes

MCP read tools are always available. Mutating tools, including experimental local publish, yank, unyank, and cache delete, require --allow-writes.

Dependency Updates

The experimental update feature (included in the CLI full build) adds a Phase 0 dependency-update engine, scoped to Cargo manifests and GitHub. It reuses PackageService as its version datasource, so update lookups go through the same pull-through cache, Blake3 verification, and policy checks as the proxy.

sm update scan .
sm update run goldziher/starmetal --dry-run

sm update scan <path> scans a local directory tree for Cargo.toml files and reports available patch, minor, and major updates. sm update run <owner>/<name> scans a GitHub repository and opens or updates a pull request; --dry-run reports available updates without opening a pull request. Both commands accept --pin and --allow-prerelease. run requires a GitHub token, passed with --token or, preferably, the STARMETAL_GITHUB_TOKEN environment variable. See ADR-0016 and ADR-0017.

Architecture

Starmetal uses hexagonal architecture: protocol adapters and storage backends sit outside a shared service/core boundary.

Crate Role
starmetal-core Domain types, config, policy, ports, lock file, registry schema types
starmetal-service Pull-through cache, Blake3 verification, policy checks, experimental local publishing
starmetal-storage OpenDAL-backed StoragePort implementation
starmetal-adapters Feature-gated protocol routers and upstream clients
starmetal-git Feature-gated gitoxide-backed GitMirror port implementation powering the Go, Zig, and Swift git-sourced adapters (ADR-0023)
starmetal-server Axum app assembly and Tower middleware
starmetal-authz LocalAuthorizer: deny-by-default Authorizer/Authenticator implementation migrating flat auth/admin/publishing tokens into a grant model (ADR-0022)
starmetal-oidc OidcAuthenticator: offline static-JWKS OIDC bearer validator behind the Authenticator port, composed ahead of the flat-token backend (ADR-0022)
starmetal-metadata Postgres-backed content model: component/asset/blob, blake3 dedup, garbage collection, retention (ADR-0020)
starmetal-ops Shared local operator API used by CLI and MCP
starmetal-cli Clap CLI and stdio MCP server
starmetal-update-core Framework-free dependency-update domain types and ports (experimental)
starmetal-versioning Cargo semver comparison, range membership, and constraint rewriting (experimental)
starmetal-managers Cargo.toml manifest parsing and formatting-preserving edits (experimental)
starmetal-forge GitHub forge backend over its HTTP API (experimental)
starmetal-updater Dependency-update engine: scan and run workflows (experimental)

See docs/architecture.md for diagrams and component details.

Development Gates

Normal PR-safe gate:

task fmt:check
task clippy
task test:all
task schema:check
task schema:validate
task conformance
task docker:proxy:e2e
task security
task ci

task docker:proxy:e2e uses disposable client containers, not host package-manager CLIs. Live E2E and live Docker pressure tests are intentionally separate from normal PR CI because they require public registry access.

Schemas

Schema provenance, fetched upstream artifacts, Starmetal-derived JSON Schemas, and grammar fixtures live under schemas/:

ADRs

License

MIT

About

High-performance, self-hosted multi-language package registry and registry proxy

Topics

Resources

Stars

10 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages