Skip to content

Migrate minidock_tools to multitool #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adam-singer merged 6 commits into from
Dec 12, 2025
Merged

Migrate minidock_tools to multitool #30

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1f4909b
[bzlmod] Use multitool for fetching remote_tools
Dec 10, 2025
2486487
use module extension
Dec 10, 2025
0143b3f
update version
Dec 10, 2025
06876f2
Review changes
Dec 11, 2025
2be8561
adding multitool to multitool.lock.json
Dec 12, 2025
26d15ed
Use multitool update script
Dec 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 15 additions & 16 deletions MODULE.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,19 @@

module(
name = "rules_minidock",
version = "0.1.0",
compatibility_level = 1,
version = "0.2.0",
compatibility_level = 2,
)

# Required dependencies
bazel_dep(name = "platforms", version = "0.0.11")

bazel_dep(name = "rules_multitool", version = "1.11.1")

multitool = use_extension("@rules_multitool//multitool:extension.bzl", "multitool")
multitool.hub(lockfile = "//minidock/remote_tools:multitool.lock.json")
use_repo(multitool, "multitool")

# Test dependencies
bazel_dep(name = "rules_python", version = "0.40.0", dev_dependency = True)

Expand All @@ -25,18 +31,11 @@ pip.parse(
use_repo(pip, "pip")

# Test container repositories
test_repos = use_extension("//minidock/remote_tools:extensions.bzl", "test_repos", dev_dependency = True)
use_repo(test_repos, "bazel_320")

# Register the toolchain extension
minidock_tools = use_extension("//minidock/remote_tools:extensions.bzl", "minidock_tools")
use_repo(
minidock_tools,
"rules_minidock__merge_app_linux_x86_64",
"rules_minidock__merge_app_macos_aarch64",
"rules_minidock__merge_app_macos_x86_64",
"rules_minidock__puller_app",
"rules_minidock__pusher_app_linux_x86_64",
"rules_minidock__pusher_app_macos_aarch64",
"rules_minidock__pusher_app_macos_x86_64",
external_containers = use_extension("//minidock:external_container_repo.bzl", "external_container_repo", dev_dependency = True)
external_containers.container(
name = "bazel_320",
digest = "sha256:08434856d8196632b936dd082b8e03bae0b41346299aedf60a0d481ab427a69f",
registry = "gcr.io",
repository = "bazel-public/bazel",
)
use_repo(external_containers, "bazel_320")
229 changes: 141 additions & 88 deletions MODULE.bazel.lock
View file
Open in desktop

Large diffs are not rendered by default.

46 changes: 33 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,17 @@ git_override(
remote = "https://github.com/bazeltools/rules_minidock",
commit = "<git_commit>",
)
```

# Load the minidock tools extension from extensions.bzl
minidock_tools = use_extension("@rules_minidock//minidock/remote_tools:extensions.bzl", "minidock_tools")
use_repo(
minidock_tools,
"rules_minidock__merge_app_linux_x86_64",
"rules_minidock__merge_app_macos_aarch64",
"rules_minidock__merge_app_macos_x86_64",
"rules_minidock__puller_app",
"rules_minidock__pusher_app_linux_x86_64",
"rules_minidock__pusher_app_macos_aarch64",
"rules_minidock__pusher_app_macos_x86_64",
)
### Updating Tools

To update the multitool binaries (merge-app, pusher-app, puller-app) to a new version:

```bash
./minidock/remote_tools/update_multitool_remote_tools.sh
```

This updates `minidock/remote_tools/multitool.lock.json` with the latest tool versions.

### Why these rules?
Less dependencies, there is no need of any other support java/go/rust to use these rules. All tools are pre-built in the `rules_minidock_tools` repo, and are supplied for a few platforms. Generally if these tools don't meet your needs, we aim to have these rules setup so that it should be easy and reasonable to swap out these tools for others as you see fit. That is, primary tooling covers:
Expand Down Expand Up @@ -64,7 +60,31 @@ While a lot of the code here has been rewritten for this narrower use case, some
The dependency configuration in these rules to align containers vs bazel, has the caveat that overrides occur in the depset order. So the depset of actions will be linearized, and then applied as a stack. With the special carve out that we will only apply one `base` configuration, so we find that first, and apply everything ontop of it.

## external_container_repo
This is the entry point for declaring an external repo, it will fetch the metadata about a remote registry
Fetches metadata about containers from a remote registry. Declare external containers in your `MODULE.bazel`:

```python
external_containers = use_extension("@rules_minidock//minidock:external_container_repo.bzl", "external_container_repo")

external_containers.container(
name = "base_image",
digest = "sha256:abcd1234567890abcdef1234567890abcdef1234567890abcdef1234567890ab",
registry = "docker.io",
repository = "library/alpine",
)

external_containers.container(
name = "app_base",
digest = "sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
registry = "gcr.io",
repository = "my-project/base-image",
)

use_repo(
external_containers,
"base_image",
"app_base",
)
```


## container_config
Expand Down
2 changes: 1 addition & 1 deletion minidock/container_assemble.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ container_assemble = rule(
doc = "The label of the image to push.",
),
"merger": attr.label(
default = "@rules_minidock//minidock/remote_tools:merge_app",
default = "@multitool//tools/merge-app",
cfg = "exec",
executable = True,
),
Expand Down
2 changes: 1 addition & 1 deletion minidock/container_push.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ container_push = rule(
doc = "The label of the file with tag values, added to tag. Can use multiple tags separated by , or whitespace",
),
"pusher": attr.label(
default = "@rules_minidock//minidock/remote_tools:pusher_app",
default = "@multitool//tools/pusher-app",
cfg = "exec",
executable = True,
allow_files = True,
Expand Down
120 changes: 117 additions & 3 deletions minidock/external_container_repo.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,61 @@ def ___external_container_repo(repository_ctx):
if not digest.startswith("sha256:"):
fail("Invalid digest value, should start with sha256, as it is the only supported digest today but is a required prefix")

# Get puller binary path
puller_path = None
if repository_ctx.attr.puller:
puller_path = repository_ctx.path(repository_ctx.attr.puller)
else:
# Read from multitool lockfile
lockfile_path = repository_ctx.path(Label("@rules_minidock//minidock/remote_tools:multitool.lock.json"))
lockfile_content = repository_ctx.read(lockfile_path)
lockfile_json = json.decode(lockfile_content)

# Determine platform
os_name = repository_ctx.os.name
os_key = None
cpu_key = None

if os_name == "linux":
os_key = "linux"
cpu_key = "x86_64"
elif os_name == "mac os x":
os_key = "macos"
res = repository_ctx.execute(["uname", "-m"])
if res.return_code == 0 and res.stdout.strip() == "arm64":
cpu_key = "arm64"
else:
cpu_key = "x86_64"
else:
fail("Unsupported OS: " + os_name)

# Find the matching binary in the lockfile
puller_tool = lockfile_json.get("puller-app")
if not puller_tool:
fail("puller-app not found in multitool lockfile")

puller_url = None
puller_sha = None
for binary in puller_tool["binaries"]:
if binary["os"] == os_key and binary["cpu"] == cpu_key:
puller_url = binary["url"]
puller_sha = binary["sha256"]
break

if not puller_url:
fail("No puller-app binary found for os=%s cpu=%s" % (os_key, cpu_key))

# Download the puller
repository_ctx.download(
url = puller_url,
output = "puller",
sha256 = puller_sha,
executable = True,
)
puller_path = repository_ctx.path("puller")

fetch_args = []
fetch_args.append(repository_ctx.path(repository_ctx.attr.puller))
fetch_args.append(puller_path)
fetch_args.append("--registry")
fetch_args.append(repository_ctx.attr.registry)
fetch_args.append("--repository")
Expand Down Expand Up @@ -53,16 +106,18 @@ external_metadata(
digest = repository_ctx.attr.digest,
))

external_container_repo = repository_rule(
_external_container_repo_rule = repository_rule(
doc = "Fetches metadata about containers",
attrs = {
"digest": attr.string(
doc = "Digest of the container image.",
),
"puller": attr.label(
cfg = "exec",
default = "@rules_minidock__puller_app//:exe",
doc = "Override generated based on settings option to fetch metadata",
executable = True,
allow_files = True,
mandatory = False,
),
"registry": attr.string(
mandatory = True,
Expand All @@ -85,3 +140,62 @@ external_container_repo = repository_rule(
},
implementation = ___external_container_repo,
)

_container_tag = tag_class(
attrs = {
"name": attr.string(
mandatory = True,
doc = "Name of the container repository.",
),
"digest": attr.string(
mandatory = True,
doc = "Digest of the container image.",
),
"registry": attr.string(
mandatory = True,
doc = "Container registry.",
),
"repository": attr.string(
mandatory = True,
doc = "The name of the image.",
),
"architecture": attr.string(
mandatory = False,
default = "amd64",
doc = "Architecture.",
),
"puller": attr.label(
cfg = "exec",
doc = "Override puller binary",
executable = True,
allow_files = True,
mandatory = False,
),
"authentication_helper": attr.label(
cfg = "exec",
executable = True,
allow_files = True,
mandatory = False,
doc = "Authentication helper binary",
),
},
)

def _external_container_repo_extension_impl(module_ctx):
"""Implementation of the external_container_repo module extension."""
for mod in module_ctx.modules:
for container in mod.tags.container:
_external_container_repo_rule(
name = container.name,
digest = container.digest,
registry = container.registry,
repository = container.repository,
architecture = container.architecture,
puller = container.puller,
authentication_helper = container.authentication_helper,
)

external_container_repo = module_extension(
implementation = _external_container_repo_extension_impl,
tag_classes = {"container": _container_tag},
)
23 changes: 2 additions & 21 deletions minidock/remote_tools/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
exports_files(["multitool.lock.json"], visibility = ["//visibility:public"])

config_setting(
name = "macos_x86",
constraint_values = [
Expand Down Expand Up @@ -28,24 +30,3 @@ config_setting(
"@platforms//cpu:x86_64",
],
)

alias(
name = "merge_app",
actual = select({
":macos_x86": "@rules_minidock__merge_app_macos_x86_64//:bin",
":macos_arm64": "@rules_minidock__merge_app_macos_aarch64//:bin",
":linux_x86": "@rules_minidock__merge_app_linux_x86_64//:bin",
}),
visibility = ["//visibility:public"],
)


alias(
name = "pusher_app",
actual = select({
":macos_x86": "@rules_minidock__pusher_app_macos_x86_64//:bin",
":macos_arm64": "@rules_minidock__pusher_app_macos_aarch64//:bin",
":linux_x86": "@rules_minidock__pusher_app_linux_x86_64//:bin",
}),
visibility = ["//visibility:public"],
)
71 changes: 0 additions & 71 deletions minidock/remote_tools/extensions.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,78 +1,7 @@
"""Module extension for rules_minidock tooling."""

load("//minidock/remote_tools:load_tool.bzl", "load_tool")
load("//minidock/remote_tools:repo_rule_load_tool.bzl", "repo_rule_load_tool")
load("//minidock:external_container_repo.bzl", "external_container_repo")

def _minidock_tools_impl(module_ctx):
"""Implementation of the minidock_tools module extension.

This extension loads all the platform-specific tools needed by rules_minidock,
including the merge and pusher applications for various platforms.
"""

# RUST_BINARIES_AUTO_GEN_REPLACE_SECTION_START
load_tool(
name = "rules_minidock__merge_app_linux_x86_64",
sha256 = "ec36e90e1dfd4ab486647715acb20c404ed5e1427ec51fc6bcd0fa2d471fbf6c",
packaged = False,
binary_path = "merge-app-linux-x86_64",
urls = ["https://github.com/bazeltools/rules_minidock_tools/releases/download/v0.0.75/merge-app-linux-x86_64"],
)

load_tool(
name = "rules_minidock__merge_app_macos_x86_64",
sha256 = "633065d4f5199c20fe0fc9a750fc41f2b5bf0b28d4f51d5d4dab9bc3f1a56c97",
packaged = False,
binary_path = "merge-app-macos-x86_64",
urls = ["https://github.com/bazeltools/rules_minidock_tools/releases/download/v0.0.75/merge-app-macos-x86_64"],
)

load_tool(
name = "rules_minidock__merge_app_macos_aarch64",
sha256 = "110d44df10c495d972533cb4db134dd1a95db7c436180655b1258e1f17c9154b",
packaged = False,
binary_path = "merge-app-macos-aarch64",
urls = ["https://github.com/bazeltools/rules_minidock_tools/releases/download/v0.0.75/merge-app-macos-aarch64"],
)

load_tool(
name = "rules_minidock__pusher_app_linux_x86_64",
sha256 = "6448bb16191cb4e0f8750003c43ab158df58ecc1cd0644179e60fbf0556638c7",
packaged = False,
binary_path = "pusher-app-linux-x86_64",
urls = ["https://github.com/bazeltools/rules_minidock_tools/releases/download/v0.0.75/pusher-app-linux-x86_64"],
)

load_tool(
name = "rules_minidock__pusher_app_macos_x86_64",
sha256 = "da24d3a25d46c7561bcdb21fa21c938cac20862a26d62f774f46e8b38ed68a32",
packaged = False,
binary_path = "pusher-app-macos-x86_64",
urls = ["https://github.com/bazeltools/rules_minidock_tools/releases/download/v0.0.75/pusher-app-macos-x86_64"],
)

load_tool(
name = "rules_minidock__pusher_app_macos_aarch64",
sha256 = "449fbbe83f680138143ace22f02d980197ddcd8c94dbf293f448833e403e37e1",
packaged = False,
binary_path = "pusher-app-macos-aarch64",
urls = ["https://github.com/bazeltools/rules_minidock_tools/releases/download/v0.0.75/pusher-app-macos-aarch64"],
)

repo_rule_load_tool(
name = "rules_minidock__puller_app",
platform_to_sha_pairs = {
"linux__x86_64": ["9a28cd949268a397e57c4faf106a942f3053438c84b70077072452d83971060a", "https://github.com/bazeltools/rules_minidock_tools/releases/download/v0.0.75/puller-app-linux-x86_64"],
"macos__x86_64": ["70bc80d0f1e098a6d2f522a4f907011f54ce15e5a8bd300ee3442a777f75eafb", "https://github.com/bazeltools/rules_minidock_tools/releases/download/v0.0.75/puller-app-macos-x86_64"],
"macos__aarch64": ["950d2192cc23ff4dd44d7dc7b942b6ec02b425a03d648924136449ff0702002d", "https://github.com/bazeltools/rules_minidock_tools/releases/download/v0.0.75/puller-app-macos-aarch64"],
},
)
# RUST_BINARIES_AUTO_GEN_REPLACE_SECTION_END

minidock_tools = module_extension(
implementation = _minidock_tools_impl,
)

def _test_repos_impl(module_ctx):
"""Implementation of the test_repos module extension.
Expand Down
Loading