This repository provides Bazel rules for the ecosystem of Kubebuilder-style Kubernetes controllers.

• Code & manifest generation using controller-gen
• Testing with envtest
• Configuration management using kustomize
• Creating local dev environments using kind
• Testing Chainsaw on kind clusters

At the moment, only Bzlmod is supported. Add this to your MODULE.bazel:

bazel_dep(name = "io_github_janhicken_rules_kubebuilder", version = "0.0.0")

The module is not yet published in the Bazel Central Registry. As a result, you will need to configure an override to source the module from GitHub like this:

git_override(
    module_name = "io_github_janhicken_rules_kubebuilder",
    remote = "https://github.com/janhicken/rules_kubebuilder.git",
    commit = "<desired ref>",
)

Next, load the kubebuilder extension and configure it for the desired Kubernetes API version:

kubebuilder = use_extension("@io_github_janhicken_rules_kubebuilder//kubebuilder:extensions.bzl", "kubebuilder")
kubebuilder.for_kubernetes(version = "1.32.0")

For envtest support, making the @envtest repository visible is required:

use_repo(kubebuilder, "envtest")

You will probably also need to set up rules_go for Go development. Please make sure to not use an alias for the rules_go module.

This repository contains exemplary usages of most of the features.

The controller-gen targets are based on go_library targets that are scanned for the Kubebuilder markers.

These rules are available:

• controller_gen_crds: generate CRD YAMLs based on API type specs
• controller_gen_objects: generate code containing DeepCopy and DeepCopyInto
• controller_gen_rbac: generate ClusterRole objects
• controller_gen_webhooks: generate (partial) {Mutating,Validating}WebhookConfiguration objects.

As there is a circular dependency between API types and its zz_generated.deepcopy.go, committing the generated file is recommended.

The write_source_file rule can be used to achieve that and have a diff test for the file:

load("@bazel_lib//lib:write_source_files.bzl", "write_source_files")
load("@io_github_janhicken_rules_kubebuilder//kubebuilder:defs.bzl", "controller_gen_objects")
load("@rules_go//go:def.bzl", "go_library")

go_library(
    name = "api",
    srcs = [
        "types.go",
        "groupversion_info.go",
        "zz_generated.deepcopy.go",
    ],
    # [...]
)

controller_gen_objects(
    name = "deepcopy.go",
    srcs = [":api"],
)

write_source_files(
    name = "generate",
    files = {"zz_generated.deepcopy.go": ":deepcopy.go"},
)

The envtest_test rule is a wrapper around a simple go_test that makes the envtest toolchain available to the test. It is convenient to use envtest_test as a replacement for the go_test rule like this:

load("@io_github_janhicken_rules_kubebuilder//kubebuilder:defs.bzl", go_test = "envtest_test")

go_test(...)

When running env.Start() in the Go test implementation, the envtest toolchain is detected through the KUBEBUILDER_ASSETS environment variable that the envtest_test rule sets up.

The kustomization rule works like calling kustomize build. Instead of writing a kustomization.yaml spec, the directives are specified in your build. This way, dependencies to other rules' outputs can be specified easily. For example, the output of the controller_gen_rbac rule can be included into a full set of manifests.

Furthermore, config maps can be created based on other file targets using the config_map rule. Similarly, generic or TLS secrets can be created using the rules generic_secret and tls_secret, respectively.

The following example creates a config map with the file contents of config_a.txt and context_b.txt. The resulting config and the deployment.yaml is then grouped to a set of manifests. The namespace for all resources is set to my-project.

load("@io_github_janhicken_rules_kubebuilder//kubebuilder:defs.bzl", "config_map", "kustomization")

config_map(
    name = "my_config",
    srcs = [
        "config_a.txt",
        "config_b.txt",
    ]
    config_map_name: "my-config",
)

kustomization(
    name = "my_project",
    resources = [
        ":my_config",
        "deployment.yaml"
    ],
    namespace = "my-project",
)

See the kustomization rule's documentation for the full set of supported directives.

kustomization targets are runnable, causing the manifest to be applied to the currently selected kubectl context. When applying, CRDs are applied first and awaited to be Established, before all other resources get applied.

When building container images using rules_oci, building reproducible container images for Go applications is easily achievable. The resulting digest of such an image or multi-platform image index is static.

rules_kubebuilder provides support for injecting digest references for images in Kubernetes pod specs as part of the kustomization rule. Using this feature, image references can be pinned to a certain version of an image.

As software development is conducted both on x86_64 and aarch64 CPU architectures due to the prevalence of Apple Silicon nowadays, it is recommended to always create multi-platform image index for at least those two architectures.

The following example creates such an index for a controller manager as well as a target for creating a tarball of it.

load("@bazel_lib//lib:tar.bzl", "tar")
load("@rules_oci//oci:defs.bzl", "oci_image", "oci_image_index", "oci_load")

tar(
    name = "cmd_layer",
    srcs = ["//cmd"],
    compress = "zstd",
    mtree = ["manager uid=0 gid=0 mode=0755 time=0 type=file content=$(location //cmd)"],
)

oci_image(
    name = "manager",
    base = "@distroless_static",
    entrypoint = ["/manager"],
    tars = [":cmd_layer"],
)

oci_image_index(
    name = "manager_index",
    images = [":manager"],
    platforms = [
        "@rules_go//go/toolchain:linux_amd64",
        "@rules_go//go/toolchain:linux_arm64",
    ],
)

oci_load(
    name = "oci_load",
    format = "oci",
    image = "//:manager_index",
    repo_tags = ["myrepo.org/manager"],
)

filegroup(
    name = "manager_index.tar",
    srcs = [":oci_load"],
    output_group = "tarball",
)

The tarball built with manager_index.tar can be easily referenced in kind_env targets.

In order to pin an image reference to a specific digest, use the kustomization rule's image_digests attribute. This example pins the image myrepo.org/manager to the digest of the index built by :manager_index.

kustomization(
    name = "manager",
    image_digests = {"myrepo.org/manager": ":manager_index.digest"},
    resources = ["deployment.yaml"],
)

See the cronjob tutorial for an example on how to build the container image and pin images with digests.

The kind_env rule can be used to define a local dev environment.

When running the rule with bazel run, it will

1. create a kind cluster with the given name, if not existing yet,
2. write a kubeconfig file named ${cluster}-kubeconfig.yaml to the repository root,
3. load all OCI container image tarballs specified into the cluster and
4. apply a kustomization, if given.

Running the rule multiple times is idempotent.

load("@io_github_janhicken_rules_kubebuilder//kubebuilder:defs.bzl", "kind_env")

kind_env(
    name = "kind_env",
    cluster_name = "my-project",
    images = ["//:my-image.tar"],
    kustomization = "//config/local",
)

The kind cluster can be deleted by giving delete as an argument, for example:

bazel run :kind_env delete

The chainsaw_test rule runs a Chainsaw tests hermetically on a kind cluster.

This rule depends on the kind_env rule for setting up the kind cluster environment. The test will re-use any cluster with the same name that is already running.

IMPORTANT: The tag supports-graceful-termination is required on the target in order to have Bazel allow the test to terminate gracefully and clean up the kind cluster when interrupted. Pressing Ctrl+C might be a source for such an interruption. See the example below on how to set the tag.

chainsaw_test(
    name = "e2e",
    size = "large",
    srcs = glob(["my-test/*.yaml"]),
    tags = [
        "requires-network",
        "supports-graceful-termination",
    ],
)