Repository navigation
fix(library): vendor the sunsingerus mergo fork so the operator imports as a Go module - #2160
Merged
alex-zaitsev merged 4 commits intoNov 19, 2026
Conversation
…ts as a Go module
Upstream wires mergo.WithSliceDeepMerge (used by the exported API
packages - type_chi.go, type_templates.go, type_configuration_chop.go,
type_chk.go) through a replace directive:
replace github.com/imdario/mergo => github.com/sunsingerus/mergo
Go replace directives do not propagate to consumers importing this
repository as a library module. Any downstream require of
clickhouse-operator fails at compile with:
undefined: mergo.WithSliceDeepMerge
(Raised as Altinity#2150, closed without a fix.)
This vendors the sunsingerus fork's stdlib-only sources at
pkg/util/mergo (BSD-3-Clause, LICENSE included, provenance pinned to
fc6fffa94450) and swaps the four API packages' imports to it. The
non-propagating replace and the imdario require are gone from go.mod.
Net effect: a clean module consumer builds the exported API packages
with zero consumer-side replaces:
require github.com/altinity/clickhouse-operator v0.27.4
No behavior change - the vendored sources are byte-identical to the
fork at the pinned commit (provenance.go documents the pin; do not
modify, re-vendor wholesale).
Context: chinit (the cloud-adaptive node bootstrapper for stateful
ClickHouse fleets on VMs) consumes the operator's config generation,
keeper machinery and API types as a library. The k8s-client wiring
stays upstream; the mergo vendoring is what makes that possible
without a fork.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
alex-zaitsev
self-requested a review
November 18, 2026 09:08
Member
|
@sks , it does not compile after rebase to 0.28.0. Please fix or apply a patch: |
The merge of 0.28.0 brought in pkg/apis/clickhouse.altinity.com/v1/merge.go importing github.com/imdario/mergo directly, which broke the build once the replace was dropped. Switch it to the vendored package (fixes the MergoOptions type mismatch in type_chi.go) and drop the now-unused import in type_templates.go.
…brary-consumption
sks
added a commit
to stackgenhq/chinit
that referenced
this pull request
Nov 18, 2026
… (backupconfig, live version, SecretGetter) (#15) ## What Adopts three `clickhouse-operator` library surfaces that chinit was hand-rolling, on top of a dependency bump to the operator 0.28.0 fork tag — plus a Docker Compose acceptance test wired into CI. ### Dependency bump: `v0.27.4` → `v0.28.0` (`v0.28.0-stackgen.1`) The fork tag carries the mergo vendoring (PR Altinity/clickhouse-operator#2160) that makes the operator importable as a library. 0.28.0 itself brings the template-precedence overhaul, the digest-pinned image version fix (#2151), URL-encoding of credentials with reserved characters (#2123), and the no-default-password auth fix (#2156) — the last two land in chinit's connection and user paths for free. ### 1. `backupconfig.Build` for clickhouse-backup's config.yml — **fixes a storage-path bug** `configureBackup` hand-formatted the config and wrote `s3.prefix` — but clickhouse-backup's config struct has **no `prefix` field**: it reads `s3.path` (verified against [pkg/config/config.go at v2.5.0](https://github.com/Altinity/clickhouse-backup/blob/v2.5.0/pkg/config/config.go), the version pinned in the golden AMI). go-yaml silently drops unknown keys, so **backups were landing at the bucket root instead of under `<prefix>/<cluster>/`**. The adoption builds the config through the operator's `pkg/model/chi/backupconfig.Build` (new in 0.28.0, stdlib-only — the same builder the operator uses for its sidecar config Secrets): - `s3.path` now carries the prefix (the bug fix) - `object_disk_path` defaults beside the backups (`object_disks/<prefix>/<cluster>`) - the inline admin password (the 2.5.0 `password_file` compatibility pin) and every `general` setting are byte-identical in effect ### 2. Live version for `ClusterSchemer` — removes a hardcoded `24.3` `SchemaClientFactory.NewClient` hardcoded `swversion.NewSoftWareVersion("24.3")` against 26.8 fleets. The schemer gates SQL shapes on that version (`sqlCreateDatabaseDistributed` switches `engine_full` on `>= 22.12`); a stale value takes the wrong branch silently. `detectSoftwareVersion` mirrors the operator's own app-based detection (`getHostSoftwareVersion` → `HostClickHouseVersion` → `SELECT version()`): one query per inventory host until one answers, bounded by 5s, falling back to the operator's `MinVersion()` when nothing answers — exactly the operator's "unable to acquire" behavior. ### 3. `SecretGetter` seam for the normalizer — fixes a latent crash The operator's normalizer resolves `security.clickhouse.tls.rootCASecretRef` **at load time** through `subst.SecretGetter`. chinit passed `nil` — a spec carrying the ref **panics** the loader (nil func call in `FetchSecretFieldValue`), not a clean rejection. `LoadCHIRequest` now takes a `SecretSource`; `secretGetter` bridges it onto the operator seam: - JSON secrets map every field to a `Data` key (the operator's key lookup finds `ca.crt` etc.) - single-value secrets expose the value under the secret's own name - the namespace is accepted and dropped (cloud secret stores are not namespaced) - nil source keeps the historical behavior — specs without refs are unaffected The boot and remote-servers loaders pass the provider's secret store (the same source the users renderer resolves `passwordFrom` with). ### 4. E2E acceptance test (`acceptance_test/e2e.go`) — write on writer, read on reader A one-command gate for the whole stack, run by CI on every PR: ```bash go run ./acceptance_test/e2e.go ``` The harness builds the `chinit-node` image from the **current checkout**, boots the compose cluster on fresh volumes, and verifies: 1. Keeper answers `ruok → imok` 2. `chinit-boot` finished on writer-0 and reader-0 (marker + live SQL ping) 3. The `compose-demo` cluster lists both replicas on both nodes 4. **A row written on writer-0 arrives on reader-0** (ReplicatedMergeTree replication — the core promise), with a per-run marker so a reused cluster can't pass on stale data 5. The `events` replica is writable with all replicas active Flags: `-keep`, `-reuse`, `-timeout`, `-cluster`, `-repo`. A failing phase prints `compose ps` and the affected nodes' journals so CI output is self-diagnosing. New CI job: `e2e (docker compose)`. **Verified locally on Docker Desktop** against this branch: three consecutive green runs (full rebuild+boot+write→read, `-keep`, `-reuse`) with distinct markers each. ## What is deliberately NOT adopted - **User password resolution via the normalizer**: the operator's `valueFrom.secretKeyRef` path wires **pod ENV vars** (`AppendAdditionalEnvVar`) — meaningless off-cluster. chinit's render-time `passwordFrom` resolution stays. - **Keeper mntr parsing**: the operator has no 4lw client (its raft-safety code marks mntr an extension point); chinit is ahead here. ## Behavior preserved - Full unit suite green: 13/13 packages, `-count=1`, conformance included. - `configureBackup` config semantics: same admin/inline-password shape, same retention/bucket/region/compression wiring, timer install untouched. - Clusterops' loader keeps the nil-source default (it has no provider). - CHK loading is untouched (no secret seam in the keeper normalizer). ## Notes - `k8s.io/api` moves to a direct require — the specloader builds a `core.Secret` for the seam. - Fork tag `v0.28.0-stackgen.1` points at the mergo-vendored merge of upstream 0.28.0 head (`fb1294eb8`). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
alex-zaitsev
approved these changes
Nov 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Upstream wires
mergo.WithSliceDeepMerge— used by the exported API packages (type_chi.go,type_templates.go,type_configuration_chop.go,type_chk.go) — through areplacedirective:Go
replacedirectives do not propagate to consumers importing this repository as a library module. Any downstreamrequireof clickhouse-operator fails at compile with:(Raised as #2150, closed without a fix. (Supersedes the just-closed #2158 — same commits, moved to the org fork.))
The fix
Vendor the sunsingerus fork's stdlib-only sources at
pkg/util/mergo(BSD-3-Clause, LICENSE included, provenance pinned tofc6fffa94450inprovenance.go) and swap the four API packages' imports to it. The non-propagatingreplaceand theimdario/mergorequire are gone fromgo.mod.Net effect
A clean module consumer builds the exported API packages with zero consumer-side replaces:
Verified with a scratch consumer module (require, import
pkg/apis/clickhouse.altinity.com/v1, exercise themergo.WithSliceDeepMergepath) before and after: fails on stock upstream, passes with this patch.No behavior change — the vendored sources are byte-identical to the fork at the pinned commit.
Why we care
stackgenhq/chinit — the cloud-adaptive node bootstrapper for stateful ClickHouse fleets on VMs (EC2 ASGs, Azure VMSS) — consumes the operator's config generation, keeper/chk machinery, and API types as a plain Go library. Today that requires carrying a fork of this repository solely for this vendoring. Landing this patch would let us drop the fork and require upstream directly, and we believe it unblocks library consumption for anyone else embedding the operator's model/config layer outside Kubernetes.
Follow-up we would like to explore (separate PRs)
pkg/chopconfig accessors (Config(), the Access/Username reads) from thek8s.io/client-goclient wiring — today importingpkg/chopfor config drags the whole k8s client surface into a non-k8s consumer's tree.Happy to adjust (path, naming, docs) to fit your conventions.