Skip to content

fix(components): propagate Component spec.dependencies to CMS dependency graph - #989

Merged
VLukiantsev merged 5 commits into
mainfrom
fix/cms-component-spec-dependencies
Sep 28, 2026
Merged

VLukiantsev merged 5 commits into
mainfrom
fix/cms-component-spec-dependencies

Conversation

@VLukiantsev

@VLukiantsev VLukiantsev commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The Component CR's spec.dependencies now reaches the CMS dependency graph. As a result, cycle detection, deactivation safety and deletion safety all apply to dependencies declared on the CR. Self-dependencies are rejected.

Stack 1/4: this PR → 1P health (B) → dependency-gated actuation (C1) → dependency-gated deactivation (C2).

Motivation & Context

The CRD declares spec.dependencies, and 1P blueprints set it (e.g. ingestion-gateway → redpanda), but the value was dropped along the way:

  • ParseSpecFromUnstructured ignored it;
  • ComponentSpec had no field for it;
  • the store only derived dependencies from description directives and manifests.

So CMS allowed disabling or deleting redpanda while ingestion-gateway depended on it.

Key Changes

  • Proto: ComponentSpec.dependencies = 10 (repeated string). Go code is generated by Bazel, so there are no checked-in .pb.go files.
  • Bridge: parseDependencies canonicalizes and dedupes entries and drops malformed ones. dropSelfDependency removes self-references before sending the intent.
  • Store: ExtractDependencies merges spec.dependencies with directive- and manifest-derived dependencies. ApplySpecUpdateMask supports a dependencies path.
  • Store errors: resolveDependencyIDs names the missing dependency, using the exported DependencyNotFoundMessagePrefix. The dead ErrDependencyNotFound is removed.
  • CMS: SetComponentIntent returns InvalidArgument for dependency names that aren't DNS-1123, and for self-dependencies.
  • Docs: READMEs describe dependency sources, and note that Spanner rejects unregistered dependencies while MemoryStore does not.
  • No DB migration: the existing ComponentDependencies table is reused.

Assumptions

  • Ordering on Spanner: a dependent reconciled before its dependency is registered gets FailedPrecondition "dependency not found: …". — The bridge gating PR (C1) treats this as "waiting" instead of an error. — If this PR lands without C1, fresh installs briefly show IntentApplicationFailed until the dependency registers.
  • Deletion with Delete/Unspecified policy still cascades to dependents rather than blocking. — This is existing store semantics and is unchanged.

Verification & Testing

  • Commands Executed: bazel test //pkg/components/... //cmd/xmc_bridge/... //api/... //cmd/experiment_state_server/... //cmd/health_service/... (all pass), ./.githooks/pre-commit (pass).
  • New tests:
    • bridge parse and self-dependency drop;
    • store merge, deactivation block, Orphan deletion block and cycle rejection, using spec.dependencies only;
    • field mask;
    • resolveDependencyIDs;
    • CMS pass-through and invalid/self-dependency rejection.
  • Live (whole stack, GKE ...): with the gateway running, kubectl patch component redpanda … enabled=false was blocked: Cannot disable "redpanda": component redpanda has active dependents: [ingestion-gateway]. Full evidence is in the C2 PR.

Related Issues / Bugs

Found while working on bug 548435582.

TAG=agy
CONV=baca6192-74d3-4579-9f73-0a3263d9a2a7

@VLukiantsev
VLukiantsev added this pull request to stack #993 September 25, 2026 12:12
@VLukiantsev VLukiantsev changed the title fix/cms component spec dependencies fix(components): propagate Component spec.dependencies to CMS dependency graph Sep 25, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a structured dependencies field to the ComponentSpec protobuf message and integrates it across the components subsystem. It updates the GitOps bridge to parse, canonicalize, and deduplicate dependencies from unstructured resources while dropping self-dependencies. Additionally, it implements dependency name validation in the CMS server, updates the dependency extraction logic, supports the new field in field masks, and improves error reporting for unregistered dependencies by replacing ErrDependencyNotFound with a detailed FailedPrecondition error. Comprehensive unit tests and documentation updates have been added to support these changes. There are no review comments, so I have no feedback to provide.

@VLukiantsev

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for a structured dependencies field in the ComponentSpec protobuf and Component Custom Resource (CR). It updates the GitOps bridge reconciler to parse, canonicalize, and deduplicate these dependencies while dropping self-dependencies. Additionally, it enhances validation in the CMS server to reject invalid dependency names and self-references, integrates the new field into the dependency extraction and field mask logic, and improves error reporting when dependencies are not registered by replacing the generic ErrDependencyNotFound with a descriptive FailedPrecondition error. Comprehensive unit tests have been added across the reconciler, server, and store packages to verify these changes. There are no review comments, so we have no additional feedback to provide.

pachopepe
pachopepe previously approved these changes Sep 25, 2026
Comment thread pkg/components/bridge/reconciler.go Resolved
Comment thread pkg/components/bridge/reconciler.go Resolved
…ncy graph

The Component CRD declares spec.dependencies and 1P blueprints set it
(e.g. ingestion-gateway -> redpanda), but the bridge dropped the field
and ComponentSpec had no way to carry it. CMS cycle detection,
deactivation safety, and deletion safety therefore never saw
CR-declared dependencies.

- Add `repeated string dependencies = 10` to ComponentSpec.
- Bridge: parse spec.dependencies (canonicalized, deduped, ignoring
  empty/non-string entries) in ParseSpecFromUnstructured.
- Store: ExtractDependencies merges spec.dependencies with manifest,
  description, and label sources; support the "dependencies" update
  mask path.
- CMS: reject invalid dependency names with InvalidArgument instead of
  silently dropping them.
- Spanner: name the missing dependency in the FailedPrecondition error.

BUG=b/548435582
TAG=agy
…NotFound

Address review feedback on spec.dependencies propagation:

- Bridge: ReconcileComponent drops self-references from
  spec.dependencies (with a warning) so a component can never wait on
  itself once dependency readiness gating lands.
- CMS: SetComponentIntent rejects self-dependencies with
  InvalidArgument (defense in depth; the store would otherwise report a
  FailedPrecondition self-cycle).
- Store: remove the now-unused ErrDependencyNotFound sentinel; the
  Spanner path returns a FailedPrecondition naming the missing
  dependency instead.
- Docs: note that MemoryStore (dev/test) does not reject unregistered
  dependencies whereas SpannerStore does, and document the rollout
  ordering interaction: on Spanner, reconciling a dependent (e.g.
  ingestion-gateway) before its dependency (redpanda) is registered
  yields FailedPrecondition "dependency not found" and an
  IntentApplicationFailed condition until retried. Explicit readiness
  gating before SetComponentIntent is left to the follow-up PR.

BUG=b/548435582
TAG=agy
…red dependencies

Clients need a stable way to recognize the FailedPrecondition returned
when a component declares a dependency that is not registered yet. The
GitOps bridge treats it as "waiting on dependencies" (gated, no error)
instead of IntentApplicationFailed. The message text is unchanged.

BUG=b/548435582
TAG=agy
…ize self-dependency check

parseDependencies now passes strings with an empty canonical name through
verbatim so CMS validateDependencyNames rejects them with InvalidArgument
instead of the bridge silently dropping a dependency edge.
dropSelfDependency canonicalizes entries before comparing, so direct
ReconcileComponent callers passing components/<self> behave like Reconcile.

Comment link: #989 (comment)
Comment link: #989 (comment)

BUG=b/548435582
TAG=agy
CONV=baca6192-74d3-4579-9f73-0a3263d9a2a7
@VLukiantsev
VLukiantsev force-pushed the fix/cms-component-spec-dependencies branch from 3b352fd to b861a3d Compare September 28, 2026 08:26
@VLukiantsev VLukiantsev added the priority::2-high top of mind. should not wait. label Sep 28, 2026
Comment thread pkg/components/bridge/reconciler.go Resolved
pachopepe
pachopepe previously approved these changes Sep 28, 2026
The bridge canonicalizes only in parseDependencies, so dropSelfDependency
now compares canonical names directly. CMS replaces validateDependencyNames
with canonicalDependencies, which validates, dedupes and writes the
canonical list back into the persisted spec instead of discarding it.
Store NormalizeDependencies remains the merger of all dependency sources.

Comment link: #989 (comment)

BUG=b/548435582
TAG=agy
CONV=baca6192-74d3-4579-9f73-0a3263d9a2a7
@VLukiantsev
VLukiantsev added this pull request to the merge queue Sep 28, 2026
Merged via the queue into main with commit edb2276 Sep 28, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority::2-high top of mind. should not wait.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants