[LFX Mentorship] Legacy Config backward compatibility and migration to CRD-based Config Management #7410

Sub-issue of #7105 · Follow-up to #7211

What would you like to be added?

A safe path for existing clusters from the legacy ConfigMap/Secret-based config storage to the config.oam.dev/v1alpha1 CRDs, in three parts:

• Unconditional backward compatibility. The controller and the vela CLI always read, show and delete across both backends. All new writes go to CRs. No flag or gate is needed to keep legacy objects working.
• vela config-template migrate, a new command that converts one legacy template and its configs into ConfigTemplate and Config CRs on the operator's schedule.
• AutoMigrateLegacyConfigs, one controller feature gate (Alpha, default off). When on, the controller runs the same migration for every legacy template at startup.

The work also brings the controller up to parity with the imperative path for the Nacos expanded writer, makes GitOps-driven configs behave the same as CLI-driven ones, and closes two API contract gaps, template scope and sensitive handling, before the API ships. After release they would be breaking changes.

# vela-core Helm values
featureGates:
  AutoMigrateLegacyConfigs: false   # Alpha, default off

The work is split into small PRs, tracked in the checklist at the end.

Why is this needed?

#7211 added the CRDs, controllers and webhooks, with migration and upgrade work planned as follow-up before release. On a cluster that already has legacy configs, upgrading today causes these problems:

1. Legacy objects are invisible to the default CLI. Once the CRDs are installed, vela config[-template] list only reads CRs, and vela config[-template] delete reports legacy objects as not found. Configs created by older CLIs, addons or VelaUX can no longer be managed.

2. There's no way to adopt legacy resources into CRs. Existing config-template-* ConfigMaps and config.oam.dev/catalog=velacore-config Secrets stay in legacy storage forever unless they are recreated by hand.

3. Nacos only works through the CLI. The imperative factory calls writer.Write for templates with a nacos block, but the Config reconciler doesn't. A Config applied through GitOps reaches Available without ever writing to Nacos.

4. The workflow provider still writes legacy storage. create-config calls the imperative factory and creates a legacy Secret, not a Config CR.

5. The storage mode is a global CLI flag. --config-mode auto|legacy|crd is a persistent flag on every vela command, so a user can still create legacy templates on a cluster that has the CRDs, and every command has two write paths to test and document.

6. Template scope isn't enforced. ConfigTemplate.spec.scope is stored, printed and copied onto the output Secret's labels, but nothing acts on it. A scope: system template can live in any namespace. A Config can reference a template in any namespace through spec.templateRef.namespace. The webhook and controller then read that template with the controller's own permissions, so anyone allowed to create a Config can render any template in the cluster.

7. Sensitive data can leak through a non-sensitive template. The output Secret's config.oam.dev/sensitive annotation comes only from the template. A Config that reads credentials through spec.propertiesFrom.secretRef against a non-sensitive template ends up with those credentials copied into input-properties and marked sensitive: "false". That makes them readable through vela config read paths and the read-config workflow step. sensitive can also be flipped from true to false on a live template, which re-marks every dependent Secret as readable on the next reconcile.

Proposal

1. CLI behaviour: single write path, dual read

The storage choice is no longer a user decision. Each command behaves as follows once the CRDs are installed:

--config-mode moves from the root command to the two list commands and loses its auto value. It is rejected on every other command.

The workflow provider's create-config writes a Config CR when the CRD is installed. read-config, list-config and delete-config read both backends. The factory path remains the fallback when the CRD is absent.

2. Controller behaviour

• Template resolution for a Config reads the ConfigTemplate CR first and falls back to the legacy config-template-* ConfigMap. This fallback is permanent; nothing switches it off.
• The legacy ConfigMap watch stays on so that a Config referencing a legacy template reconciles when that ConfigMap changes.
• Admission doesn't depend on apply order. A Config applied before or together with its ConfigTemplate is admitted and converges once the template exists; schema checks run only when the template is Available.

3. Migration

Migration is implemented once, in a shared library in pkg/config, and driven from two places: the vela config-template migrate command and the AutoMigrateLegacyConfigs gate.

Mapping rules

# Before: legacy config (Secret)
apiVersion: v1
kind: Secret
metadata:
  name: my-registry
  labels:
    config.oam.dev/catalog: velacore-config
    config.oam.dev/type: image-registry
---
# After: the same Secret, now owned by a Config
apiVersion: config.oam.dev/v1alpha1
kind: Config
metadata:
  name: my-registry
  labels:
    config.oam.dev/created-by: migration
spec:
  templateRef:
    name: image-registry
  propertiesFrom:            # sensitive templates only
    secretRef:
      name: my-registry-properties

• Templates: config-template-<name> becomes ConfigTemplate <name> in the same namespace. template, sensitive, alias, description and the expanded-writer data carry over. Legacy scope global and system map to system; project, env and anything else map to namespace.
• Configs: each legacy Secret becomes a Config with the same name and namespace. templateRef comes from the config.oam.dev/type label. Non-sensitive input goes into spec.properties. Sensitive input goes into a <name>-properties Secret owned by the Config and referenced through spec.propertiesFrom, so it never enters the CR.
• Adopt in place. The existing Secret gets the Config as its controller owner and keeps its name, UID and data. The first reconcile after adoption must produce an identical Secret, including the default type /<template-name>. Objects listed in objects-reference and the distribute-<name> Application also become owned by the Config.
• Idempotent. Adopted sources are marked with config.oam.dev/migrated-to: <uid>. If a CR with the same name already exists and wasn't created by migration, the object is skipped and a MigrationConflict Event is recorded.
• Opt-out per object with config.oam.dev/migrate: "false".
• Legacy objects are never deleted by migration. Downgrading to an older controller keeps working.

vela config-template migrate

vela config-template migrate <name> [-n <namespace>] [--all] [--dry-run]

Migrates the named legacy template and every legacy config that references it. --all migrates every legacy template in the namespace. --dry-run prints the CRs that would be created and the objects that would be adopted, and changes nothing. The command reports adopted, skipped and conflicting objects, and exits non-zero if any conflict.

AutoMigrateLegacyConfigs gate

Added to pkg/features/controller_features.go as Alpha, default off. When on, a leader-elected runnable runs the migration for every legacy template and config in the cluster once at controller startup, then exits. Each run emits Events on the source objects, a summary log line, and the metric kubevela_config_migration_total{result="adopted|skipped|conflict|failed"}. Turning the gate off stops future sweeps; already adopted objects stay adopted.

4. Nacos expanded writer in the controller

• The Config reconciler reads the nacos block from ConfigTemplate.spec.template (or from the legacy ConfigMap's expanded-writer key), renders it with writer.RenderForExpandedWriter, and writes it with writer.Write.
• The endpoint config (nacos.endpoint.name) is resolved from either CR-backed or legacy storage.
• The controller writes only when the rendered content changes, stores a content hash in status, and reports a NacosSynced condition. Network and 5xx failures requeue with backoff; they aren't treated as terminal errors.
• Deleting a Config removes nothing from Nacos, same as the CLI today.
• The ConfigTemplate webhook validates the nacos block.
• Once this lands, the interim legacy fallback in vela config create for Nacos templates is removed.

5. GitOps compatibility

• Legacy objects managed by Argo CD or Flux are adopted like any other. Adoption adds only owner references and the migrated-to annotation, doesn't change Secret data, and uses its own field manager, so the GitOps tool sees no diff. An informational MigratedGitOpsManaged Event tells owners to commit the matching CR to git.
• Two risks are documented. If git later reapplies legacy data that differs from the rendered output, the controller and the GitOps tool will fight until the CR is committed to git. Deleting a migration-created Config also garbage-collects its Secret.
• kubectl delete config and GitOps pruning clean up the same objects as vela config delete: the output Secret, outputs objects, and the distribution Application.

6. Template scope and cross-namespace access

The rules below are enforced in the Config and ConfigTemplate webhooks (failurePolicy: Fail). The reconciler repeats the structural checks as a second line of defence.

• scope: system is only valid in vela-system. The ConfigTemplate webhook denies scope: system in any other namespace. CEL can't express this, because it can't read metadata.namespace.

• Config.spec.templateRef access rules:

  Template	Referenced from	Allowed
  any scope	the same namespace	yes
  scope: system (in vela-system)	any namespace	yes
  scope: namespace	another namespace	only if a SubjectAccessReview shows the requesting user can get that ConfigTemplate

• templateRef is already immutable, so an access check when the Config is created covers its whole lifetime.

• Legacy templates: legacy config-template-* ConfigMaps in vela-system are treated as system scope, both when the controller falls back to them and when migration adopts them. Today they are used from every namespace, so this keeps existing configs working. Legacy templates in other namespaces map to namespace scope.

• Workflow steps: create-config runs as the controller's service account, which would pass any access check. When AuthenticateApplication is on, the provider runs the SubjectAccessReview for the Application's user (app.oam.dev/username). Otherwise it allows only same-namespace or system templates.

• The chart's RBAC stays as it is. The controller still needs to read templates cluster-wide, and the access decision is made per requesting user at admission.

Open question for review: the alternative reading is that Config objects built from a system template must themselves live in vela-system. That would break every addon-shipped template, which today lives in vela-system and is used from application namespaces, so this proposal keeps system templates usable cluster-wide and constrains only where the template itself may live. Please object here if the stricter rule is what's wanted.

7. Sensitive handling

• sensitive can't be turned off. On update, the ConfigTemplate webhook denies sensitive: true → false. A CEL transition rule on ConfigTemplateSpec enforces the same thing where the webhook isn't installed:

  // +kubebuilder:validation:XValidation:rule="!has(oldSelf.sensitive) || !oldSelf.sensitive || (has(self.sensitive) && self.sensitive)",message="sensitive cannot be changed from true to false"

• Turning sensitive on (false → true) is allowed. Dependent Configs that still carry inline spec.properties move to Error with a message telling the user to switch to propertiesFrom. Their existing Secret is marked sensitive right away.

• propertiesFrom always means sensitive. A Config that uses spec.propertiesFrom gets an output Secret marked config.oam.dev/sensitive: "true", whatever the template says. Read paths (vela config, read-config, list-config) treat it as sensitive and don't return the input properties.

• Migration keeps sensitive as it is on templates, and marks adopted configs as sensitive when they end up using propertiesFrom.

Verification plan

• Unit tests: --config-mode accepted on list and rejected elsewhere; dual-read merge, shadowing and SOURCE column; apply fails without the CRD and warns on a shadowed legacy template; dual-delete routing; legacy-to-CR mapping for templates and configs, sensitive and non-sensitive, including the scope mapping; the adoption parity check (identical Secret after the first reconcile); migrate --dry-run output and idempotency; Nacos render and change detection.
• Webhook tests: scope: system outside vela-system is denied; each row of the templateRef access table, with the SubjectAccessReview allowed and denied; sensitive true → false is denied; false → true is admitted; a propertiesFrom Config against a non-sensitive template produces a sensitive Secret.
• envtest controller tests: startup sweep with the gate on; running the sweep twice changes nothing; conflict and opt-out handling; legacy fallback and watch always active; Nacos retry on transient errors; the CEL transition rule on sensitive, with no webhook; dependent Configs going to Error after sensitive is turned on.
• RBAC e2e: as a non-admin user bound to one namespace, check that a Config referencing a system template is admitted; one referencing a namespace-scoped template in another namespace is denied, then admitted once the user is granted get on it; and a create-config workflow step with AuthenticateApplication on applies the same rule to the Application's user.
• Upgrade e2e (kind, old image → new image): on the old controller, create a plain config, a sensitive config, a config with outputs, a distributed config, a Nacos config, a config created by the workflow step, and an addon-shipped template. Upgrade, then check:
  1. vela config and vela config-template list, show and delete, with both the old and the new CLI, with the gate off.
  2. Applications that consume these configs keep working, with no Secret churn (same UID and resourceVersion until something is edited).
  3. vela config-template migrate --all adopts each object exactly once; a second run changes nothing.
  4. With the gate on, a controller restart adopts whatever is left exactly once, and a further restart changes nothing.
  5. Nacos content stays the same, and a later edit through the Config CR reaches Nacos.
  6. Downgrading to the old controller works with the legacy objects intact.
• Nacos e2e: against a standalone nacos/nacos-server container, a Config applied with kubectl alone shows up under the expected dataId and group.
• GitOps e2e: Argo CD (or Flux) syncs a ConfigTemplate and a Config, including a Nacos template. The sync is healthy, there's no diff loop after reconcile, pruning removes everything, and an adopted GitOps-managed legacy Secret shows no diff.
• Docs: the gate in the Helm chart featureGates values, the migrate command and the list --config-mode flag in the CLI reference, along with the upgrade and rollback steps.

PR checklist

Each PR is small and links back to one box.

• CLI backward compatibility: apply always CR, --config-mode scoped to list with merged default and SOURCE column, dual-delete for configs and templates, show and create verified against both backends (section 1)
• Shared migration library and vela config-template migrate (section 3)
• AutoMigrateLegacyConfigs gate: startup sweep, Events, metric, opt-out (section 3)
• Nacos expanded writer in the Config reconciler, remove the create fallback (section 4)
• Workflow provider: create-config writes a Config CR, read/list/delete read both backends (section 1)
• Scope enforcement in webhooks and reconciler (section 6)
• sensitive rules (section 7)
• GitOps behaviour, upgrade and downgrade e2e, docs (section 5, verification plan)

Out of scope (potential follow-ups)

• VelaUX changes.
• Event-driven adoption of legacy objects created after the startup sweep (for example by an older vela binary). Rerun vela config-template migrate or restart the controller with the gate on.
• Removing legacy storage, and a deprecation timeline.