Skip to content

Fix(config): Nacos sync follow-ups from #7457 - #7467

Merged
briankane merged 2 commits into
kubevela:masterfrom
guidewire-oss:feat/7410-config-nacos-sync-followups
Oct 15, 2026
Merged

briankane merged 2 commits into
kubevela:masterfrom
guidewire-oss:feat/7410-config-nacos-sync-followups

Conversation

@jerrinfrancis

@jerrinfrancis jerrinfrancis commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Description of your changes

Follow-up to #7457 (Part of #7410, PR checklist box 4): the five inline comments from the
review, four as changes here and one tracked for the scope work (box 6).

A read never serves the SDK's snapshot (nacos.go:220). With nacos-sdk-go v2.2.2, a
GetConfig whose server query fails returns the snapshot the SDK wrote after the last
successful read, with no error. The read-before-publish then compared against stale content:
revert a change while Nacos is briefly down and the controller recorded AlreadyCurrent while
Nacos kept the newer value, with nothing left to retry once the hash matched. The client now
sets DisableUseSnapShot, so that read is ReadFailed with backoff, and each endpoint has its
own cache directory (the SDK keys entries by dataId, group and namespaceId alone, so two servers
with the same ids would otherwise share them).

group is optional again (sync.go:85). The SDK publishes to and reads from
DEFAULT_GROUP when the group is empty, and templates written before the controller published
never had to set it; requiring it put every Config of such a template into Error after the
upgrade and refused edits to the template. The kind check stays (group: 42 is refused) and an
empty group is normalised to DEFAULT_GROUP in the render, so the hash, the status, the read and
the publish name the same group.

The nacos block comes out of the Secret's compile (nacos_sync.go:71). The reconciler and
the CLI's ParseConfig read template.nacos from the value the CueX render already produced,
instead of rendering a second time with the plain compiler. A template that imports a vela
package renders like any other (it was admitted and then failed at reconcile with
builtin package "vela/config" undefined), and a template's provider calls run once per
reconcile rather than twice.

One Nacos client per sync (nacos_sync.go:116). syncNacos builds the client once and
hands it to the read and the publish through NacosData.Client, releasing it afterwards. Every
construction re-initialises the SDK's global logger and opens a rolling log writer nothing
closes, so one per sync that reaches Nacos is what the controller pays.

The endpoint is watched and part of "current" (nacos_sync.go:166, the watch half). A
change to an endpoint config Secret (a velacore-config Secret whose properties carry servers
or client) enqueues the Configs with a nacos block, so EndpointNotFound clears when the
endpoint appears rather than on backoff. The status gains endpoint, the endpoint config the
template names, recorded on every sync attempt whether it succeeded or not, and
endpointHash, a digest of that config's server part (addresses, ports, discovery endpoint,
region; never its credentials, the status being readable by whoever can read Configs), and the
fast path requires the digest to match too: a Config pointed at another server re-syncs the way
changed content does, read first, published only when the server does not hold it. The watch
finds the Configs that use a changed, created or deleted endpoint config through a field index
on that reference, the ones still waiting for it included. A synced Config whose endpoint is
deleted reports EndpointNotFound instead of staying True. On upgrade every stored status
lacks the digest, so each Config with a nacos block is read once and recorded AlreadyCurrent;
nothing is published for that. The access half of the same comment (endpoint credentials read with the controller's
permissions where the CLI read them as the user) is tracked for the scope enforcement PR: an
endpoint in the template's own namespace or in vela-system allowed, elsewhere a
SubjectAccessReview for get on the Secret as the requesting user, at ConfigTemplate admission.

One commit: Fix(config): Nacos sync follow-ups from #7457.

Follow-up to #7457. Part of #7410

I have:

  • Read and followed KubeVela's pull request process.
  • All commits are signed off (DCO).
  • Related Docs updated properly. In a new feature or configuration option, an update to the documentation is necessary.
  • Run make reviewable to ensure this PR is ready for review.
  • Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

  • pkg/config/writer: the client param carries DisableUseSnapShot and a per-endpoint cache
    directory; NewClient is the shared seam (one construction for a read and a publish, released
    once); renderNacos reads a rendered value, renders a template behind a vela import and
    defaults an empty group; ValidateExpandedWriter passes a block without group and still
    refuses group: 42; EndpointHash is stable across key order, changes with a server or the
    discovery endpoint, ignores the credentials, and leaves the caller's map alone.
  • Config controller (envtest): one client per sync and every client released; the status
    records the endpoint on success and while waiting on EndpointNotFound or a failed publish;
    an endpoint Secret change re-syncs the Config through the watch with a read and no publish,
    and the endpoint digest changes while the content hash does not; a template without a group
    publishes to DEFAULT_GROUP and records it; the index key function.
  • Webhook, configtemplate, migration, pkg/config and the CLI config suites green; lint clean.
  • Manual, on a k3d cluster with a Nacos 2.4.3 container, upgraded from PR 4 as merged
    (262341c0c) to this branch: the stale-snapshot bug reproduced on the merged image (edit,
    stop Nacos, revert: AlreadyCurrent recorded, Nacos keeps the edit after the restart) and
    healed by the upgrade (the new digest forces one read, the revert is published); with the fix
    the same revert is ReadFailed until Nacos is back and then AlreadyCurrent; a template
    without a group admitted and published to DEFAULT_GROUP; the template behind vela/config
    renders through the CLI and the controller; a Config whose endpoint arrives later syncs within
    seconds of the endpoint's creation, re-syncs without a publish when the endpoint gains a
    field, fails cleanly on a wrong port, and reports EndpointNotFound when the endpoint is
    deleted; one cache directory per endpoint in the controller's /tmp/nacos/cache.

Special notes for your reviewer

  • The hash scheme does not change here; endpointHash is a second field next to
    contentHash, additive on the CRD, and an upgrade costs one read per nacos Config, never a
    publish.
  • The endpoint read moved before the fast path (it is one cached GET): a synced Config now
    notices a deleted endpoint and says so, where before it stayed True.
  • The watch is an indexed lookup (status.expandedWriter.nacos.endpoint, registered in
    SetupWithManager), gated on the velacore-config label, so a Secret event costs one cache
    query for the Configs that name it and nothing for the rest. The reference is written in
    nacosFailed as well as nacosSynced, which is what lets a waiting Config be found by name;
    a Config whose status predates the field is indexed at its next reconcile, which every Config
    gets at controller start.
  • The endpoint Secret is read with config.ReadConfigSecret, the Secret-level half of the
    legacy ReadConfig, so the controller and the CLI refuse a sensitive one the same way.
  • A template with a nacos block no longer tolerates a partial render: config.RenderTemplate
    holds the one rule (the IsFieldNotExist allowance stays for templates without the block,
    as the legacy path always had it), and the reconciler and ParseConfig both call it, so the
    real CUE error is the one reported. Making RunAndOutputWithCueX stop producing that error
    class for a template without template.parameter is the deeper fix and is left for its own
    change, since the webhook shares the allowance.
  • The SDK re-initialises its logger on every client construction with no opt-out, so one
    client per sync is the floor without a per-endpoint client cache; that cache would need the
    invalidation this PR's watch now provides, so it is a natural next step if the logger shows up
    in profiles.
  • The CLI's render shares the fix: vela config create from a template behind a vela import
    used to fail with the plain compiler's error before writing the CR.

View guided diff

Brian's five review comments on kubevela#7457, four as changes here and the
fifth's access half tracked for the scope enforcement work.

The Nacos client no longer falls back to the SDK's snapshot: with
nacos-sdk-go v2.2.2 a GetConfig whose server query fails returned the
snapshot of the last successful read with no error, so a revert while
Nacos was down compared against stale content, recorded AlreadyCurrent
and never retried once the hash matched. DisableUseSnapShot is set, so
that read is ReadFailed with backoff, and each endpoint has its own
cache directory because the SDK keys entries by dataId, group and
namespaceId alone.

group is optional again, kind-checked when present, and an empty group
is normalised to DEFAULT_GROUP in the render, the group the SDK uses;
requiring it had put every Config of a group-less template into Error
after the upgrade.

The nacos block is read from the value the Secret render produced,
in the reconciler and in the CLI's ParseConfig, instead of a second
render with the plain compiler: a template that imports a vela package
renders like any other and a template's provider calls run once.

syncNacos builds one Nacos client and hands it to the read and the
publish through NacosData.Client; every construction re-initialises
the SDK's global logger and opens a log writer nothing closes.

The status records the endpoint the template names on every sync
attempt, success or failure, and the Secret watch finds the Configs
that use a changed, created or deleted endpoint config through a field
index on it, so EndpointNotFound clears when the endpoint appears
instead of on backoff. The status also gains endpointHash, a digest of
the endpoint's server part (never its credentials: the status is
readable), and the fast path requires it to match: a Config pointed at
another server re-syncs read-first, and a synced Config whose endpoint
is deleted reports EndpointNotFound. On upgrade each Config with a
nacos block is read once and recorded AlreadyCurrent; nothing is
published for that.

The render of a template with a nacos block fails on any compile error
instead of the "field does not exist" allowance the Secret render
keeps for templates without one, so the real CUE error is reported.

Follow-up to kubevela#7457. Part of kubevela#7410.

Signed-off-by: Jerrin Francis <jerrinfrancis7@gmail.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 14 files

Tip: Get a Slack DM when cubic finishes reviewing your PRs. Connect Slack

View guided diff | Re-trigger cubic

Comment thread pkg/config/writer/nacos.go Outdated
Comment thread pkg/config/writer/sync.go
cubic round 1 on kubevela#7467. The per-endpoint cache directory joined the
endpoint's namespace and name into one path component, which a
253-character Secret name pushes past the 255-byte file name limit;
they are two components now. The ValidateExpandedWriter comment says
group is optional, as the code has it.

Signed-off-by: Jerrin Francis <jerrinfrancis7@gmail.com>
@codecov

codecov Bot commented Oct 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.28829% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.84%. Comparing base (035d51f) to head (810a352).

Files with missing lines Patch % Lines
...oller/config.oam.dev/v1alpha1/config/nacos_sync.go 87.75% 6 Missing ⚠️
...onfig.oam.dev/v1alpha1/config/config_controller.go 76.47% 4 Missing ⚠️
pkg/config/factory.go 76.92% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7467      +/-   ##
==========================================
- Coverage   79.85%   79.84%   -0.01%     
==========================================
  Files         460      460              
  Lines       48089    48136      +47     
==========================================
+ Hits        38401    38436      +35     
- Misses       9685     9697      +12     
  Partials        3        3              
Flag Coverage Δ
core-unittests 79.84% <88.28%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jerrinfrancis

Copy link
Copy Markdown
Collaborator Author

/retest e2e-helm-test

@jerrinfrancis

Copy link
Copy Markdown
Collaborator Author

/retest-failed e2e-test e2e-multicluster-test

@briankane
briankane merged commit 47709e1 into kubevela:master Oct 15, 2026
59 of 61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants