Skip to content

refactor(export-configs): minimal-diff output + masked-secret placeholders#900

Open
vvnpn-nv wants to merge 3 commits into
mainfrom
vpan/export-configs-improvements
Open

refactor(export-configs): minimal-diff output + masked-secret placeholders#900
vvnpn-nv wants to merge 3 commits into
mainfrom
vpan/export-configs-improvements

Conversation

@vvnpn-nv

@vvnpn-nv vvnpn-nv commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Turns export_configs_to_helm.py from a full config dump into a minimal override file. Five behavior changes driven by real pain when running this on a 6.2→6.3 migration on sandbox-azure, where the previous output was ~290 lines of mostly chart defaults with literal ********** in every credential field.

Changes

  1. Masked secret values → secretName placeholders. The docs previously claimed this happened but the script emitted access_key: '**********' literally, silently breaking any deployment that applied the YAML as-is. Now every masked field becomes {secretName: TODO-REPLACE-ME, secretKey: <field>} and each path is listed on stderr so the operator knows which K8s Secrets they still need to create.

  2. Drop None values and empty containers recursively. override_url: null, gpu: null, max_num_workflows: null, etc. are Pydantic-default no-ops that just clutter diffs. Stripped post-fetch.

  3. Strip pinned tags in workflow.backend_images.{init,client} with a stderr note. Keeping them silently locks workflow pods to whatever init/client container was running at export time — easy to miss during an upgrade. Falls back to global.osmoImageTag after stripping.

  4. Handle optional endpoints gracefully. HTTP 400 with "relation ... does not exist" + error_code: DATABASE on endpoints in OPTIONAL_CONFIG_ENDPOINTS (currently just group_template) is recognized as "table absent on this schema version" and logged as a one-line (skipped …) note instead of dumping the raw SQL error. Previously looked like a script bug.

  5. Diff against chart defaults. Output is deep-diffed against ../charts/service/values.yaml's services.configs.* block. Scalars equal to the chart default are dropped; a named-dict entry (a role, a podTemplate, a backend, a pool) that exactly matches the shipped default is removed wholesale. --no-strip-defaults restores the full dump for auditing. --chart-values <path> overrides the baseline location.

Also fixes collect_secret_names so it doesn't report TODO-REPLACE-ME as an existing K8s Secret in its stderr summary.

Doc updates

6_2_to_6_3_upgrade.md's "The script" bullet list is rewritten to match the new behavior and explain the two new flags.

Test plan

Exercised end-to-end on sandbox-azure:

  • Export from a live 6.3 service + ConfigMap-backed instance produces 258 lines vs. 288 with the old script
  • group_template 400 logs as (skipped …: table not present on this schema) instead of the SQL error body
  • workflow.backend_images.{init,client} stripped with the stderr note; configmap_loader on re-deploy logs Loaded Docker registry credentials for backend_images.credential from nvcr.io (so the secretName/secretKey Docker-auths branch still resolves correctly after the backend_images.credential rewrite)
  • Masked auth field becomes secretName: TODO-REPLACE-ME, secretKey: auth; stderr lists the path under "Replaced masked secret values with TODO placeholders at"
  • --no-strip-defaults yields the full tree
  • Re-applied the exported minimal values to sandbox-azure via helm upgrade osmo-service — service pod started successfully, /api/version returns 200, workflows submit/complete end-to-end

🤖 Generated with Claude Code

Issue - None

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Summary by CodeRabbit

  • New Features

    • Exports now omit empty/None fields and produce minimal Helm override files.
    • New option to preserve shipped defaults in exports (--no-strip-defaults).
    • Masked credentials are replaced with TODO placeholders containing secretName/secretKey; export prints secret locations to stderr.
    • Exported workflow images no longer keep pinned backend tags and will follow the global image tag.
  • Bug Fixes

    • Improved handling of optional endpoints to avoid export failures.
  • Documentation

    • Post-export guidance updated to reference stderr and to require listing final secretNames in service config secret refs.

…lders

Turns the export script from a full config dump into a minimal override
file. Five behavior changes, all triggered by a real 6.2→6.3 migration
where the previous output was ~300 lines of mostly chart defaults with
literal '**********' in every credential field.

1. Masked secret values are rewritten into
   {secretName: TODO-REPLACE-ME, secretKey: <field>} placeholders and
   each path is listed on stderr. Previously the docs claimed this but
   the script emitted '**********' verbatim, which silently broke
   deployments that applied the YAML as-is.

2. None values and empty dicts/lists are dropped recursively. They're
   equivalent to Pydantic defaults on the service side and just add
   noise. `gpu: null`, `override_url: null`, and friends all go.

3. Pinned tags in workflow.backend_images.{init,client} get stripped
   with a stderr note. Keeping them silently locks workflow pods to
   whatever init/client container was running at export time — easy to
   miss during an upgrade.

4. HTTP 400 with 'relation does not exist' + error_code DATABASE on
   `group_template` (or any other endpoint in OPTIONAL_CONFIG_ENDPOINTS)
   is recognized as "table absent on this schema version" and logged as
   '(skipped …: table not present on this schema)' instead of dumping
   the raw SQL error. Looked like a script bug before.

5. Output is diffed against the chart's services.configs.* defaults.
   Scalars equal to the chart default are dropped; a named-dict entry
   (a role, a podTemplate, a backend, a pool, …) that exactly matches
   the shipped default is removed wholesale. --no-strip-defaults
   restores the full dump for auditing purposes.

Baseline comes from ../charts/service/values.yaml relative to the
script; overridable via --chart-values.

Also fixes collect_secret_names so it doesn't report
'TODO-REPLACE-ME' as an existing K8s Secret the operator needs to
create.

Docs: 6_2_to_6_3_upgrade.md's 'The script' bullet list is rewritten to
match the new behavior.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vvnpn-nv vvnpn-nv requested a review from a team as a code owner April 24, 2026 18:02
@coderabbitai

coderabbitai Bot commented Apr 24, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Export script and docs updated: exports now prune None/empty fields, strip workflow backend image tag pins, replace masked secrets with {secretName: TODO-REPLACE-ME, secretKey: <field>} placeholders (logged to stderr), and optionally diff against chart values.yaml to omit defaults; new CLI flags added.

Changes

Cohort / File(s) Summary
Documentation
deployments/upgrades/6_2_to_6_3_upgrade.md
Updates upgrade guide to document exporter behavior: dropping empty/None structures, unpinning workflow backend image tags, rewriting masked secrets to {secretName: TODO-REPLACE-ME, secretKey: <field>} placeholders while emitting secret paths to stderr, and describing default-value stripping via chart diff.
Export Script
deployments/upgrades/export_configs_to_helm.py
Adds recursive removal of None/empty containers, strips pinned tags from services.configs.workflow.backend_images.*, converts masked credential sentinels into secret placeholders and collects dotted paths for stderr guidance, optionally loads --chart-values and deep-diffs services.configs to remove fields matching chart defaults (opt-out via --no-strip-defaults), changes fetch() to accept optional=False and raise ExportError on required failures while treating certain 400 responses as non-fatal, and expands stderr output. New public API items: ExportError, fetch(..., optional=False), --chart-values, --no-strip-defaults, and _default_chart_values_path().

Sequence Diagram(s)

sequenceDiagram
  participant User as Client (CLI)
  participant Script as export_configs_to_helm.py
  participant API as Service API
  participant Chart as Chart values.yaml
  participant Out as stdout/stderr & files

  User->>Script: run exporter (--chart-values? --no-strip-defaults?)
  Script->>API: fetch named configs (fetch(..., optional=?))
  API-->>Script: config JSON / 400 (optional skip)
  Script->>Script: drop None/empty fields
  Script->>Script: unpin workflow backend image tags
  Script->>Script: rewrite masked secrets -> TODO placeholders (collect paths)
  Script->>Chart: load baseline values (if --chart-values provided)
  Chart-->>Script: chart defaults
  Script->>Script: diff & strip defaults (unless --no-strip-defaults)
  Script->>Out: write Helm values override (stdout/file)
  Script->>Out: emit stderr: secret TODO paths, secretRefs, notes on stripped defaults and image-pin removals
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 I hopped through configs, pruning None and stray pins,
Replaced hidden stars with TODOs and tiny grins.
I checked chart defaults, diffed what should stay,
Wrote notes on stderr to guide the way—
A neat export trail for engineers and their bins. 🥕

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the two main changes: converting to minimal-diff output and replacing masked secrets with placeholders.
Docstring Coverage ✅ Passed Docstring coverage is 86.67% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch vpan/export-configs-improvements

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@deployments/upgrades/6_2_to_6_3_upgrade.md`:
- Around line 54-58: The docs lack explicit instructions for turning per-field
placeholders {secretName: TODO-REPLACE-ME, secretKey: <field>} into real
Kubernetes Secrets; update the guide to tell operators to inspect the stderr
list of TODO placeholders and either (1) create a Kubernetes Secret named as you
choose with keys matching each listed secretKey (so each placeholder maps to a
key in that Secret) or (2) edit the generated YAML under services.configs to
replace the per-field placeholder block with a whole-secret reference
(secretRef) that your configmap_loader.py expects, and ensure the mounted Secret
filenames/keys match what configmap_loader.py checks before it clears the
placeholders; also call out that existing secretRefs listed in stderr must have
matching K8s Secrets in the target namespace.

In `@deployments/upgrades/export_configs_to_helm.py`:
- Around line 251-263: The list branch of drop_empty currently filters on the
original item instead of the recursively cleaned value, so nested empty
containers (e.g., [{'gpu': None}]) become [{}]; update the list handling in
drop_empty to call drop_empty(item) for each element, then include only elements
where the cleaned result is not None and not an empty dict/list (i.e., filter
out cleaned == None, cleaned == {}, and cleaned == []), ensuring recursive
pruning of empty containers inside lists while preserving other values.
- Around line 542-552: After strip_backend_image_pins(configs) may leave empty
dicts (e.g. workflow.backend_images -> {}), run drop_empty(configs) again before
diffing so empty containers are removed and diff_against_defaults(configs,
defaults) produces a minimal override; specifically, after calling
strip_backend_image_pins, call drop_empty on the configs (respecting the
args.no_strip_defaults flow and existing load_chart_defaults/
diff_against_defaults usage) so empty structures are not preserved by
diff_against_defaults.
- Around line 130-145: The exception handler inside the fetch routine (the
except urllib.error.HTTPError block) currently returns None for non-optional
failures which lets main() produce a partial export; modify it so that if the
call is not optional (the optional variable is False) and the HTTPError is not
the specific 400/"does not exist"/"\"DATABASE\"" case, the process exits
non-zero (e.g., sys.exit(1) or re-raise the error) instead of returning None;
keep the existing skip behavior for the optional true + 400/"does not
exist"/"\"DATABASE\"" case and preserve printing of the error body for
diagnostics.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9c7e9c2d-cbb3-4363-ba87-d0285815e710

📥 Commits

Reviewing files that changed from the base of the PR and between 001d490 and 585cb58.

📒 Files selected for processing (2)
  • deployments/upgrades/6_2_to_6_3_upgrade.md
  • deployments/upgrades/export_configs_to_helm.py

Comment thread deployments/upgrades/6_2_to_6_3_upgrade.md
Comment thread deployments/upgrades/export_configs_to_helm.py Outdated
Comment thread deployments/upgrades/export_configs_to_helm.py
Comment on lines +542 to +552
# 2. Strip pinned backend image tags — they silently lock workflow pods
# to whatever version was running at export time.
stripped_image_pins = strip_backend_image_pins(configs)

# 3. Diff against the chart's shipped defaults. Only fields that differ
# from the chart's out-of-the-box configs remain.
defaults_used = False
if not args.no_strip_defaults:
defaults = load_chart_defaults(args.chart_values)
if defaults:
configs = diff_against_defaults(configs, defaults)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Run drop_empty() again after stripping backend image pins.

If workflow.backend_images only contained init / client, strip_backend_image_pins() turns it into {} after the initial normalization pass. diff_against_defaults() then preserves that empty dict because the chart has no backend_images default, so the export stops being a minimal override.

🧽 Suggested fix
     # 2. Strip pinned backend image tags — they silently lock workflow pods
     # to whatever version was running at export time.
     stripped_image_pins = strip_backend_image_pins(configs)
+    configs = drop_empty(configs)
 
     # 3. Diff against the chart's shipped defaults. Only fields that differ
     # from the chart's out-of-the-box configs remain.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@deployments/upgrades/export_configs_to_helm.py` around lines 542 - 552, After
strip_backend_image_pins(configs) may leave empty dicts (e.g.
workflow.backend_images -> {}), run drop_empty(configs) again before diffing so
empty containers are removed and diff_against_defaults(configs, defaults)
produces a minimal override; specifically, after calling
strip_backend_image_pins, call drop_empty on the configs (respecting the
args.no_strip_defaults flow and existing load_chart_defaults/
diff_against_defaults usage) so empty structures are not preserved by
diff_against_defaults.

Addresses items from self-review of #900:

- _is_masked: tolerate server-side mask-length changes. Previously a
  strict string-equality check against '**********' (10 chars exactly);
  now accepts any pure-asterisk string >= MASKED_MIN_ASTERISKS long.
- rewrite_masked_secrets: handle masked scalars that appear as bare list
  items (previous code recursed into lists but only the dict branch did
  the rewrite — silent miss). Also fold the public function and its
  _inner helper into one — the split wasn't buying anything.
- Delete the misleading "we don't collapse credential dicts" comment
  that read like behavior description but was just a design note; moved
  the actual contract into the docstring.
- drop_empty: symmetrize dict vs. list handling via a shared _is_empty
  helper. Lists now drop items that clean down to None/{}/[] the same
  way dict values do.
- Extract TODO_PLACEHOLDER constant. The string appeared in the
  rewriter, the collect_secret_names skip list (via brittle
  `.startswith('TODO')` — would false-match any legitimate secretName
  starting with 'TODO'), and the stderr message.
- load_chart_defaults: drop copy.deepcopy (and the import). drop_empty
  already returns a new tree.

No behavior change beyond the masked-value length tolerance, list-scalar
rewrite fix, and stricter collect_secret_names filter. Verified
end-to-end against https://sqa.osmo.nvidia.com — same 811-line export as
before the cleanup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
deployments/upgrades/export_configs_to_helm.py (2)

565-575: ⚠️ Potential issue | 🟡 Minor

Run drop_empty() again after stripping backend image pins.

If workflow.backend_images only had init/client, stripping pins leaves backend_images: {} after the initial normalization pass. diff_against_defaults() will keep that empty dict when the chart has no matching default, so the export stops being a minimal override.

Suggested fix
     # 2. Strip pinned backend image tags — they silently lock workflow pods
     # to whatever version was running at export time.
     stripped_image_pins = strip_backend_image_pins(configs)
+    configs = drop_empty(configs)
 
     # 3. Diff against the chart's shipped defaults. Only fields that differ
     # from the chart's out-of-the-box configs remain.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@deployments/upgrades/export_configs_to_helm.py` around lines 565 - 575, After
stripping pinned backend image tags with strip_backend_image_pins(configs), run
the existing drop_empty() cleanup again to remove now-empty structures (e.g.,
backend_images: {}) before running diff_against_defaults; specifically, call
configs = drop_empty(configs) immediately after stripped_image_pins =
strip_backend_image_pins(configs) so diff_against_defaults only sees meaningful
keys.

134-152: ⚠️ Potential issue | 🟠 Major

Fail hard on required fetch errors instead of emitting a partial override.

This still returns None for non-optional 401/500/timeout failures, and main() just omits that section while always writing services.configs.enabled: true. A transient API failure can therefore produce a valid-looking but incomplete migration file.

Suggested fix
     except urllib.error.HTTPError as error:
         body = ''
         try:
             body = error.read().decode()
-        except Exception:  # pylint: disable=broad-except
+        except (OSError, UnicodeDecodeError):
             pass
         if (optional and error.code == 400
                 and 'does not exist' in body
                 and '"DATABASE"' in body):
             print(f'  (skipped {path}: table not present on this schema)',
                   file=sys.stderr)
             return None
-        print(f'Error fetching {path}: HTTP {error.code}', file=sys.stderr)
-        if body:
-            print(f'  {body[:200]}', file=sys.stderr)
-        return None
+        message = f'Error fetching {path}: HTTP {error.code}'
+        if body:
+            message = f'{message} - {body[:200]}'
+        raise RuntimeError(message) from error
     except urllib.error.URLError as error:
-        print(f'Error connecting to {url}: {error.reason}', file=sys.stderr)
-        return None
+        raise RuntimeError(
+            f'Error connecting to {url}: {error.reason}'
+        ) from error
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@deployments/upgrades/export_configs_to_helm.py` around lines 134 - 152, The
current exception handlers for urllib.error.HTTPError and urllib.error.URLError
swallow failures and return None even when the fetch was not optional, which
lets main() produce an incomplete migration file; update the handlers in the
fetch routine (the block that references optional, path and url) so that if
optional is False you either re-raise the exception or exit with a non-zero
status (rather than returning None), while preserving the current “skip”
behavior for optional==True and the existing debug prints for optional errors;
apply this change to both the HTTPError and URLError branches so main() can
detect hard failures instead of silently emitting a partial override.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@deployments/upgrades/export_configs_to_helm.py`:
- Around line 360-374: The code assumes the YAML root and nested nodes are
mappings and chains .get(), which can raise AttributeError for non-mapping YAML
(values_path, values); change the logic to validate shapes before chaining:
after loading values (variable values), check isinstance(values, dict) else
return {}; then assign services = values.get('services', {}) only if
isinstance(values.get('services', {}), dict) else return {}; then set configs =
services.get('configs', {}) only if isinstance(services.get('configs', {}),
dict) else return {}; keep the subsequent drop of the 'enabled' key and return
drop_empty(configs) so behavior falls back to an empty mapping for non-mapping
YAML rather than crashing.

---

Duplicate comments:
In `@deployments/upgrades/export_configs_to_helm.py`:
- Around line 565-575: After stripping pinned backend image tags with
strip_backend_image_pins(configs), run the existing drop_empty() cleanup again
to remove now-empty structures (e.g., backend_images: {}) before running
diff_against_defaults; specifically, call configs = drop_empty(configs)
immediately after stripped_image_pins = strip_backend_image_pins(configs) so
diff_against_defaults only sees meaningful keys.
- Around line 134-152: The current exception handlers for urllib.error.HTTPError
and urllib.error.URLError swallow failures and return None even when the fetch
was not optional, which lets main() produce an incomplete migration file; update
the handlers in the fetch routine (the block that references optional, path and
url) so that if optional is False you either re-raise the exception or exit with
a non-zero status (rather than returning None), while preserving the current
“skip” behavior for optional==True and the existing debug prints for optional
errors; apply this change to both the HTTPError and URLError branches so main()
can detect hard failures instead of silently emitting a partial override.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 74703af3-0897-4883-a29c-9a201e9cf599

📥 Commits

Reviewing files that changed from the base of the PR and between 585cb58 and b5adc79.

📒 Files selected for processing (1)
  • deployments/upgrades/export_configs_to_helm.py

Comment thread deployments/upgrades/export_configs_to_helm.py
Three behavioral fixes from PR review feedback:

1. fetch() fails fast on non-optional errors instead of returning None.
   Previously a 401 / 500 / timeout would log the error, return None,
   and main() would silently omit that section while still emitting
   `services.configs.enabled: true`. The operator could apply a
   syntactically valid but partial export. New ExportError class +
   top-level catch in __main__ exits non-zero with the failure context.
   Optional-endpoint behavior (group_template on pre-6.3 schemas)
   unchanged.

2. Re-run drop_empty() after strip_backend_image_pins() so a
   workflow.backend_images that only held init/client tags doesn't
   survive as `{}` past the diff stage. diff_against_defaults preserves
   it because the chart has no backend_images default, breaking the
   "minimal override" contract.

3. Guard YAML shape in load_chart_defaults. --chart-values can point at
   any YAML; if the root or services/ or services.configs/ isn't a
   mapping, the previous .get().get() chain raised AttributeError mid-
   export instead of falling back to "skip diffing" as the docstring
   promised. Walk explicitly with isinstance checks.

Plus a docs fix:

4. 6_2_to_6_3_upgrade.md now explains how to turn the new
   {secretName, secretKey} placeholders into real K8s Secrets — either
   per-field (--from-literal Secret with matching keys) or
   whole-credential (single Secret containing cred.yaml that the loader
   merges into the parent dict). Previously the doc just said "create
   matching K8s Secrets" without explaining the two layouts.

Verified end-to-end against sqa.osmo.nvidia.com:
- Happy path: 811-line export, identical to pre-change.
- Bad token: exits 1 with "Export aborted: …", no partial stdout.
- Junk --chart-values YAML: warns and falls back to no-strip behavior.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
deployments/upgrades/export_configs_to_helm.py (1)

381-389: ⚠️ Potential issue | 🟡 Minor

Warn when services.configs itself is not a mapping.

If --chart-values contains something like services: {configs: 123}, this falls back to {} silently, so default-diff is skipped without the warning the new flow otherwise emits.

Suggested fix
     for key in ('services', 'configs'):
         if not isinstance(configs, dict):
             print(f'Warning: unexpected structure at {values_path} '
                   f'(no services.configs mapping); skipping default-diff',
                   file=sys.stderr)
             return {}
         configs = configs.get(key, {})
     if not isinstance(configs, dict):
+        print(f'Warning: unexpected structure at {values_path} '
+              f'(no services.configs mapping); skipping default-diff',
+              file=sys.stderr)
         return {}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@deployments/upgrades/export_configs_to_helm.py` around lines 381 - 389, The
current loop over for key in ('services', 'configs') silences the case where
services.configs exists but is not a mapping (e.g., services: {configs: 123})
because the final isinstance check happens after the loop; change the logic to
validate immediately after fetching each key: after configs = configs.get(key,
{}), if key == 'configs' and not isinstance(configs, dict) then print the
existing warning (using values_path) and return {} so we emit a warning when
services.configs is not a mapping; keep using the same warning text and
sys.stderr output and preserve the surrounding loop and returns.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@deployments/upgrades/export_configs_to_helm.py`:
- Around line 141-158: The current code calls json.loads(resp.read()) and will
let JSONDecodeError escape; modify the success path to read the response into a
string (e.g. body = resp.read().decode()), then try to parse with
json.loads(body) and catch json.JSONDecodeError (or ValueError) and raise
ExportError with a message like f'Error fetching {path}: invalid JSON' and
include a snippet of body (first ~200 chars), preserving "raise ExportError(...
) from error" semantics; follow the same message/format used in the
urllib.error.HTTPError branch so callers see a consistent ExportError instead of
a traceback.

---

Duplicate comments:
In `@deployments/upgrades/export_configs_to_helm.py`:
- Around line 381-389: The current loop over for key in ('services', 'configs')
silences the case where services.configs exists but is not a mapping (e.g.,
services: {configs: 123}) because the final isinstance check happens after the
loop; change the logic to validate immediately after fetching each key: after
configs = configs.get(key, {}), if key == 'configs' and not isinstance(configs,
dict) then print the existing warning (using values_path) and return {} so we
emit a warning when services.configs is not a mapping; keep using the same
warning text and sys.stderr output and preserve the surrounding loop and
returns.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1d7b0e45-afc2-4cad-9ea6-843e4685b2e6

📥 Commits

Reviewing files that changed from the base of the PR and between b5adc79 and a285c32.

📒 Files selected for processing (2)
  • deployments/upgrades/6_2_to_6_3_upgrade.md
  • deployments/upgrades/export_configs_to_helm.py
✅ Files skipped from review due to trivial changes (1)
  • deployments/upgrades/6_2_to_6_3_upgrade.md

Comment on lines 141 to +158
with urllib.request.urlopen(req, timeout=30) as resp:
return json.loads(resp.read())
except urllib.error.HTTPError as error:
print(f'Error fetching {path}: HTTP {error.code}', file=sys.stderr)
body = ''
try:
body = error.read().decode()
print(f' {body[:200]}', file=sys.stderr)
except Exception as read_error:
print(f' (Could not read error body: {read_error})',
except OSError:
pass
if (optional and error.code == 400
and 'does not exist' in body
and '"DATABASE"' in body):
print(f' (skipped {path}: table not present on this schema)',
file=sys.stderr)
return None
return None
message = f'Error fetching {path}: HTTP {error.code}'
if body:
message = f'{message}\n {body[:200]}'
raise ExportError(message) from error

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Wrap JSON decode failures in ExportError.

A 200 response with HTML or truncated JSON will currently raise JSONDecodeError here and bypass the clean abort path at Lines 654-658, so operators get a traceback instead of the intended non-zero export failure.

Suggested fix
     try:
         with urllib.request.urlopen(req, timeout=30) as resp:
-            return json.loads(resp.read())
+            payload = resp.read()
+        try:
+            return json.loads(payload)
+        except json.JSONDecodeError as error:
+            preview = payload[:200].decode(errors='replace')
+            raise ExportError(
+                f'Error parsing {path} response as JSON:\n  {preview}'
+            ) from error
🧰 Tools
🪛 Ruff (0.15.12)

[error] 141-141: Audit URL open for permitted schemes. Allowing use of file: or custom schemes is often unexpected.

(S310)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@deployments/upgrades/export_configs_to_helm.py` around lines 141 - 158, The
current code calls json.loads(resp.read()) and will let JSONDecodeError escape;
modify the success path to read the response into a string (e.g. body =
resp.read().decode()), then try to parse with json.loads(body) and catch
json.JSONDecodeError (or ValueError) and raise ExportError with a message like
f'Error fetching {path}: invalid JSON' and include a snippet of body (first ~200
chars), preserving "raise ExportError(... ) from error" semantics; follow the
same message/format used in the urllib.error.HTTPError branch so callers see a
consistent ExportError instead of a traceback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant