Skip to main content
← Back to list
01Issue
FeatureOpenSwamp CLITeam
AssigneesNone

Relationships

#1641 Quality rubric + push gate for version upgrade chain correctness

Opened by webframp · 8/13/2026

Problem

Agents authoring extensions reliably miss the version upgrade discipline. The guidance exists in references/extension/references/model/upgrades.md but the loading path is too deep for agents to discover organically during extension development. Nothing in the observable feedback loop (quality score, push checks, type-check) surfaces the requirement.

The result: extensions get published with bumped version fields but no corresponding upgrades entry, leaving existing model instances stranded at their old typeVersion forever. Downstream, those instances silently skip future upgrade chains that assume the intermediate version was applied.

Current state

  • The upgrades doc clearly specifies the rules (last toVersion must equal version, chain must be chronologically ordered, no-op upgrades required even for non-schema-changing bumps).
  • The registry validates upgrade chain consistency at startup (runtime).
  • swamp extension push gates on adversarial review presence and quality score.
  • swamp extension quality evaluates 12 factors — none related to upgrade chain correctness.
  • Agents learn through the score/push feedback loop. If the loop doesn't surface the problem, agents don't learn about it.

Proposal

1. New quality rubric factor: upgrade-chain-valid (1 pt)

Conditions to earn:

  • If version equals the currentPublished version (or no published version exists): auto-pass (no upgrade needed for first publish or unchanged version).
  • If version differs from currentPublished: an upgrades array entry with toVersion == version must exist.
  • The upgrade chain must be chronologically ordered.
  • The last entry's toVersion must equal the model's version field.

This teaches agents through the normal workflow: run quality → see failure → read guidance → fix. The failure message should reference upgrades.md directly so agents can self-correct.

2. Push pre-flight warning (or block)

When swamp extension push detects a version bump without a corresponding upgrade entry:

  • Emit a warning: "Version bumped from {old} to {new} but no upgrades entry with toVersion={new} found. Existing instances will not auto-migrate. Add an upgrades entry (even a no-op) or pass --skip-upgrade-check to acknowledge."
  • Optionally block (exit 2) unless --skip-upgrade-check is passed, similar to how missing adversarial reviews prompt for confirmation.

3. Skill routing improvement

The extension guide's model-specific section references references/model/upgrades.md but only in a bullet list that agents rarely drill into. Consider:

  • Adding a brief inline callout in the extension guide's "Shared Development Workflow" section (step between "Version + manifest" and "Quality check"): "If bumping version from a published baseline, add an upgrades entry — see upgrades.md."
  • Having the quality factor's failure message include the exact file path to read.

Impact

  • Agents in the feedback loop learn the pattern after one failed quality check.
  • Extension authors (human or agent) get a guardrail before publishing broken upgrade chains.
  • Existing instances stop silently falling behind on typeVersion.

Context

Discovered while building a new extension (@webframp/aws/s3-inventory-observation) — first publish, so no upgrade was needed, but the question "did we handle version bumps correctly?" had no observable answer in the toolchain. An agent working on a subsequent version bump would have no signal that an upgrade entry is required until the registry throws at runtime on someone else's instance.

02Bog Flow
OPENTRIAGEDIN PROGRESSSHIPPED

Open

8/13/2026, 7:40:01 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.