Releases: systeminit/swamp
swamp 20260317.225941.0-sha.96bd3573
What's Changed
- fix: guard against path traversal in extension archive extraction and add driver/datastore tests (#746)
Summary
Addresses two issues raised in PR review:
-
Path traversal during archive extraction (
src/cli/commands/extension_pull.ts): Aftertarextracts an extension archive to a temp directory, each extracted file path is now resolved and validated to confirm it starts within the temp directory. If any entry contains path traversal sequences (e.g.../../.bashrc) that would escape the temp dir, aUserErroris thrown before any further processing occurs. -
Missing test coverage for driver/datastore content extraction (
src/domain/extensions/extension_content_extractor_test.ts): Added 8 unit tests forextractDriverFromSourceandextractDatastoreFromSourcefollowing the same patterns as the existing vault tests:- Extracts type, name, and description
- Extracts
configSchemafields (inlinez.object) - Skips files without the relevant export (
driver/datastore) - Skips exports that are missing the required
typefield
Test Plan
- All 3183 existing tests continue to pass (
deno run test) - 8 new unit tests added and passing for driver/datastore extraction
-
deno checkpasses (no type errors) -
deno lintpasses (no lint errors) -
deno fmt --checkpasses (no formatting issues)
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.225941.0-sha.96bd3573/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.225941.0-sha.96bd3573/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.225941.0-sha.96bd3573/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.225941.0-sha.96bd3573/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/swamp 20260317.223719.0-sha.9003347e
What's Changed
- fix: create models dir before writing upstream_extensions.json lock (#747)
Summary
Fixes #734.
When auto-resolving a vault-only extension (e.g. @swamp/1password), the installer fails if the extensions/models/ directory does not exist:
No such file or directory (os error 2): open '/private/tmp/swamp-vault-test/extensions/models/upstream_extensions.json.lock'
The root cause: installExtension always calls updateUpstreamExtensions (which creates upstream_extensions.json.lock inside modelsDir) regardless of whether the extension contains any models. Every other destination directory — vaults, workflows, drivers, datastores, bundles — was already guarded with Deno.mkdir({ recursive: true }) before use, but modelsDir was not.
The fix adds await Deno.mkdir(absoluteModelsDir, { recursive: true }) before the models copyDir call, making it consistent with all other directories.
Steps to Reproduce (from issue)
swamp repo initin a fresh directory (noextensions/models/exists)- Create a vault config referencing
@swamp/1password - Run a vault command that triggers auto-resolution (e.g.
swamp vault list-keys <name>) - Extension is found and downloaded, but installation fails
Workaround: mkdir -p extensions/models/ before triggering auto-resolution.
Test Plan
- Existing
updateUpstreamExtensionsunit tests pass deno check,deno lint,deno fmt --checkall pass
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.223719.0-sha.9003347e/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.223719.0-sha.9003347e/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.223719.0-sha.9003347e/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.223719.0-sha.9003347e/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/swamp 20260317.221248.0-sha.57baec4a
What's Changed
- perf: defer self-contained bundle creation to first out-of-process execution (#743)
Problem
With 104+ user extension models, swamp model type search (and any other command that loads extensions) took ~46 seconds to start. After the fix in #741 that parallelised the 4 loader types and skipped loading for help/version commands, the remaining bottleneck was in UserModelLoader itself.
At startup, loadModels() eagerly built a self-contained bundle for every model file — a separate deno bundle subprocess per model that inlines all dependencies (including zod) so the bundle can run inside Docker containers without network access. With 104 models, this was 104 sequential subprocesses on every single CLI invocation, regardless of whether Docker execution would ever be used.
Architecture Decision & Tradeoffs
What changed
bundleSource?: string on ModelDefinition (a pre-built JS string stored at load time) has been replaced with bundleSourceFactory?: () => Promise<string> — a memoizing closure that defers the expensive work to the point of actual need.
// Before: runs at startup, for every model, every invocation
modelDef.bundleSource = await bundleExtension(absolutePath, denoPath, { selfContained: true });
// After: closure set at load time, executed only on first Docker execution
let cachedBundle: string | undefined;
modelDef.bundleSourceFactory = async () => {
if (!cachedBundle) {
cachedBundle = await bundleExtension(absolutePath, denoPath, { selfContained: true });
}
return cachedBundle;
};Tradeoff: startup cost vs. first-execution cost
| Before | After | |
|---|---|---|
| Every CLI invocation | Pays bundling cost for all N models | Pays nothing |
| First Docker execution of model A | Already paid at startup | Pays bundling cost for model A only |
| Second Docker execution of model A | Pre-built | Memoized in-process — instant |
The first out-of-process execution of a given model will be slightly slower than before — it now bundles on demand rather than having it pre-built. This is the right tradeoff because:
- Docker execution is rare relative to everyday CLI usage (
type search,model get,data list, etc.) - Cost is proportional to actual need — only the models you actually run out-of-process are ever bundled
- Memoization ensures the cost is paid at most once per model per process invocation
- The startup tax was paid unconditionally regardless of what command you ran — even read-only commands that never touch Docker
Why only models?
Vaults, drivers, and datastores do not create self-contained bundles at all — they only use the externalized (cached) bundle for in-process execution. This change is model-specific because only models support out-of-process/Docker execution via bundleSource.
User Impact
Measured on a real repo with 104 extension models:
| Command | Before | After |
|---|---|---|
swamp model type search aws |
~46 seconds | ~2.8 seconds |
swamp model get my-model |
~46 seconds | ~2.8 seconds |
swamp data list |
~46 seconds | ~2.8 seconds |
First Docker model run |
Instant (pre-built) | ~same as before (built on demand) |
16x speedup for the everyday command path. The remaining ~2.8s is the warm-cache cost of the externalized bundle loading (disk reads + dynamic imports for 104 files), which is a separate optimization opportunity.
Files Changed
| File | Change |
|---|---|
src/domain/models/model.ts |
bundleSource?: string → bundleSourceFactory?: () => Promise<string> |
src/domain/models/user_model_loader.ts |
Remove eager bundling, set memoizing factory closure |
src/domain/models/method_execution_service.ts |
Await bundleSourceFactory?.() at execution time |
Test Plan
-
deno check— type checking passes -
deno lint— no lint errors -
deno fmt --check— formatting correct -
deno run test— 3175 tests passed, 0 failed -
deno run compile— binary compiled successfully - Manual:
swamp model type searchin 104-model repo — 46s → 2.8s
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.221248.0-sha.57baec4a/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.221248.0-sha.57baec4a/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.221248.0-sha.57baec4a/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.221248.0-sha.57baec4a/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/swamp 20260317.221128.0-sha.4ff54080
What's Changed
- feat: add driver and datastore support to extension push/pull (#745)
Summary
- Extend the extension system to support packaging and distributing drivers and datastores alongside models, workflows, and vaults
- Extensions can now be driver-only or datastore-only — no longer require models or workflows
- This is PR 2 of 4 in the Extension Drivers & Datastores series (builds on #735)
What changed
Manifest schema
- New optional
driversanddatastoresarray fields - Validation accepts at least one of: models, workflows, vaults, drivers, or datastores
Push (extension push)
- Resolves driver/datastore files from
extensions/drivers/andextensions/datastores/with transitive import resolution - Bundles each entry point to standalone JS
- Adds
drivers/,driver-bundles/,datastores/,datastore-bundles/to archive - Runs safety analysis and quality checks on all TypeScript files
- Validates collective naming for driver/datastore types
- Extracts content metadata (type, name, description, configSchema fields)
Pull (extension pull)
- Extracts and installs driver/datastore files to correct directories
- Conflict detection for driver/datastore paths and bundle paths
- Safety analysis on driver/datastore TypeScript files
- Tracks all files in
upstream_extensions.json
Content extraction
ExtractedDriverandExtractedDatastoretypes- Detects
export const driver+createDriverandexport const datastore+createProviderpatterns - Config schema field extraction for both
Updated commands
extension search,extension update,extension fmt, and auto-resolver all passdriversDir/datastoresDirthrough install contexts
Documentation
design/extension.md— updated archive structure, file extraction table, manifest fieldsswamp-extension-modelskill — updated publishing reference (manifest schema, field reference, content mapping, push workflow, error messages)swamp-reposkill — updated repository structure and.swamp.yamlconfig reference
User-facing behavior
Users can create and distribute driver-only or datastore-only extensions:
manifestVersion: 1
name: "@myorg/custom-driver"
version: "2026.03.17.1"
drivers:
- my_driver.tsswamp extension push bundles and uploads. swamp extension pull installs to the correct directories. The server accepts the new archive directories without changes (they pass through opaquely).
Test plan
-
deno check— type checking passes -
deno lint— no lint errors (649 files) -
deno fmt— formatting clean (702 files) -
deno run test— 3175 passed, 0 failed -
deno run compile— binary compiles
🤖 Generated with Claude Code
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.221128.0-sha.4ff54080/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.221128.0-sha.4ff54080/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.221128.0-sha.4ff54080/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.221128.0-sha.4ff54080/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/swamp 20260317.220019.0-sha.62783440
What's Changed
- fix: improve vault create UX for deprecated types and optional config (#744)
Summary
- Deprecated type hint: When a user passes a renamed vault type (e.g.
aws-sm,azure-kv), the error now says "The type 'aws-sm' has been renamed to '@swamp/aws-sm'. Use: swamp vault create @swamp/aws-sm " instead of the generic "Unknown vault type" message. Achieved by exportingRENAMED_VAULT_TYPESfromvault_service.tsand checking it in thevault_createerror path. - Optional
--configfor extension vaults: Extension vault types no longer require--configto be passed. Omitting it defaults to{}, which is then validated againstconfigSchemaif one is defined. Users with config-free extension vaults no longer need to pass--config '{}'.
Test Plan
-
deno fmt --checkpasses -
deno lintpasses -
deno task testpasses (3175 tests, 0 failed) -
swamp vault create aws-sm my-vaultgives rename hint instead of generic error -
swamp vault create @swamp/aws-sm my-vault(without--config) defaults to{}
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.220019.0-sha.62783440/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.220019.0-sha.62783440/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.220019.0-sha.62783440/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.220019.0-sha.62783440/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/swamp 20260317.212534.0-sha.5ac636c1
What's Changed
- fix: use pre-built bundle when dependency freshness check fails (#742)
Closes #737
Summary
When an extension is pulled from the registry, swamp discards the valid pre-built bundle and fails with Module not found if any local dependency file is missing from disk. This fix makes bundleWithCache fall back to the cached bundle when the freshness check fails, rather than attempting a re-bundle that will also fail.
Root cause
bundleWithCache validates cache freshness by resolving all local imports from the .ts source file and comparing mtimes. If a dependency file is missing (because the extension was pushed with an older swamp that had a single-line import regex and missed multi-line import declarations), Deno.stat() throws inside the try block. The catch block then falls through to re-bundle from source — which fails with the same Module not found error.
The @keeb/grafana extension hits this exactly: grafana_instance.ts has a multi-line import for ./lib/grafana.ts. The older push regex only matched single-line imports, so lib/grafana.ts was never included in the archive. The pre-built bundle at .swamp/bundles/grafana_instance.js is perfectly valid (compiled at push time with all deps), but every load attempt discards it and fails.
The catch block comment said "Bundle doesn't exist, stat failed, or import resolution failed — rebundle", conflating two distinct cases:
- Bundle file does not exist → rebundle from source ✅ correct
- Bundle exists but freshness check threw → rebundle from source ❌ wrong — the bundle is valid
Fix
Track bundleExists before entering the try/catch. If the bundle file exists but the freshness check throws for any reason, use the cached bundle as a fallback and log at debug level. Only attempt a re-bundle when the bundle genuinely doesn't exist.
Applied to all four loaders: model, vault, driver, datastore.
User impact
Before: swamp extension install @keeb/grafana succeeds but swamp model type search grafana fails with deno bundle failed ... Module not found "lib/grafana.ts".
After: The pre-built bundle is used as a fallback. @keeb/grafana/instance appears in search results with no errors.
Verification
-
deno check— passes -
deno lint— passes -
deno fmt— passes -
deno run test— 3161 passed, 0 failed -
deno run compile— binary compiles - Manual:
swamp extension install @keeb/grafana→swamp model type search grafanareturns@keeb/grafana/instance
🤖 Generated with Claude Code
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.212534.0-sha.5ac636c1/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.212534.0-sha.5ac636c1/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.212534.0-sha.5ac636c1/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.212534.0-sha.5ac636c1/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/swamp 20260317.212014.0-sha.ede9f2a1
What's Changed
Problem
Every CLI invocation — including swamp --help — was unconditionally loading all user extensions before running any command. This caused a ~42 second startup time, making even basic usage feel broken.
The root cause was four sequential loader calls, each independently:
- Creating a
RepoMarkerRepositoryand reading the marker file from disk - Creating an
EmbeddedDenoRuntimeand callingensureDeno() - Spawning
deno bundlesubprocesses to compile extension files
With 4 sequential loaders and potentially many extension files, startup time was dominated by this overhead even when no extensions were needed.
Changes
1. Skip extension loading for commands that don't need it
Added commandNeedsExtensions() which checks the pre-parsed command against a set of commands that never use user extensions:
"", "help", "version", "completions", "init", "update", "auth", "telemetry", "issue"
swamp --help, swamp version, swamp completions bash, etc. are now instant.
2. Read marker and runtime once, share across all loaders
Previously each of the 4 loadUser* functions independently read the repo marker and created an EmbeddedDenoRuntime. Now the marker is read once (reusing the value already needed for resolveLogLevel) and a single EmbeddedDenoRuntime instance is shared — eliminating 3 redundant disk reads and 3 redundant runtime instantiations.
3. Run all 4 loaders in parallel
Replaced sequential await calls with Promise.all():
await Promise.all([
loadUserModels(repoDir, marker, denoRuntime),
loadUserVaults(repoDir, marker, denoRuntime),
loadUserDrivers(repoDir, marker, denoRuntime),
loadUserDatastores(repoDir, marker, denoRuntime),
]);This is safe because each loader writes to its own registry, reads from a separate source directory, and writes to a separate bundle cache directory. ensureDeno() is idempotent (checks a version marker file before extracting). JavaScript's single-threaded event loop means Map.set() calls within registries are atomic within a tick.
User Impact
| Command | Before | After |
|---|---|---|
swamp --help |
~42 seconds | < 1 second |
swamp version |
~42 seconds | < 1 second |
swamp completions bash |
~42 seconds | < 1 second |
swamp model type search aws |
~42 seconds | significantly faster (parallel loading) |
Commands that need extensions (e.g. model, workflow, data, vault) still load all extensions, but now in parallel rather than sequentially.
Files Changed
src/cli/mod.ts— all changes in this single file
Test Plan
-
deno check— type checking passes -
deno lint— no lint errors -
deno fmt --check— formatting correct -
deno run test— 3156 tests passed, 0 failed -
deno run compile— binary compiled successfully
Fixes #738
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.212014.0-sha.ede9f2a1/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.212014.0-sha.ede9f2a1/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.212014.0-sha.ede9f2a1/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.212014.0-sha.ede9f2a1/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/swamp 20260317.201339.0-sha.bad69941
What's Changed
- fix: resolve CJS/ESM interop failure loading Azure Key Vault extension bundle (#740)
Closes #733
Summary
- Fix
@swamp/azure-kvextension vault bundle failing to load in the compiled binary due to a CJS/ESM interop issue in the esbuild-generated__toESMhelper - Fix silent error swallowing in all four extension loaders that hid the real error and dumped ~2.1MB of base64-encoded bundle content to the terminal
- Apply the same fix to all four loaders (model, vault, driver, datastore) for consistency
Root cause
When deno bundle (esbuild) bundles npm packages with --platform deno, it generates a __toESM helper that conditionally sets a .default property on CJS module wrappers. The condition checks an isNodeMode flag that is false for Deno platform builds. CJS modules that set module.exports.__esModule = true (like tslib) do NOT get a .default property on their ESM wrapper.
The Azure SDK depends on tslib, and the bundled code destructures import_tslib.default to get __extends, __awaiter, etc. Since .default was never set, the destructuring gets undefined, crashing with:
Cannot destructure property '__extends' of 'import_tslib.default' as it is undefined
This real error was silently swallowed by a bare catch {} in the file URL import path. The data URL fallback also failed (because createRequire(import.meta.url) doesn't accept data URLs), and THAT error message included the entire 1.6MB base64-encoded bundle — dumping ~2.1MB of "encrypted looking" data to the terminal.
Fix (3 parts)
-
fixCjsEsmInterop()— Post-processes bundles to patch the__toESMhelper so it always sets.defaulton CJS module wrappers, matching--platform nodebehavior. Applied at both bundle time and import time (auto-fixes old cached bundles on disk). -
Log file URL import errors — Silent
catch {}blocks now log the actual error at debug level, making future bundle issues diagnosable withSWAMP_DEBUG=1. -
sanitizeDataUrlError()— Truncates base64 data URLs in error messages to prevent flooding the terminal.
Why this is the correct fix
- The
__toESMchange is equivalent to what esbuild generates with--platform node. Since Deno has full Node compat, always setting.defaultis correct. - The fix targets a specific esbuild-generated pattern via regex (
isNodeMode || !mod || !mod.__esModule ? __defProp(...) : target), so it only affects the exact helper function and is idempotent. - Old cached bundles are automatically fixed on first load and written back to disk — no manual cache clearing needed.
- Extensions without tslib (e.g.,
@swamp/aws/ec2with 104 model types) are completely unaffected — the regex simply doesn't match. - No extension re-upload required — the fix is in swamp's runtime, not the extension.
User impact
Before: swamp extension install @swamp/azure-kv succeeds but every subsequent command dumps ~2.1MB of base64 to terminal with a confusing filename must be a file URL error.
After: The extension loads cleanly and appears in swamp vault type list.
Verification
-
deno check— passes -
deno lint— passes -
deno fmt— passes -
deno run test— 3166 passed, 0 failed (5 new tests forfixCjsEsmInteropandsanitizeDataUrlError) -
deno run compile— binary compiles - Manual:
swamp repo init→swamp extension install @swamp/azure-kv→swamp vault type listshows@swamp/azure-kvwith no errors - Manual:
swamp extension install @swamp/aws/ec2→ all 104 model types load with zero warnings
🤖 Generated with Claude Code
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.201339.0-sha.bad69941/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.201339.0-sha.bad69941/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.201339.0-sha.bad69941/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.201339.0-sha.bad69941/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/swamp 20260317.193300.0-sha.d7a919c1
What's Changed
- feat: move AWS, Azure, and 1Password vault providers to extensions (#736)
Closes #665
Summary
Moves the aws-sm, azure-kv, and 1password vault providers from built-in types to extension vaults published at swamp.club. After this change, only local_encryption (and mock for testing) remain as built-in vault types. The three cloud/external vault providers are now independently versioned extensions that auto-resolve from the registry on first use.
What changed
Removed from core:
- Deleted
aws_vault_provider.ts,azure_kv_vault_provider.ts,onepassword_vault_provider.tsand their test files (-1,545 lines) - Removed
aws-sm,azure-kv,1passwordfromBUILT_IN_VAULT_TYPESinvault_types.ts— onlylocal_encryptionremains - Removed their switch cases from
VaultService.registerVault() - Removed
@aws-sdk/client-secrets-manager,@azure/identity,@azure/keyvault-secretsfromdeno.jsondependencies
Migration path via RENAMED_VAULT_TYPES:
aws/aws-sm→@swamp/aws-smazure/azure-kv→@swamp/azure-kv1password→@swamp/1password
When VaultService.fromRepository() loads an existing vault config with an old type name, it remaps to the @swamp/* extension type and auto-resolves it from the registry (installed by PR #725's auto-resolution infrastructure).
vault create simplified:
- Removed
--region,--vault-url,--op-vault,--op-accountflags - All extension vault types now use
--config <json>for provider configuration resolveProviderConfig()only handleslocal_encryptionnow
ensureDefaultVaults() is now a no-op:
- Previously auto-created an AWS vault when
AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY, andAWS_REGIONwere set - This behavior is removed since AWS is now an extension
Error messages updated:
- "No vaults configured" error now suggests
swamp extension pull @swamp/aws-sminstead of setting AWS env vars
Published extensions
The three vault providers have been published to swamp.club as:
@swamp/1password@2026.03.17.1— shells out toopCLI, no npm SDK deps@swamp/aws-sm@2026.03.17.1— uses@aws-sdk/client-secrets-manager@3.1010.0@swamp/azure-kv@2026.03.17.1— uses@azure/identity@4.13.0+@azure/keyvault-secrets@4.10.0
Source lives at https://github.com/systeminit/swamp-extensions
User impact
Existing users with vault configs on disk
No action required. Existing .swamp/vault/*.yaml files with type: aws-sm, type: azure-kv, or type: 1password continue to work. On first use, swamp will:
- Log a deprecation warning about the old type name
- Remap it to the
@swamp/*extension type - Auto-resolve and install the extension from the registry
- Load the vault and proceed normally
Creating new vaults
The CLI syntax changes from dedicated flags to --config <json>:
# Before
swamp vault create aws-sm my-vault --region us-east-1
swamp vault create azure-kv my-vault --vault-url https://myvault.vault.azure.net/
swamp vault create 1password my-vault --op-vault Engineering
# After
swamp vault create @swamp/aws-sm my-vault --config '{"region":"us-east-1"}'
swamp vault create @swamp/azure-kv my-vault --config '{"vault_url":"https://myvault.vault.azure.net/"}'
swamp vault create @swamp/1password my-vault --config '{"op_vault":"Engineering"}'Offline users
Users without registry access can manually install extensions by placing the .ts source files in extensions/vaults/.
Binary size
The compiled binary no longer includes the AWS SDK, Azure SDK, or 1Password provider code. These dependencies are now bundled into the extensions at publish time.
Known issues
- Azure Key Vault extension bundle fails to load in compiled binary due to large bundle size (#733)
- Auto-resolver fails when
extensions/models/directory doesn't exist for vault-only extensions (#734)
Verification
deno check— passesdeno lint— passesdeno fmt— passesdeno run test— 3138 passed, 0 faileddeno run compile— binary compiled successfully- Manual testing: auto-resolution verified for all three extensions (1password fails at
opCLI check, aws-sm fails at credential check — both expected)
🤖 Generated with Claude Code
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.193300.0-sha.d7a919c1/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.193300.0-sha.d7a919c1/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.193300.0-sha.d7a919c1/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.193300.0-sha.d7a919c1/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/swamp 20260317.192830.0-sha.8c163926
What's Changed
- feat: add datastore type registry and user datastore loader (#735)
Summary
- Add extension infrastructure for user-defined datastores, mirroring the existing patterns for drivers, models, and vaults
- Users can place TypeScript files in
extensions/datastores/that export adatastoreobject with a type, name, description, andcreateProviderfactory — they are discovered, bundled, validated, and registered on CLI startup - This is the first step toward allowing custom datastores to be packaged and distributed via the swamp extension system (PR 1 of 4)
What this PR does
New domain types
DatastoreSyncServiceinterface — pull/push contract for remote syncDatastoreProviderinterface — factory for locks, verifiers, and syncDatastoreTypeRegistryclass + singleton — register/lookup datastore types- Built-in type registration for
filesystemands3(viadatastore_types.ts)
Loader infrastructure
UserDatastoreLoader— discovers.tsfiles inextensions/datastores/, bundles with mtime-based caching to.swamp/datastore-bundles/, validates theexport const datastoreshape via Zod, and registers with the global registryresolveDatastoresDir— resolution priority:SWAMP_DATASTORES_DIRenv var >.swamp.yamldatastoresDir> defaultextensions/datastores
Modified files
DatastoreHealthResult.datastoreTypewidened from"filesystem" | "s3"tostringso custom datastores can report their own typeSWAMP_SUBDIRSgainsdatastoreBundles: "datastore-bundles"for bundle cacheRepoMarkerDatagains optionaldatastoresDirfield- CLI startup calls
loadUserDatastores()alongside existing model/vault/driver loaders - DDD ratchet count bumped 17→18 for the new domain→infrastructure import (same pattern as
UserDriverLoader)
User-facing behavior
No user-visible changes. This PR is purely additive infrastructure. Existing CLI commands, config files, and workflows behave identically. The loader silently no-ops when extensions/datastores/ does not exist (which is the case for all current repos). Users who place a valid datastore extension file in that directory will see it loaded at startup (visible with SWAMP_DEBUG=1).
Why this is correct
- Follows the exact same architecture as
UserDriverLoaderandUserVaultLoader— same bundling, caching, validation, and error handling patterns - All 3161 tests pass (3143 existing + 18 new)
deno check,deno lint,deno fmtall clean- Binary compiles successfully
- Manually verified end-to-end: init repo → place extension → confirm
Loaded user datastore type from my_store.tsat startup → bundle cached at.swamp/datastore-bundles/
What comes next
This is PR 1 of 4 in the Extension Drivers & Datastores series:
- This PR — Datastore registry & loader infrastructure
- Extension manifest + push/pull support for drivers & datastores
- Wire custom datastores into repo context (datastore commands)
- Skills & documentation
Test plan
-
deno check— type checking passes -
deno lint— no lint errors -
deno fmt— formatting clean -
deno run test— 3161 passed, 0 failed -
deno run compile— binary compiles - Manual:
swamp init→ place.tsinextensions/datastores/→SWAMP_DEBUG=1 swamp model type searchshowsLoaded user datastore type from my_store.ts
🤖 Generated with Claude Code
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
Installation
macOS (Apple Silicon):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.192830.0-sha.8c163926/swamp-darwin-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/macOS (Intel):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.192830.0-sha.8c163926/swamp-darwin-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (x86_64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.192830.0-sha.8c163926/swamp-linux-x86_64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/Linux (aarch64):
curl -L https://github.com/systeminit/swamp/releases/download/v20260317.192830.0-sha.8c163926/swamp-linux-aarch64 -o swamp
chmod +x swamp && sudo mv swamp /usr/local/bin/