Repository navigation
Migrate to manifest-based capabilities (Closes CloudTAK#165) - #9
Merged
Merged
Conversation
- Add capabilities.json (feature:submit only, 1024 MB / 120 s, rate(2 minutes) schedule) - Build and push with docker buildx in the demo and production deploy jobs, embedding the manifest as the com.cloudtak.capabilities OCI annotation - Keep the existing EcrEtlTasksRepoArn CloudFormation export lookup - Not adopting the cloudtak-etl CLI: it hardcodes a different ECR repo name Refs TAK-NZ/CloudTAK#165
- Add tsx and a real test suite (node:test via tsx) covering static config, Env and output schemas, and capabilities.json validation - Replace the no-op test script and lint the test directory - Use Task.init() for the local and Lambda handler entry points - Bump Node to 24 in etl-deploy.yml and add engines >= 24 - Document the buildx OCI annotation and npm test in the README Refs TAK-NZ/CloudTAK#165
- npm update within existing ranges: @tak-ps/etl 10.22.2, eslint 10.12.0, typescript-eslint 8.71.1, tsx 4.23.15; npm audit reports 0 vulnerabilities - typescript stays on 6.0.3: typescript-eslint still caps typescript at <6.1.0 - task.ts: make EphemeralState a type alias instead of an interface so it satisfies the Record<string, unknown> parameter of setEphemeral() in @tak-ps/etl 10.22 (type-only change, no runtime difference) - Test: check manifest permissions against the exported PERMISSIONS map, since @tak-ps/etl no longer exports isValidPermission from its entry point Refs TAK-NZ/CloudTAK#165
- Bump version to 1.1.0 - Document the capabilities manifest, OCI annotation build, tests, Node 24, dependency updates and the EphemeralState type alias - Record the deliberate decision not to adopt the cloudtak-etl CLI (it hardcodes a different ECR repository name than base-infra's <stackname>-etltasks) Closes TAK-NZ/CloudTAK#165
Excludes .git, .github, node_modules, dist, test, docs, .agents, .env* and markdown files. capabilities.json, task.ts, package*.json and tsconfig.json (everything the Dockerfile build needs) remain in the context; checked with a COPY . listing of the context. Refs TAK-NZ/CloudTAK#165
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates etl-floodhub to CloudTAK's manifest-based capabilities model (v1.1.0). The ETL now ships a
capabilities.jsonmanifest that is embedded in the container image as thecom.cloudtak.capabilitiesOCI annotation, so CloudTAK can read the task's compute, permissions and invocations without running it.Closes TAK-NZ/CloudTAK#165
Issue: TAK-NZ/CloudTAK#165
Changes per issue task
Google Flood Hub, memory 1024, timeout 120, required permissionfeature:submit, incomingscheduleinvocation defaulting torate(2 minutes)). The only permission declared isfeature:submit, because task.ts only callsthis.submit(fc)plus layer-scoped env/ephemeral helpers, andPERMISSIONSin@tak-ps/etl10.22.2 hasfeature: ['submit'].@tak-ps/etlrange^10.13.0(installed 10.22.2), npm audit advisories resolved,EphemeralStatetype alias adjusted.typescriptstays at 6.0.3 because thetypescript-eslintpeer is<6.1.0.etl-deploy.ymlusedocker/setup-buildx-action@v3and a singledocker buildx buildwith--annotation "com.cloudtak.capabilities=$(jq -c . capabilities.json)".tsx --test; switched toTask.init(); Node 24 inlint.yml,etl-deploy.ymlandengines.node >= 24..dockerignoretrimmed (adds.agents/) to reduce the docker build context.capabilities.json,task.ts,package*.jsonandtsconfig.jsonremain in the context, and the real Dockerfile built successfully.Deliberate deviation:
cloudtak-etlCLI not usedThe issue asked to use the
cloudtak-etlCLI for building and pushing. We deliberately did not. Itsbin/build.tshardcodes the ECR repositorytak-vpc-<Environment>-cloudtak-tasks. TAK-NZ base-infra names the repository<stackname>-etltasksand publishes it through theEcrEtlTasksRepoArnCloudFormation export. Using the CLI would push to a repository that does not exist in our environments, or force changes to base-infra naming.Instead we use
docker buildx buildwith thecom.cloudtak.capabilitiesOCI annotation and keep the existing CFN-export ECR lookup. The result is the same annotation contract CloudTAK reads, without changing our infrastructure naming.Verification
Run in a clean worktree with
npm ci(all commits individually build green; final HEAD re-checked: lint=0, build=0, tests 5/5):capabilities.json validation (checklist section 1 snippet): exit 0, name
Google Flood Hub, permissionsfeature:submit.{ "version": "1.0", "name": "Google Flood Hub", "description": "Ingests river gauge flood status and forecasts, flash flood events and significant flood events from the Google Flood Forecasting API for a configured region and posts them to the map as colour-coded gauge points and impact polygons.", "compute": { "memory": 1024, "timeout": 120 }, "permissions": [ { "resource": "feature:submit", "required": true, "description": "Submits river gauge points and flash flood / significant event polygons to CloudTAK" } ], "invocations": { "incoming": { "schedule": { "description": "Polls the Google Flood Forecasting API for updated gauge flood status, flash floods and significant events on a schedule.", "default": { "enabled": true, "schedule": "rate(2 minutes)" } } } } }OCI annotation: Docker 29.x was available locally. Using a throwaway buildx builder and a local
registry:2, the exact workflowdocker buildx buildcommand was run andimagetools inspect --raw | jq .annotationsshowed the annotation byte-identical tojq -c . capabilities.json. This was run before.dockerignorewas added. The push to real ECR is NOT verified.Post-merge manual verification
Not done by an agent. Nothing has been deployed.
<stackname>-etltasksECR repo)GET /api/task/raw/etl-floodhub/version/:versionreturns the capabilities