Migrate rpl_sdl to MADSci v0.8.0 - #8
Open
RyanTheRobothead wants to merge 4 commits into
Open
Conversation
The location_locations: block in this file is being deleted in the v0.8.0 migration; these calibration values will live in the new locations.yaml. Committing first so the diff is in git history as a reference point for the post-migration retraining pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- infra.compose.yaml: replace mongo/redis/single-postgres with FerretDB +
Valkey + split Postgres (rpl_ferretdb_postgres on 5432 backing FerretDB,
rpl_postgres_resources on 5434 for Resource Manager). SeaweedFS omitted
(no object-storage usage). pgadmin reseated against the resource DB.
- managers.compose.yaml: rename depends_on (mongodb->ferretdb,
redis->valkey, postgres->postgres_resources); pin dashboard image to
:v0.8.0.
- migration.compose.yaml (new): overlay used by `madsci migrate foss`
to bring up old-port mongo/postgres/redis pointed at the existing
.madsci/{mongodb,postgres,redis}/ data dirs. RPL-customized: old
postgres mounts .madsci/postgres/ with rpl:rpl creds (project default
is .madsci/postgresql_old/data/ with madsci:madsci).
.env updated locally (gitignored): MADSCI_VERSION=v0.8.0, added
DOCUMENT_DB_URL with FerretDB auth, RESOURCE_DB_URL to port 5434 with
madsci:madsci, deleted all *_MANAGER_DEFINITION lines (dead since v0.7.0).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 3c of the v0.8.0 migration. The location_locations: block in location.settings.yaml is gone in v0.8.0; lab locations live in locations.yaml (LabLocationConfig format, auto-discovered via walk-up). All 7 entries are authored here as lab-managed for now. The four ot2_* slots (ot2_beta.deck_2, ot2_gamma.deck_2) SHOULD be node-intrinsic via the OT-2 node's intrinsic_locations ClassVar, but that refactor is deferred — they'll move to the OT-2 node module in a follow-up. Calibration values are preserved verbatim from v0.7.0. Hardware retraining is planned post-migration; treat these as starting points. location_transfer_capabilities: remains unchanged (alias still resolves to transfer_capabilities in v0.8.0). Direct pydantic LabLocationConfig validation passes; the `madsci validate` CLI doesn't yet dispatch on locations.yaml so its terse '✗' there is a CLI gap, not a real error. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
End-to-end migration of the SDL deployment to MADSci v0.8.0:
location.settings.yamlbefore deleting thelocation_locations:block, so they survive in git history as a reference for the post-migration retraining pass.docker/infra.compose.yamlrewritten from mongo/redis/single-postgres to FerretDB + Valkey + split Postgres (rpl_ferretdb_postgreson 5432 backing FerretDB;rpl_postgres_resourceson 5434 for the Resource Manager). Dashboard image pinned to:v0.8.0. Newdocker/migration.compose.yamloverlay added formadsci migrate foss's old-port containers, customized for RPL paths (.madsci/postgres/rather than the upstream default.madsci/postgresql_old/data/)..envupdates land locally (gitignored):MADSCI_VERSION=v0.8.0, newDOCUMENT_DB_URLwith FerretDB auth,RESOURCE_DB_URLto 5434, all*_MANAGER_DEFINITION=lines deleted.location_locations:block inlocation.settings.yamldeleted; lab-managed entries moved to newlocations.yaml(LabLocationConfig format, v0.8.0). All 7 entries are lab-managed for now — the fourot2_*.deck_Nslots SHOULD be node-intrinsic via the OT-2 node'sintrinsic_locationsClassVar (deferred follow-up).Also includes 1 pre-existing local-only commit (641a9ca, Add otel support; fix ot2_gamma calibration issue) that was 1-ahead of origin/main before the migration branch — it rides along with this PR. Happy to split if you prefer.
What was actually run (against this branch, on
parker)madsci migrate foss --apply --skip-docker --skip-backup --step document-dbthen--step postgresql(postgres dumped/restored manually via docker exec since host's pg_dump is v16 and the new postgres is v17).madsci.resource_manager.migration_tool(0.7.0rc1 → 0.8.0, alembic upgrade)..madsci/registry.jsonwas found truncated mid-entry (line 233, "RPL Data Manager" had no closing}). Repaired in place so manager identity resolution works.Test plan
docker compose up -dbrings up all 7 managers healthy on ports 8000–8006 (verified)"version": "0.8.0"on/health(verified)madsci validateagainst the rpl_sdl/ directory returns all*.settings.yamlfiles valid (verified: 35 valid)locations.yaml(deferred — needs hardware retraining)🤖 Generated with Claude Code