Skip to content

feat: add SilaNodeClient for native SiLA2 device integration - #291

Merged
RyanTheRobothead merged 15 commits into
unstablefrom
sila_node_client
May 1, 2026
Merged

feat: add SilaNodeClient for native SiLA2 device integration#291
RyanTheRobothead merged 15 commits into
unstablefrom
sila_node_client

Conversation

@RyanTheRobothead

@RyanTheRobothead RyanTheRobothead commented Apr 16, 2026

Copy link
Copy Markdown
Member

Summary

SiLA2 client support is shipped as an experimental preview. The client surface, [sila] install extra, and binary/error handling may change. Authoring a MADSci node as a SiLA2 server is not yet supported — only client-side consumption of an existing SiLA2 server. The broader migration (server-side Sila2Node, async parity, admin commands, lock/cancel/pause integration, file-input Binary Transfer, MADSci-specific Features) is scoped in the design-only OpenSpec change sila2-native-node-design under umbrella issue #293.

What ships in this PR

  • SilaNodeClient (madsci.client.node.sila_node_client): New AbstractNodeClient subclass that connects to SiLA2 servers via gRPC using the sila2 SDK. Installed as an optional dependency (pip install "madsci.client[sila]") with graceful degradation when the SDK is absent. Supports:

    • sila://host:port URL dispatch (auto-routed by find_node_client() alongside http://).
    • Unobservable + observable command execution with exponential-backoff polling.
    • Server introspection (get_info, get_status, get_state).
    • FeatureName.CommandName dot-notation resolution; short-form when unambiguous.
    • Command introspection uses isinstance(_, ClientObservable/UnobservableCommand) against the SDK's public command base classes — properties, SDK helpers, and arbitrary callables are correctly excluded.
  • Bytes / ActionFiles support (with path-traversal hardening): SiLA commands returning bytes are written to disk under .madsci/sila_files/{action_id}/ as .bin files, surfaced as ActionFiles on ActionResult.files, and base64-encoded in json_result. Both action_id and response keys are sanitized via _safe_path_component() so a malicious server response or upstream caller cannot escape the per-action subdirectory. action_files=True in capabilities.

  • Layered connection-error diagnostics: Errors include structured context — classification (dns_resolution, connection_refused, connection_timeout, tls_error, grpc_error, unknown), target host:port, TLS mode, and actionable hints. send_action is split into three try/except layers (connection / resolution / execution); only connection-category execution failures get the wrapper. Command-resolution errors and plain command errors keep their original messages — no misleading "SiLA connection error: ..." for arg errors. Original messages always preserved; never double-wrapped.

  • Example SiLA2 server (examples/example_lab/example_modules/sila_example_server/): Minimal server with one ExampleDevice Feature exposing Greet (unobservable), CountDown (observable, intermediate progress), GenerateData (binary), and ServerUptime (Property). Runs as a Docker Compose service on 0.0.0.0:50052 with a Python TCP-socket healthcheck (the base image lacks nc/grpcurl); notebook_validator.depends_on uses condition: service_healthy to gate papermill startup. Bind failure now sys.exit(1) (was silently return). Wired into workcell_nodes in examples/example_lab/settings.yaml as sila_example: sila://localhost:50052.

  • Notebook & validation: sila_node_notebook.ipynb exercises every supported SilaNodeClient capability end-to-end (introspection, observable polling, binary data round-trip with content verification, error handling). It doubles as the SiLA validation harness — just validate_nb_sila runs it via papermill against the live compose service, integrated into just validate_notebooks. The recipe is self-sufficient (no prior just up required; healthcheck gates startup).

  • Documentation sync (post-audit):

    • CHANGELOG: New SiLA2 Native Node Client (Experimental) subsection under [Unreleased] / Added with explicit experimental framing.
    • docs/guides/node_development.md: New "Consuming SiLA2 Devices (Experimental)" section covering install, quick start, and what's not yet available.
    • src/madsci_client/README.md: New "SiLA2 Client (Experimental)" section under Node Clients.
    • examples/example_lab/README.md: sila_example_server added to node list, port 50052 to network requirements + netstat troubleshooting, TCP-socket healthcheck snippet, notebook to notebook list, and a dedicated SiLA Example Server section.
    • Notebook intro carries an experimental notice + tutorial-vs-validation-harness disclosure; install cell switched to the canonical pip install "madsci.client[sila]".
    • Stray unitelabs-sila comment in src/madsci_client/madsci/client/node/__init__.py corrected to reference the [sila] extra.
  • OpenSpec history: Spec-driven design artifacts for every increment (sila-node-client, example-sila-node, sila-client-review-fixes, sila-bytes-to-action-files, sila-client-review-cleanup, sila-client-connection-errors, sila-client-pr291-must-fixes) with proposals, designs, delta specs, and task lists. 6 capability specs synced to openspec/specs/.

  • 110 tests (test_sila_node_client.py) covering URL dispatch, command execution, observable polling, introspection, error handling, FAILED-status propagation, non-callable command rejection, graceful import failures, bytes serialization, ActionFiles extraction, end-to-end bytes handling, connection error classification, enriched error formatting, path-traversal sanitization, get_info exclusion of properties and non-command callables, and layered error classification (arg errors, unknown commands, double-wrapping, recognized vs unknown execution categories).

What is not in this PR (deferred)

The current preview is client-only. The following are intentionally out of scope and tracked as downstream implementation issues against the design proposal:

  • A Sila2Node server base class for authoring MADSci nodes natively as SiLA2 servers (downstream issue #A).
  • Async (async_*) method parity on SilaNodeClient (#B).
  • File input (parameter) support via SiLA Binary Transfer (#C — currently bytes responses only).
  • send_admin_command dispatch over SiLA — Lock/Cancel/Pause/Reset/Shutdown/SafetyStop (#D, #E).
  • Legacy state_handler shim (#F), NodeStatus/NodeInfo decomposition (#G).
  • get_log / get_action_history / get_resources (#B removes get_resources wholesale; #H re-homes the others as MADSci Features).

If you need any of these now, stay on RestNodeClient — the migration is "side-by-side with full parity until REST removal" by design.

SiLA2 Migration design proposal

This PR also lands the design-only OpenSpec change sila2-native-node-design — the standards-exploration deliverable for issue #294. It scopes the broader SiLA2 Migration project (#293) across the seven scope areas of #294: SiLA core standards audit (adopt/extend/replace/propose-upstream for each org.silastandard/core Feature), action → Command mapping, state → Properties decomposition, NodeStatus/NodeInfo evolution, file/binary handling, admin commands as a SiLA Feature (MadsciAdminController), and the AbstractNodeClient abstraction story.

Includes eight resolved decisions (RD1–RD8) and a draft list of ~12 downstream implementation issues (#A–#M) to open against #293 once accepted. Notable decisions: vendor-and-PR strategy for missing standard SiLA core Features (CancelController, PauseController, SimulationController), Sila2Node defaults to Observable Commands, legacy state shim namespaced under MadsciState.LegacyState.<key> (no top-level back-compat), get_resources capability removed wholesale, and push observability deferred to v2 (sketched but not adopted in v1).

This is design-only — no spec deltas, no code. Each downstream implementation issue will land its own OpenSpec change with concrete deltas.

Test plan

  • pytest src/madsci_client/tests/test_sila_node_client.py — 110 tests pass (mocked SiLA SDK)
  • pytest src/madsci_client/tests/ — full client suite passes (1365 tests, no regressions)
  • pytest src/madsci_common/tests/test_templates/test_template_engine.py — 311 pass (no template regressions)
  • Pre-commit hooks (ruff check, ruff format, nbstripout, etc.) — all pass
  • madsci-release-audit skill — CHANGELOG, docs, guides, example lab, notebooks, templates, skills all in sync
  • just validate_nb_sila — notebook executes against live example server via papermill (self-sufficient — no prior just up needed; healthcheck gates papermill startup)
  • just validate_notebooks — no regressions in other notebook validations
  • OpenSpec exploration deliverable reviewed; downstream impl issues #A–#M opened against Project: SiLA2 Migration #293; Research: SiLA2 standards audit and node API mapping (SiLA2 Migration) #294 closed

🤖 Generated with Claude Code

RyanTheRobothead and others added 6 commits April 15, 2026 10:54
Add a SilaNodeClient implementing AbstractNodeClient to enable MADSci
workcells to communicate directly with SiLA2 servers over gRPC without
REST middleware. Uses the sila2 Python SDK and integrates via the existing
URL-scheme dispatch mechanism with sila:// URLs.

Key changes:
- SilaNodeClient with support for observable/unobservable SiLA commands,
  dot-notation action names, server introspection, and property reading
- SilaNodeClientConfig with gRPC-specific settings (TLS, timeouts, polling)
- Conditional registration in NODE_CLIENT_MAP when sila2 is available
- sila2 as optional dependency (pip install "madsci.client[sila]")
- Rename MadsciClientConfig -> MadsciHttpClientConfig (with compat alias)
  to clarify it's HTTP-specific now that gRPC clients exist
- 46 unit tests covering all client capabilities

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a working SiLA2 example server to the example lab with a validated
Jupyter notebook demonstrating all SilaNodeClient capabilities.

Key changes:
- SiLA example server with Greet (unobservable), CountDown (observable),
  and ServerUptime (property) — generated via sila2-codegen
- Example server registered in compose.yaml and settings.yaml workcell_nodes
  so it appears on the dashboard
- sila_node_notebook.ipynb exercising get_info, get_status, get_state,
  send_action (sync + async), polling, and error handling
- validate_nb_sila justfile recipe added to validate_notebooks
- get_info() now populates ActionDefinition.args with ArgumentDefinition
  entries extracted from SiLA command parameters
- get_info() sets asynchronous flag based on observable vs unobservable
- Fixed FQI-to-short-identifier conversion for feature discovery
- Fixed observable command polling (instance.done instead of get_responses)
- Fixed Docker issues: replaced typer with argparse, added --disable-discovery
- Updated OpenSpec specs to match implementation learnings
- ruff per-file ignores for SiLA codegen output

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Archive sila-node-client and example-sila-node changes, syncing 5 delta
specs (sila-client-dispatch, sila-command-execution, sila-server-introspection,
sila-example-server, sila-notebook-validation) to openspec/specs/.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@RyanTheRobothead RyanTheRobothead added the enhancement New feature or request label Apr 16, 2026
@RyanTheRobothead RyanTheRobothead self-assigned this Apr 16, 2026
@github-actions

github-actions Bot commented Apr 16, 2026

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/madsci_client/madsci/client
  http.py 42-44
  src/madsci_client/madsci/client/cli/commands
  new.py 112, 131, 262-270
  src/madsci_client/madsci/client/node
  __init__.py 16-17
  sila_node_client.py 38, 47-50, 106, 132, 147-148, 190-196, 307, 421, 450-451, 468, 597-611, 625-626, 652-653, 702-717, 761, 764, 801
  src/madsci_common/madsci/common
  http_client.py 22-23
  utils.py 26-28
  src/madsci_common/madsci/common/types
  client_types.py
  event_types.py
Project Total  

This report was generated by python-coverage-comment-action

RyanTheRobothead and others added 9 commits April 16, 2026 11:07
Fix get_action_result to propagate FAILED status instead of hardcoding
SUCCEEDED, add callable() guard in qualified-name command resolution,
align validate_url type signature with parent class, and document
private SiLA SDK attribute usage. Includes 3 new tests and synced
OpenSpec specs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SiLA commands can return binary data (bytes), but the SilaNodeClient
previously converted these to lossy string representations via str().
This change detects bytes in SiLA responses, writes them to disk as
.bin files in .madsci/sila_files/{action_id}/, and populates
ActionResult.files as ActionFiles. The bytes are also base64-encoded
in json_result for lightweight inspection.

- Add bytes detection in _serialize_value() with sentinel dict pattern
- Add _extract_bytes_files() post-processor for file extraction
- Integrate into all action result paths (unobservable, observable, polling)
- Set action_files=True in NodeClientCapabilities
- Add GenerateData command to SiLA example server
- Add Binary Data / ActionFiles section to SiLA notebook
- Add 16 new tests covering all bytes handling scenarios

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove unused action_id parameter from _serialize_value(), document
top-level-only bytes extraction limitation and get_state() property
detection heuristic, narrow ruff per-file-ignores for SiLA example
server, simplify validate_url() to match parent class, add
sila_example_server to notebook_validator depends_on, and fix
validate_nb_sila recipe cleanup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… images

Two bug fixes:

1. `madsci new lab -n <name>` (and other `madsci new` subcommands) ignored
   the -n/--name argument when running in interactive mode (the default).
   The interactive prompt function now accepts an overrides dict so
   CLI-provided values are used as defaults for all 7 parameter types
   (string, integer, float, boolean, choice, multi_choice, path).

2. Generated Docker Compose templates used nonexistent local image names
   (e.g. `madsci-squid:latest`). Updated to use the correct GHCR images:
   `ghcr.io/ad-sdl/madsci_dashboard:latest` for lab_manager and
   `ghcr.io/ad-sdl/madsci:latest` for all other services. Also removed
   the stale `build:` section from lab_manager.

Adds 10 regression tests covering overrides for every parameter type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Classify connection errors into actionable categories (DNS, refused,
timeout, TLS, gRPC) and include target host:port, TLS mode, and
diagnostic hints in all error messages from SilaNodeClient. Applies
consistently across _get_sila_client, send_action, get_status,
get_state, and get_info. Adds 28 new tests covering classification,
formatting, and integration with client methods.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix: CLI interactive mode respects --name arg; correct compose images
Captures the standards-exploration deliverable for SiLA2 Migration issue
#294. Design-only proposal (no spec deltas) covering the seven scope
areas: SiLA core standards audit, action mapping, state decomposition,
NodeStatus/NodeInfo evolution, file/binary handling, admin commands as
a SiLA Feature, and AbstractNodeClient abstraction.

Includes resolved decisions RD1-RD8 and a draft list of ~12 downstream
implementation issues (#A-#M) to open against the parent project (#293)
once the design is accepted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Path-traversal hardening: _extract_bytes_files now sanitizes both
action_id and response keys via _safe_path_component (Path(name).name +
explicit reject of empty/./..) so malicious inputs can no longer write
outside the per-action sila_files/ subdirectory.

Command introspection: import sila2.client.{ClientObservableCommand,
ClientUnobservableCommand} and use isinstance(_, _SILA_COMMAND_TYPES)
in get_info, _resolve_sila_command (both dotted and short form), and
_is_command_observable. Properties (ClientObservable/Unobservable
Property), SDK helpers, and arbitrary callables no longer surface as
ActionDefinitions or resolution matches.

Layered error classification in send_action: split the single
try/except into three blocks - (A) _get_sila_client, (B) command
resolution + arg prep, (C) command execution. Only block C enriches,
and only when _classify_connection_error returns a non-"unknown"
category. Resolution errors and plain command errors keep their
original message; connection failures from block A pass through
already-enriched (no double-wrap).

Server startup: __main__.py now sys.exit(1) on bind failure instead of
returning silently with exit 0.

Compose: sila_example_server gets a Python-based TCP healthcheck on
port 50052; notebook_validator.depends_on is converted to long-form
with sila_example_server: condition: service_healthy. validate_nb_sila
in .justfile drops --no-deps so Compose handles the dep graph itself.

Tests: 86 -> 110 (+24). New TestSafePathComponent,
TestExtractBytesFilesSafety, TestSendActionErrorLayering classes; new
get_info exclusion tests for properties and non-command callables;
fixed test_unknown_action_returns_failed mock ordering. Test helpers
now build proper MagicMock(spec=ClientObservableCommand /
ClientUnobservableCommand) instances so isinstance checks succeed.

OpenSpec: sila-client-pr291-must-fixes archived. Modified deltas on
sila-command-execution, sila-server-introspection,
sila-example-server, sila-notebook-validation. One-time normalization
of the four affected finalized specs from "## ADDED Requirements" to
"## Purpose" + "## Requirements" so openspec's MODIFIED matcher could
locate existing requirements.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Following audit of PR #291, sync documentation, examples, and the
CHANGELOG with the SiLA artifacts that ship in this branch and frame
them consistently as an experimental client-only preview pending the
broader sila2-native-node-design migration.

- example_lab README: add sila_example_server to node list, port 50052
  to network/troubleshooting, TCP healthcheck snippet, notebook to
  notebook list, and a dedicated SiLA Example Server section.
- sila_node_notebook: switch install cell to '[sila]' extra; intro now
  carries the experimental notice and a tutorial-vs-validation-harness
  disclosure.
- sila_example_server/server.py: drop misleading code-generator header
  and document the file as hand-customized.
- docs/guides/node_development.md: add Consuming SiLA2 Devices section
  with what's available now vs. deferred (Sila2Node, async, admin,
  file inputs, log/history/resources).
- docs/CHANGELOG.md: add SiLA2 Native Node Client (Experimental)
  subsection under [Unreleased] / Added.
- src/madsci_client/README.md: add SiLA2 Client (Experimental) section
  alongside the REST client.
- src/madsci_client/.../node/__init__.py: fix stale 'unitelabs-sila'
  comment to reference the [sila] extra.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@RyanTheRobothead
RyanTheRobothead merged commit c2581bc into unstable May 1, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant