Skip to content

feat(sensing-server): opt-in OpenTelemetry log export with a weaver-checked ruview.* registry#1382

Open
jensholdgaard wants to merge 4 commits into
ruvnet:mainfrom
jensholdgaard:otel-instrumentation
Open

feat(sensing-server): opt-in OpenTelemetry log export with a weaver-checked ruview.* registry#1382
jensholdgaard wants to merge 4 commits into
ruvnet:mainfrom
jensholdgaard:otel-instrumentation

Conversation

@jensholdgaard

Copy link
Copy Markdown

What

Opt-in OpenTelemetry logs export for the sensing server, with a curated set of sensing events under a ruview.* namespace whose names and attribute keys are defined in a semantic-conventions registry and enforced in CI:

  • semconv/registry/ — a weaver-validated (0.23, registry check --future) OTel semantic-conventions registry: 13 attributes and 8 log event names, all stability: development, with requirement levels on the event attribute refs and enums mirroring the code's actual value sets (the adaptive classifier's class list, the sensing source labels).

  • src/semconv.rs — Rust constants generated from the registry (weaver registry generate, template under templates/registry/rust/). The new semconv workflow pins weaver v0.23.0 by sha256, validates the registry, regenerates the module, and fails on drift (codegen no-diff).

  • src/telemetry.rs — tracing bootstrap with optional OTLP export, doubly gated:

    1. build: a new otel cargo feature (same gating principle as mqtt) keeps the opentelemetry 0.32 stack out of the default binary;
    2. runtime: even with the feature, export only activates when OTEL_EXPORTER_OTLP_ENDPOINT is set.

    Unset ⇒ the OTLP pipeline is never constructed and behavior is byte-identical to today's stderr fmt subscriber. When enabled, every tracing event exports as an OTel log record (service.name = "ruview") with a telemetry-induced-telemetry loop guard muting the exporter's own crates.

  • docker/otel-compose.yml + docs/observability.md — a one-command demo pipeline (sensing server on synthetic CSI → OTel Collector → an OTLP log backend) and the guide.

Why

The sensing server already narrates the interesting moments (presence flips, vitals, node churn, falls, MQTT failures) as free-text log lines. Giving the highest-value ones stable, registry-backed event names and attributes makes them consumable by any OTLP collector/backend — dashboards, alerting on ruview.fall.detected, per-node filtering — without scraping stderr, and the weaver CI gate keeps the schema from drifting silently.

The events

Event Emitted when
ruview.node.online first frame from a sensing node (CSI or edge-vitals path)
ruview.node.offline node evicted after 60 s without frames (now logged per node id)
ruview.presence.changed smoothed presence classification flips — transition-only, from all five SensingUpdate publish paths via one helper
ruview.vitals.estimate breathing / heart-rate estimate with confidences, every 100 ticks
ruview.csi.stats capture snapshot (frames processed, active nodes), every 100 ticks
ruview.fall.detected edge-vitals fall flag, edge-triggered per node
ruview.mqtt.error HA publisher publish/connect failures
ruview.model.loaded model loaded via the model-management API

Rate discipline: nothing new logs at frame rate — transitions and the 100-tick cadence only.

Deliberate omissions: ruview.room and ruview.mesh.channel were considered but not added — the registry only defines attributes for data this crate actually has (rooms live in the calibration crate; channel isn't in the frame structs).

Verification

  • cargo check -p wifi-densepose-sensing-server — default features, --features otel,mqtt, and --all-targets: clean (only pre-existing warnings, verified against the base commit). Each commit in this PR builds independently (default features checked at every commit boundary).
  • weaver registry check --future: pass; the generate + rustfmt cycle is hash-stable, so the CI no-diff gate is sound.
  • Runtime smoke: with the endpoint unset the binary behaves identically; with the endpoint set and no backend listening it runs without panic (export retries in the background).
  • End-to-end: ran the instrumented server (--source simulated) against a real OTLP logs backend — we dogfood with Ourios, but the export is backend-neutral and works with any OTLP collector/backend. Logs arrived under service.name = ruview, the curated events came through with their registry names/attributes, and template-level queries (event frequency by mined log template, severity >= warn filtering, log-template drift between time windows) answered correctly.

Caveats (honest limits)

  • The sensing server has no graceful-shutdown path, so the final OTLP batch flush on kill is best-effort (the guard's Drop never runs on SIGKILL/SIGTERM); the periodic batch export bounds the loss to the last export interval.
  • docker/otel-compose.yml and docker/otel-collector.yaml are parser-validated only (no docker CLI on the dev machine); the same env/feature path was verified end-to-end outside compose as described above.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y

jensholdgaard and others added 4 commits July 21, 2026 19:10
… CI gate

Add a weaver-validated (0.23, `registry check --future`) OpenTelemetry
semantic-conventions registry at semconv/registry/ defining the
`ruview.*` telemetry namespace: 13 attributes (node id, CSI
source/stats, presence state, motion level, inference confidence,
person count, vitals rates + confidences, model id) and 8 curated log
event names (node online/offline, presence.changed, vitals.estimate,
fall.detected, csi.stats, mqtt.error, model.loaded). Everything is
`stability: development` with requirement levels on the event
attribute refs, and enums mirror the code's actual value sets (the
adaptive classifier's class list, the sensing source labels).

templates/registry/rust/ holds the weaver-forge template that
generates the Rust constants module (landed with its first consumer in
a follow-up commit). The new semconv workflow pins weaver v0.23.0 by
sha256, validates the registry, regenerates the module, and fails on
drift (codegen no-diff).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
Add a `telemetry` module that installs the tracing subscriber, with
optional OpenTelemetry log export doubly gated so both the default
build and the default runtime are byte-identical to before:

- build: a new `otel` cargo feature (same gating principle as `mqtt`)
  keeps the opentelemetry 0.32 SDK/OTLP/appender stack out of the
  default binary; without it the module is exactly the long-standing
  stderr fmt subscriber
- runtime: even with the feature, export only activates when
  OTEL_EXPORTER_OTLP_ENDPOINT is set — unset means the OTLP pipeline
  is never constructed

When enabled, every tracing event exports as an OTel log record over
OTLP/gRPC with resource service.name = "ruview", through a batch
exporter, with a telemetry-induced-telemetry loop guard muting the
exporter's own crates (hyper/tonic/h2/tower/opentelemetry*) so a
failed export can never feed back into the exporter.

src/semconv.rs is the module GENERATED from semconv/registry/ by
`weaver registry generate` (see the module header for the exact
command); the semconv CI workflow keeps it in sync. Call sites adopt
the constants in the next commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
…names

Emit the highest-value sensing signals as tracing events carrying the
generated `ruview.*` event names and attribute keys (semconv/registry/),
and switch main() to telemetry::init() so they export over OTLP when
enabled:

- ruview.node.online / ruview.node.offline — first frame per node
  (both the CSI and edge-vitals UDP paths) and the 60 s staleness
  eviction, which now logs each evicted node id
- ruview.presence.changed — smoothed presence transitions, emitted
  from every SensingUpdate publish path (esp32 CSI, edge vitals,
  wifi, multi-BSSID, simulated) via one observe_sensing_update
  helper; transition-only, never per frame
- ruview.vitals.estimate + ruview.csi.stats — cadenced snapshots
  every 100 sensing ticks
- ruview.fall.detected — edge-vitals fall flag, edge-triggered per
  node against the previously stored packet
- ruview.mqtt.error — the HA publisher's publish/connect failure sites
- ruview.model.loaded — the model-management API load handler

Rate discipline: nothing new logs at frame rate; all per-frame paths
only emit on transitions or the 100-tick cadence. Stderr output is
unchanged apart from the new lines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
docker/otel-compose.yml brings up sensing-server (synthetic CSI by
default, built with `mqtt,otel` via a new SENSING_FEATURES build arg
in Dockerfile.rust) → an OpenTelemetry Collector → an OTLP-native log
backend, giving a queryable log pipeline from a single
`docker compose -f docker/otel-compose.yml up`.

docs/observability.md documents the ruview.* event registry, the
double gating (build feature + endpoint env var), and example queries
against the bundled backend (template frequency, warning/error
filter, log-template drift after a deploy).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQY9wfrfRggqSpMLH8Xj3Y
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant