Skip to content

fix(pose): black canvas + stacked ghost skeletons in Live Demo#1381

Open
VolatusLab wants to merge 6 commits into
ruvnet:mainfrom
VolatusLab:VolatusLabRadar
Open

fix(pose): black canvas + stacked ghost skeletons in Live Demo#1381
VolatusLab wants to merge 6 commits into
ruvnet:mainfrom
VolatusLab:VolatusLabRadar

Conversation

@VolatusLab

Copy link
Copy Markdown

Summary

Fixes the Live Demo Human Pose Detection rendering a black canvas with stacked/scrambled labels ("tudo preto e tudo em cima de tudo"), and folds in the current independent dev work so the branch builds green as a unit.

Root causes & fixes (pose pipeline)

1. Per-keypoint confidence dropped at the tracker boundary.
PoseTrack::new/update_keypoints/create_track only accepted [[f32;3]] positions, so KeypointState.confidence kept its 0.0 init value forever. The server emitted all 17 keypoints with confidence 0, which caused both symptoms at once:

  • renderer skips every joint (guard conf > 0.1) → black canvas
  • bbox collapses to the degenerate 0.6×1.0 fallback → every Conf: 90% label piled on one point → scrambled text

Fixed by threading per-keypoint confidence through create/update; centroid-filled slots stay 0.0 (unobserved).

2. Kalman tracker fed pixels but tuned for metres.
measurement_noise=0.08 m + Mahalanobis gate 9.0 gave an effective association gate of ~1.5 px. A single person's natural ±20 px jitter never matched its own track → every frame spawned a fresh track and stale ones lingered as confirmed ghost skeletons (Persons: 4/5 for one real person). Fixed by converting px↔m at the tracker boundary (PIXELS_PER_METER=100).

UI: pose-renderer keypoint-confidence thresholds made inclusive (>=), since the server clamps confidence to [0.1, 1.0] and joints at the floor were dropped.

Also merged (independent dev work)

  • ADR-271 Depth Anything Rust camera-depth backend (agent/depth-anything-rust) — no overlap with the pose fix; 33 pointcloud tests pass.
  • vendor/ruvector bump to latest upstream (6a6c39e6) — most recent of the automated submodule-update branches (the other 7 point at the same SHA).

Verification

  • wifi-densepose-signal: 478 + integration tests pass against the bumped ruvector
  • wifi-densepose-sensing-server tracker_bridge: all pass, incl. 3 new regression tests (confidence round-trip, multi-person separation, single-person no-churn)
  • wifi-densepose-pointcloud: 33 tests pass
  • Docker image rebuilt and validated end-to-end: stable Persons: 1 over 426/426 frames, one clean human skeleton.

🤖 Generated with claude-flow

https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz

ruvnet and others added 6 commits July 20, 2026 10:12
…churn

The Live Demo "Human Pose Detection" rendered a black canvas with stacked,
scrambled labels ("tudo preto e tudo em cima de tudo").

Two root causes, both in the pose pipeline:

1. Keypoint confidence was structurally dropped at the tracker boundary.
   `PoseTrack::new`/`update_keypoints`/`create_track` only accepted [[f32;3]]
   positions, so `KeypointState.confidence` kept its 0.0 init value forever.
   The server emitted all 17 keypoints with confidence 0, which (a) made the
   renderer skip every joint (guard `conf > 0.1`) → black canvas, and (b)
   collapsed the bbox to the degenerate 0.6x1.0 fallback → every "Conf: 90%"
   label piled onto one point. Fixed by threading per-keypoint confidence
   through create/update; centroid-filled slots stay 0.0 (unobserved).

2. The Kalman tracker is tuned for world-space metres (measurement_noise
   0.08 m, Mahalanobis gate 9.0) but was fed image pixels, making the effective
   association gate ~1.5 px. A single person's natural ±20 px jitter never
   matched its own track, so every frame spawned a fresh track and stale ones
   lingered as confirmed ghost skeletons ("Persons: 4/5" for one real person).
   Fixed by converting px<->m at the tracker boundary (PIXELS_PER_METER=100).

UI: pose-renderer keypoint-confidence thresholds made inclusive (>=), since the
server clamps confidence to [0.1, 1.0] and joints at the floor were dropped.

Tests: confidence round-trip, multi-person separation, and single-person
no-churn regression tests added. Verified end-to-end against the real binary:
stable "Persons: 1" over 427/427 frames, one clean human skeleton.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz
Independent feature in wifi-densepose-pointcloud (camera depth backend);
no overlap with the pose-tracker fixes on this branch. Clean auto-merge.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz
Most recent of the automated submodule-update branches; the other 7 point at
the same ruvector SHA, so only this one is merged.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz
…ything

Records the two pose-pipeline fixes (keypoint confidence round-trip, px<->m
tracker scaling) under [Unreleased] Fixed, and the merged ADR-271 Depth Anything
Rust camera-depth backend under Added.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants