fix(pose): black canvas + stacked ghost skeletons in Live Demo#1381
Open
VolatusLab wants to merge 6 commits into
Open
fix(pose): black canvas + stacked ghost skeletons in Live Demo#1381VolatusLab wants to merge 6 commits into
VolatusLab wants to merge 6 commits into
Conversation
…churn
The Live Demo "Human Pose Detection" rendered a black canvas with stacked,
scrambled labels ("tudo preto e tudo em cima de tudo").
Two root causes, both in the pose pipeline:
1. Keypoint confidence was structurally dropped at the tracker boundary.
`PoseTrack::new`/`update_keypoints`/`create_track` only accepted [[f32;3]]
positions, so `KeypointState.confidence` kept its 0.0 init value forever.
The server emitted all 17 keypoints with confidence 0, which (a) made the
renderer skip every joint (guard `conf > 0.1`) → black canvas, and (b)
collapsed the bbox to the degenerate 0.6x1.0 fallback → every "Conf: 90%"
label piled onto one point. Fixed by threading per-keypoint confidence
through create/update; centroid-filled slots stay 0.0 (unobserved).
2. The Kalman tracker is tuned for world-space metres (measurement_noise
0.08 m, Mahalanobis gate 9.0) but was fed image pixels, making the effective
association gate ~1.5 px. A single person's natural ±20 px jitter never
matched its own track, so every frame spawned a fresh track and stale ones
lingered as confirmed ghost skeletons ("Persons: 4/5" for one real person).
Fixed by converting px<->m at the tracker boundary (PIXELS_PER_METER=100).
UI: pose-renderer keypoint-confidence thresholds made inclusive (>=), since the
server clamps confidence to [0.1, 1.0] and joints at the floor were dropped.
Tests: confidence round-trip, multi-person separation, and single-person
no-churn regression tests added. Verified end-to-end against the real binary:
stable "Persons: 1" over 427/427 frames, one clean human skeleton.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz
Independent feature in wifi-densepose-pointcloud (camera depth backend); no overlap with the pose-tracker fixes on this branch. Clean auto-merge. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz
Most recent of the automated submodule-update branches; the other 7 point at the same ruvector SHA, so only this one is merged. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz
…ything Records the two pose-pipeline fixes (keypoint confidence round-trip, px<->m tracker scaling) under [Unreleased] Fixed, and the merged ADR-271 Depth Anything Rust camera-depth backend under Added. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the Live Demo Human Pose Detection rendering a black canvas with stacked/scrambled labels ("tudo preto e tudo em cima de tudo"), and folds in the current independent dev work so the branch builds green as a unit.
Root causes & fixes (pose pipeline)
1. Per-keypoint confidence dropped at the tracker boundary.
PoseTrack::new/update_keypoints/create_trackonly accepted[[f32;3]]positions, soKeypointState.confidencekept its0.0init value forever. The server emitted all 17 keypoints with confidence0, which caused both symptoms at once:conf > 0.1) → black canvas0.6×1.0fallback → everyConf: 90%label piled on one point → scrambled textFixed by threading per-keypoint confidence through create/update; centroid-filled slots stay
0.0(unobserved).2. Kalman tracker fed pixels but tuned for metres.
measurement_noise=0.08 m+ Mahalanobis gate9.0gave an effective association gate of ~1.5 px. A single person's natural ±20 px jitter never matched its own track → every frame spawned a fresh track and stale ones lingered as confirmed ghost skeletons (Persons: 4/5for one real person). Fixed by converting px↔m at the tracker boundary (PIXELS_PER_METER=100).UI: pose-renderer keypoint-confidence thresholds made inclusive (
>=), since the server clamps confidence to[0.1, 1.0]and joints at the floor were dropped.Also merged (independent dev work)
agent/depth-anything-rust) — no overlap with the pose fix; 33 pointcloud tests pass.vendor/ruvectorbump to latest upstream (6a6c39e6) — most recent of the automated submodule-update branches (the other 7 point at the same SHA).Verification
wifi-densepose-signal: 478 + integration tests pass against the bumped ruvectorwifi-densepose-sensing-servertracker_bridge: all pass, incl. 3 new regression tests (confidence round-trip, multi-person separation, single-person no-churn)wifi-densepose-pointcloud: 33 tests passPersons: 1over 426/426 frames, one clean human skeleton.🤖 Generated with claude-flow
https://claude.ai/code/session_01BztNyDLaawF9UUCJ1wrwTz