Skip to content

PR 25 of #508 — centroid-only inference + mean-of-visible-nodes anchor fallback#562

Draft
gitttt-1234 wants to merge 1 commit into
divya/inf-refactor-24-bugfix-527-528from
divya/inf-refactor-25-centroid-only-inference
Draft

PR 25 of #508 — centroid-only inference + mean-of-visible-nodes anchor fallback#562
gitttt-1234 wants to merge 1 commit into
divya/inf-refactor-24-bugfix-527-528from
divya/inf-refactor-25-centroid-only-inference

Conversation

@gitttt-1234
Copy link
Copy Markdown
Collaborator

Stacked on #560 (PR 24).

Closes #98 — centroid-only inference is the first new user-facing capability after the inference refactor (PRs 0–24 were all parity-preserving).

Two changes in one PR

1. Centroid-only inference (the feature)

  • Predictor.from_model_paths now accepts a centroid-only model_paths list and returns a Predictor whose layer is a bare CentroidLayer. Auto-detected when only "centroid" is among the detected model types.
  • Outputs.to_instances / to_labels accept an anchor_ind kwarg. When pred_keypoints is None but pred_centroids is populated, the centroid is packaged into a PredictedInstance with the coordinate at the anchor slot (or node 0 if unset) and NaN at every other node. Per-instance score = centroid confidence.
  • Predictor._packaging_anchor_ind() resolves the slot from self.layer.anchor_ind when the layer is a CentroidLayer; None otherwise.
  • --centroid-only CLI flag (with --centroid_only underscore variant) for the case where both centroid + centered-instance models are configured but the user wants centroid-only output. Routes through from_model_paths(centroid_only=True).
  • FilterPipeline emits a UserWarning and falls back to IoU when overlapping_method='oks' is requested on centroid-only outputs (OKS needs keypoints).
  • docs/guides/centroid-only-inference.md documents the NaN semantics, anchor-node convention, and interaction with filters / tracking / metrics.

2. Anchor-default convention: bbox-midpoint → mean of visible nodes

Per-user-request, the project-wide convention for "what's the centroid when anchor_part isn't set, or the anchor node isn't visible" is changed from bounding-box midpoint to the NaN-ignoring mean of all visible nodes.

  • sleap_nn/data/instance_centroids.py: new find_points_mean helper. generate_centroids now uses it for both the no-anchor and missing-anchor fallback paths.
  • find_points_bbox_midpoint retained for callers that explicitly want bbox-midpoint behavior.
  • Docstrings updated in config/model_config.py, inference/topdown.py, inference/layers/centroid.py.
  • Both training (via custom_datasets) and inference (via CentroidLayer + legacy CentroidCrop) pick up the change automatically — they share the same generate_centroids helper.

Behavior shift to call out for review. Centroid models trained on the old bbox-midpoint convention will see a slightly different GT centroid target whenever the anchor node is missing for an instance. For symmetric instances the two are identical; for asymmetric ones (long tails, sprawled limbs) they differ by up to a few pixels. Re-training is recommended only when anchor_part was unset in the original training config.

Test plan

  • pytest tests/data/test_instance_centroids.py — 3 passed (1 new — confirms mean ≠ bbox-midpoint on skewed instances; 1 new — find_points_mean ignores NaN).
  • pytest tests/inference/test_centroid_only.py — 9 passed (factory auto-detect; anchor=None default; explicit anchor_ind; per-instance score; out-of-range raise; .slp round-trip; OKS-warn + IoU-fallback; IoU silent; tracking-via-centroids smoke).
  • pytest tests/cli/test_centroid_only_cli.py — 4 passed (--centroid-only in --help; flag → factory kwarg; default-off; underscore variant accepted).
  • pytest tests/inference/ tests/cli/ tests/data/test_instance_centroids.py — 404 passed, 23 skipped (CUDA-gated).
  • pytest tests/data/ — 104 passed (full data suite — confirms anchor-default change doesn't regress training pipeline).
  • pytest tests/test_predict.py tests/inference/test_topdown.py — 20 passed (legacy run_inference + legacy CentroidCrop paths still work with the new mean-of-nodes fallback).
  • black --check sleap_nn tests — clean.
  • ruff check sleap_nn/ — clean.

Anchor-slot decision

When packaging centroid-only Outputs into a PredictedInstance, we have to pick one skeleton-node index to receive the centroid coordinate (sleap-io has no "mean of nodes" slot in PredictedInstance). The packaging slot defaults to node 0 when anchor_part is unset — distinct from the centroid value fallback (mean of visible nodes). These are different concerns: the value is "what's the centroid here?", the slot is "where in the output array does it go?".

Files

Modified:

  • sleap_nn/data/instance_centroids.py — new find_points_mean; generate_centroids uses it.
  • sleap_nn/inference/factory.py_select_layer accepts centroid-only; from_model_paths(centroid_only=True) forces dispatch.
  • sleap_nn/inference/outputs.pyto_instances / to_labels accept anchor_ind; new _to_instances_centroid_only.
  • sleap_nn/inference/predictor.py_packaging_anchor_ind + _to_labels threads it through.
  • sleap_nn/inference/filters.py — OKS-on-centroid-only warn + IoU fallback.
  • sleap_nn/cli.py--centroid-only flag wired into both _run_in_memory_new_flow and _run_stream_to_file.
  • sleap_nn/config/model_config.py, sleap_nn/inference/topdown.py, sleap_nn/inference/layers/centroid.py — docstring updates.
  • tests/data/test_instance_centroids.py, tests/inference/test_factory.py, tests/inference/test_tracking.py — test updates for new defaults + signature changes.

Created:

  • docs/guides/centroid-only-inference.md
  • tests/inference/test_centroid_only.py (9 tests)
  • tests/cli/test_centroid_only_cli.py (4 tests)

🤖 Generated with Claude Code

…r fallback

Closes #98 (centroid-only inference) — the first new user-facing capability
after the refactor. Adds:

* `Predictor.from_model_paths` accepts a centroid-only model_paths list
  (auto-detected when no centered-instance model is also configured) and
  returns a predictor whose layer is a bare `CentroidLayer`.
* `Outputs.to_instances` / `to_labels` handle the centroid-only packaging
  path: when `pred_keypoints` is None but `pred_centroids` is populated,
  produce a NaN-padded skeleton with the centroid at `anchor_ind` (or
  node 0 if unset). Per-instance score = centroid value.
* `Predictor._packaging_anchor_ind()` resolves the anchor slot from the
  underlying `CentroidLayer`.
* `--centroid-only` CLI flag for the explicit case (both centroid +
  centered-instance configured, but user wants centroid-only output).
* `FilterPipeline` emits a `UserWarning` and falls back to IoU when
  `overlapping_method='oks'` is requested on centroid-only outputs.
* `docs/guides/centroid-only-inference.md` describes NaN semantics,
  anchor convention, and interaction with filters/tracking/metrics.

Also changes the project-wide anchor-default convention: when
`anchor_part` / `anchor_ind` is not provided (or the anchor node is NaN
for an instance), the centroid falls back to the **NaN-ignoring mean of
all visible nodes** instead of the bounding-box midpoint. Replaces
`find_points_bbox_midpoint` with `find_points_mean` at the canonical
`generate_centroids` helper, which both training (`custom_datasets`) and
inference (`CentroidLayer`, `CentroidCrop`) consume.

`find_points_bbox_midpoint` is retained for callers that explicitly want
bbox-midpoint behavior. Docstrings updated in `model_config`,
`inference/topdown`, `inference/layers/centroid`.

Behavior shift: centroid models trained on the old bbox-midpoint
convention will see a slightly different GT centroid target whenever
the anchor node is missing; for symmetric instances the two are
identical, for asymmetric ones (long tails, sprawled limbs) they
differ. Re-training is recommended only if `anchor_part` was unset.

Tests
-----

```
tests/data/test_instance_centroids.py             3 passed (1 new)
tests/inference/test_centroid_only.py             9 passed (new)
tests/cli/test_centroid_only_cli.py               4 passed (new)
tests/inference/ + tests/cli/ + test_instance_centroids
                                                404 passed, 23 skipped
tests/data/ (full)                              104 passed
tests/test_predict.py + test_topdown.py          20 passed
black --check sleap_nn tests                     clean
ruff check sleap_nn/                             clean
```

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented May 14, 2026

Codecov Report

❌ Patch coverage is 90.38462% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.20%. Comparing base (6703299) to head (60575f1).

Files with missing lines Patch % Lines
sleap_nn/inference/factory.py 57.14% 3 Missing ⚠️
sleap_nn/cli.py 75.00% 1 Missing ⚠️
sleap_nn/inference/predictor.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@                           Coverage Diff                            @@
##           divya/inf-refactor-24-bugfix-527-528     #562      +/-   ##
========================================================================
+ Coverage                                 64.13%   64.20%   +0.06%     
========================================================================
  Files                                       124      124              
  Lines                                     19019    19066      +47     
========================================================================
+ Hits                                      12198    12241      +43     
- Misses                                     6821     6825       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant