PR 25 of #508 — centroid-only inference + mean-of-visible-nodes anchor fallback#562
Draft
gitttt-1234 wants to merge 1 commit into
Draft
Conversation
…r fallback Closes #98 (centroid-only inference) — the first new user-facing capability after the refactor. Adds: * `Predictor.from_model_paths` accepts a centroid-only model_paths list (auto-detected when no centered-instance model is also configured) and returns a predictor whose layer is a bare `CentroidLayer`. * `Outputs.to_instances` / `to_labels` handle the centroid-only packaging path: when `pred_keypoints` is None but `pred_centroids` is populated, produce a NaN-padded skeleton with the centroid at `anchor_ind` (or node 0 if unset). Per-instance score = centroid value. * `Predictor._packaging_anchor_ind()` resolves the anchor slot from the underlying `CentroidLayer`. * `--centroid-only` CLI flag for the explicit case (both centroid + centered-instance configured, but user wants centroid-only output). * `FilterPipeline` emits a `UserWarning` and falls back to IoU when `overlapping_method='oks'` is requested on centroid-only outputs. * `docs/guides/centroid-only-inference.md` describes NaN semantics, anchor convention, and interaction with filters/tracking/metrics. Also changes the project-wide anchor-default convention: when `anchor_part` / `anchor_ind` is not provided (or the anchor node is NaN for an instance), the centroid falls back to the **NaN-ignoring mean of all visible nodes** instead of the bounding-box midpoint. Replaces `find_points_bbox_midpoint` with `find_points_mean` at the canonical `generate_centroids` helper, which both training (`custom_datasets`) and inference (`CentroidLayer`, `CentroidCrop`) consume. `find_points_bbox_midpoint` is retained for callers that explicitly want bbox-midpoint behavior. Docstrings updated in `model_config`, `inference/topdown`, `inference/layers/centroid`. Behavior shift: centroid models trained on the old bbox-midpoint convention will see a slightly different GT centroid target whenever the anchor node is missing; for symmetric instances the two are identical, for asymmetric ones (long tails, sprawled limbs) they differ. Re-training is recommended only if `anchor_part` was unset. Tests ----- ``` tests/data/test_instance_centroids.py 3 passed (1 new) tests/inference/test_centroid_only.py 9 passed (new) tests/cli/test_centroid_only_cli.py 4 passed (new) tests/inference/ + tests/cli/ + test_instance_centroids 404 passed, 23 skipped tests/data/ (full) 104 passed tests/test_predict.py + test_topdown.py 20 passed black --check sleap_nn tests clean ruff check sleap_nn/ clean ``` Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## divya/inf-refactor-24-bugfix-527-528 #562 +/- ##
========================================================================
+ Coverage 64.13% 64.20% +0.06%
========================================================================
Files 124 124
Lines 19019 19066 +47
========================================================================
+ Hits 12198 12241 +43
- Misses 6821 6825 +4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #560 (PR 24).
Closes #98 — centroid-only inference is the first new user-facing capability after the inference refactor (PRs 0–24 were all parity-preserving).
Two changes in one PR
1. Centroid-only inference (the feature)
Predictor.from_model_pathsnow accepts a centroid-onlymodel_pathslist and returns aPredictorwhose layer is a bareCentroidLayer. Auto-detected when only"centroid"is among the detected model types.Outputs.to_instances/to_labelsaccept ananchor_indkwarg. Whenpred_keypointsisNonebutpred_centroidsis populated, the centroid is packaged into aPredictedInstancewith the coordinate at the anchor slot (or node 0 if unset) and NaN at every other node. Per-instance score = centroid confidence.Predictor._packaging_anchor_ind()resolves the slot fromself.layer.anchor_indwhen the layer is aCentroidLayer;Noneotherwise.--centroid-onlyCLI flag (with--centroid_onlyunderscore variant) for the case where both centroid + centered-instance models are configured but the user wants centroid-only output. Routes throughfrom_model_paths(centroid_only=True).FilterPipelineemits aUserWarningand falls back to IoU whenoverlapping_method='oks'is requested on centroid-only outputs (OKS needs keypoints).docs/guides/centroid-only-inference.mddocuments the NaN semantics, anchor-node convention, and interaction with filters / tracking / metrics.2. Anchor-default convention: bbox-midpoint → mean of visible nodes
Per-user-request, the project-wide convention for "what's the centroid when
anchor_partisn't set, or the anchor node isn't visible" is changed from bounding-box midpoint to the NaN-ignoring mean of all visible nodes.sleap_nn/data/instance_centroids.py: newfind_points_meanhelper.generate_centroidsnow uses it for both the no-anchor and missing-anchor fallback paths.find_points_bbox_midpointretained for callers that explicitly want bbox-midpoint behavior.config/model_config.py,inference/topdown.py,inference/layers/centroid.py.custom_datasets) and inference (viaCentroidLayer+ legacyCentroidCrop) pick up the change automatically — they share the samegenerate_centroidshelper.Behavior shift to call out for review. Centroid models trained on the old bbox-midpoint convention will see a slightly different GT centroid target whenever the anchor node is missing for an instance. For symmetric instances the two are identical; for asymmetric ones (long tails, sprawled limbs) they differ by up to a few pixels. Re-training is recommended only when
anchor_partwas unset in the original training config.Test plan
pytest tests/data/test_instance_centroids.py— 3 passed (1 new — confirms mean ≠ bbox-midpoint on skewed instances; 1 new —find_points_meanignores NaN).pytest tests/inference/test_centroid_only.py— 9 passed (factory auto-detect; anchor=None default; explicitanchor_ind; per-instance score; out-of-range raise;.slpround-trip; OKS-warn + IoU-fallback; IoU silent; tracking-via-centroids smoke).pytest tests/cli/test_centroid_only_cli.py— 4 passed (--centroid-onlyin--help; flag → factory kwarg; default-off; underscore variant accepted).pytest tests/inference/ tests/cli/ tests/data/test_instance_centroids.py— 404 passed, 23 skipped (CUDA-gated).pytest tests/data/— 104 passed (full data suite — confirms anchor-default change doesn't regress training pipeline).pytest tests/test_predict.py tests/inference/test_topdown.py— 20 passed (legacyrun_inference+ legacyCentroidCroppaths still work with the new mean-of-nodes fallback).black --check sleap_nn tests— clean.ruff check sleap_nn/— clean.Anchor-slot decision
When packaging centroid-only
Outputsinto aPredictedInstance, we have to pick one skeleton-node index to receive the centroid coordinate (sleap-io has no "mean of nodes" slot inPredictedInstance). The packaging slot defaults to node 0 whenanchor_partis unset — distinct from the centroid value fallback (mean of visible nodes). These are different concerns: the value is "what's the centroid here?", the slot is "where in the output array does it go?".Files
Modified:
sleap_nn/data/instance_centroids.py— newfind_points_mean;generate_centroidsuses it.sleap_nn/inference/factory.py—_select_layeraccepts centroid-only;from_model_paths(centroid_only=True)forces dispatch.sleap_nn/inference/outputs.py—to_instances/to_labelsacceptanchor_ind; new_to_instances_centroid_only.sleap_nn/inference/predictor.py—_packaging_anchor_ind+_to_labelsthreads it through.sleap_nn/inference/filters.py— OKS-on-centroid-only warn + IoU fallback.sleap_nn/cli.py—--centroid-onlyflag wired into both_run_in_memory_new_flowand_run_stream_to_file.sleap_nn/config/model_config.py,sleap_nn/inference/topdown.py,sleap_nn/inference/layers/centroid.py— docstring updates.tests/data/test_instance_centroids.py,tests/inference/test_factory.py,tests/inference/test_tracking.py— test updates for new defaults + signature changes.Created:
docs/guides/centroid-only-inference.mdtests/inference/test_centroid_only.py(9 tests)tests/cli/test_centroid_only_cli.py(4 tests)🤖 Generated with Claude Code