Skip to content

perf(inference): skip known-valid image range scans - #1387

Merged
Borda merged 2 commits into
roboflow:developfrom
JESUSROYETH:perf/skip-known-valid-range-scans
Aug 22, 2026
Merged

Borda merged 2 commits into
roboflow:developfrom
JESUSROYETH:perf/skip-known-valid-range-scans

Conversation

@JESUSROYETH

Copy link
Copy Markdown
Contributor

predict() converts PIL images to 8-bit RGB, and to_tensor scales both those images and uint8 NumPy arrays into [0, 1]. It then runs (img > 1).any() and (img < 0).any() on the resulting tensor anyway.

Changes

  • Record when the conversion already fixed the range, and short-circuit the existing deferred checks to False for those inputs. This also covers file-path and URL inputs, since both get opened into a PIL image before the check runs.
  • Keep both reductions for tensor and non-uint8 NumPy inputs.
  • Add regression tests that fail if PIL, uint8 NumPy, or file-path conversion ever reaches Tensor.any (including under the public default include_source_image=True), plus float32 NumPy controls that still have to raise on both bounds.

Performance

I benchmarked with the public predict(image, threshold=0.5, include_source_image=False) call, on one real COCO image as a uint8 NumPy array, with torch.set_num_threads(1). It alternates baseline and candidate within 101 pairs, repeating each configuration in three independent processes; CUDA numbers ran serially on an RTX 4060 Laptop GPU.

include_source_image=False isolates the preprocessing path from the source-image copy. Under the public default include_source_image=True, predict() does more work per call (it also captures the source image), so this same absolute saving becomes a smaller share of a bigger total — the percentages below don't carry over to the default configuration as-is.

Model Precision Baseline median (ms) Candidate median (ms) Median reduction across processes
Nano FP32 10.99-11.13 9.86-10.06 9.77-10.76%
Nano FP16 8.14-8.21 6.99-7.06 13.86-14.06%
Small FP32 15.86-15.98 14.71-14.75 7.13-7.24%
Small FP16 10.25-10.45 9.15-9.23 10.66-11.84%

Each run compares the baseline with itself first, matches boxes by IoU, and compares the matched public detection fields as raw bytes. Baseline and candidate outputs matched in every reported run.

On CPU, the removed reductions take 4.13-4.35 ms per four real images at their original sizes and allocate 6,113,984 bytes per call. The skipped path takes 0.267-0.289 us and allocates no tensor memory. Public CPU predict() timing was noisy, so I'm not reporting it as a speedup.

Validation

  • New regression command: 4 failed / 2 passed before the implementation; 6 passed after it.
  • tests/inference/test_predict.py: 122 passed.
  • CPU inference tests: 281 passed, 1 skipped.
  • Precommit: 19/19 hooks passed, including strict mypy.
  • Split CPU suite: 4178 passed, 71 skipped, 1 failed. The failure is in tests/export/test_onnx_notes.py's _export_tiny_model doctest. It expects a fixed string, but under -n 2 it picks up an ONNX-export INFO log line from other work on the same xdist worker — it doesn't call predict() or touch this diff's code. I re-ran the same command on unmodified develop three times and got the identical failure, so it's pre-existing and unrelated.
  • The patch applies to current develop, and the open PRs touching detr.py don't change this validation block.

COCO mAP wasn't re-run separately. The skip only substitutes a value for (img > 1).any()/(img < 0).any() that to_tensor's [0, 1] scaling of PIL and uint8 NumPy input already guarantees — it doesn't change the tensor the model sees. Public detections on real COCO images came out byte-identical to baseline across every Nano/Small, FP32/FP16 configuration I measured. Since mAP is a deterministic function of detections and ground truth, byte-identical detections mean byte-identical mAP.

@codecov

codecov Bot commented Aug 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86%. Comparing base (bdff7a7) to head (98e0c04).
⚠️ Report is 2 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #1387   +/-   ##
=======================================
  Coverage       86%     86%           
=======================================
  Files          112     112           
  Lines        14628   14632    +4     
=======================================
+ Hits         12586   12590    +4     
  Misses        2042    2042           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Borda Borda added the enhancement New feature or request label Aug 22, 2026
@Borda
Borda requested a balanced review from Copilot August 22, 2026 09:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Optimizes inference preprocessing by skipping redundant pixel-range scans for inputs already normalized by conversion.

Changes:

  • Skips range reductions for PIL and uint8 NumPy inputs.
  • Preserves validation for tensors and other NumPy dtypes.
  • Adds regression coverage and changelog documentation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/rfdetr/detr.py Short-circuits guaranteed-valid range checks.
tests/inference/test_predict.py Adds regression and validation tests.
CHANGELOG.md Documents the inference optimization.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/inference/test_predict.py Outdated
Changes:
- Add a mocked successful URL regression that fails if converted image inputs reach pixel-range scans.
- Add a mixed PIL/float-NumPy regression and annotate the `tmp_path: Path` fixture.

Impact:
- Protect the public URL conversion path and per-image range-validation state from regressions.
- Resolve the open test typing review thread under the project's mandatory annotation policy.

Verification:
- `ruff check tests/inference/test_predict.py` — passed.
- `ruff format --check tests/inference/test_predict.py` — passed.
- `uv run --no-sync mypy src/rfdetr --no-error-summary` — passed.
- `uv run --no-sync pytest tests/inference/test_predict.py -n 1 -m "not gpu" --timeout=240 -p no:rerunfailures` — 119 passed, 1 skipped.
- `git diff --check` — passed.

Residual limits:
- Independent read-only specialist review and local CUDA execution remain external runtime gates.

---

Co-authored-by: Codex <codex@openai.com>
@Borda
Borda merged commit 73867d3 into roboflow:develop Aug 22, 2026
41 checks passed
@Borda Borda mentioned this pull request Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants