perf(inference): avoid redundant eval-mode assignments in predict - #1419

What

RFDETR.predict() calls model.eval() on every unoptimised inference call. nn.Module.eval() just calls train(False), and that reassigns training on every registered module through nn.Module.__setattr__. It does this even when the whole tree is already in eval mode, so most calls pay a recursive walk that changes nothing.

This keeps the same per-call check, but skips the recursive assignments when there is nothing to change:

if any(module.training for module in model.modules()):
    model.eval()

The scan still covers the root and every registered submodule. So a replacement model left in training mode after RFDETR.train() still triggers the existing recursive eval() call, and so does a single submodule toggled directly under an eval-mode root. The optimised inference path still returns before this code, that part doesn't change.

Why it is safe

The unconditional call was added on purpose in d0740fd (#1146). Before that, predict() only entered eval mode while it was emitting its once-only warning, so predict() → train() → predict() could leave dropout active on the second prediction. This change keeps that fix working: RFDETR.train() rebinds self.model.model to the module Lightning returns, that replacement reports training mode, and the scan finds it and calls eval() on it.

I also considered checking only the root's training flag, but that breaks a mixed-mode tree, for example when a child gets toggled directly. Scanning every module keeps the old behaviour and still skips the expensive recursive assignments for the common case, everything already in eval.

On the RTX 4060, the read-only scan costs 0.18–0.22 ms for Nano/Small against 0.61–0.72 ms for the unconditional eval(). So it saves 0.43–0.51 ms of host work on every call that finds the tree already in eval. A call that reaches a module still training pays that same recursive eval() cost plus the scan on top, in that case it doesn't get faster.

Results

Public RFDETR.predict() on one real COCO image, pinned to develop@ffc69fd. Each process reports the median of 60 timed calls after 30 warm-up ones; brackets are the full range across counterbalanced baseline/patched fresh processes.

All six baseline/patched envelopes are disjoint. Every run gave the same detection count and the same SHA-256 fingerprint over xyxy, confidence and class_id in both arms. The removed work is host-side Python, so how much you save in absolute terms depends on the host CPU and the torch build. Its share also depends on how much GPU work is left.

Tests

• New regression test is red on pristine develop (call(False)) and green with the patch.
• All 7 focused eval-mode tests pass: model rebind, mixed-mode child, redundant-assignment skip, and the optimised-inference early return.
• tests/inference/ with the repo's pytest addopts and CPU marker selection: 300 passed, 5 skipped, 13 deselected.
• The complete local CPU-CI marker surface, split into bounded shards: 4,486 passed, 78 skipped, 13 deselected. The only failure was the ONNX-notes doctest, which fails the same way on pristine develop.
• The complete gpu and not e2e_tensorrt surface on the L4: 71 passed, including all 22 GPU benchmarks and all 17 pretrained COCO val2017 inference cases across detection, segmentation, keypoints, predict(), and PTL evaluation.
• RF-DETR pre-commit: all 19 hooks pass, strict mypy included.
• Same 7 focused tests pass on the L4 validation VM too.

Not covered

• The 17 GPU COCO val2017 tests recomputed their metrics and all passed the repository's thresholds, but their successful pytest output does not print the exact mAP/F1 values.
• The isolated scan and assignment costs were only measured on the RTX 4060 host. The end-to-end public path was measured on both the RTX 4060 and the L4.