fix(export): pass CUDA device to ONNX Runtime benchmark - #1346
Merged
Borda merged 1 commit intoAug 15, 2026
Merged
Conversation
Pass the requested logical CUDA device to the ONNX Runtime execution provider so model execution stays aligned with PyTorch post-processing. Add CPU-only regression coverage for default and non-default device indices.
voropaevv
requested review from
Borda,
SkalskiP,
isaacrob and
probicheaux
as code owners
August 14, 2026 17:50
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes ONNX benchmark CUDA routing so ONNX Runtime and PyTorch post-processing use the same requested GPU.
Changes:
- Passes
device_idto the CUDA Execution Provider. - Adds regression tests for default and non-default devices.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/rfdetr/export/benchmark.py |
Configures ONNX Runtime with the requested CUDA device. |
tests/inference/test_trt_inference.py |
Verifies consistent device routing. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1346 +/- ##
=======================================
Coverage 85% 85%
=======================================
Files 111 111
Lines 14058 14058
=======================================
+ Hits 11983 11998 +15
+ Misses 2075 2060 -15 🚀 New features to boost your workflow:
|
Borda
approved these changes
Aug 15, 2026
Borda
pushed a commit
that referenced
this pull request
Aug 17, 2026
Pass the requested logical CUDA device to the ONNX Runtime execution provider so model execution stays aligned with PyTorch post-processing. Add CPU-only regression coverage for default and non-default device indices.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The ONNX benchmark documents
deviceas the CUDA device index and routes PyTorch post-processing tocuda:{device}, but the ONNX Runtime session did not receive that device index.Because the CUDA Execution Provider defaults
device_idto0, requesting a non-default logical device could run ONNX model execution on logical GPU 0 and PyTorch post-processing on another GPU.A two-GPU Tesla T4 reproduction with
device=1confirmed that the current implementation selected ORT device 0 and allocated memory on both GPUs.Fixes #1345
What changed
Verification
device_iddevice_id=0mutant:13 passed3910 passed, 76 skippedgit diff --checkReal CUDA validation:
device=1, two repetitions:device_id=0device_id=1[126, 140]MiB across GPUs 0 and 1[0, 162]MiBKnown local baseline:
src/rfdetr/export/_topk.py:31-32andsrc/rfdetr/evaluation/coco_eval.py:178,190Not verified
The hardware tests used the exact production session-construction and post-processing path with deterministic ONNX graphs to isolate CUDA device routing.
Risks / rollback
The change is limited to one existing ONNX Runtime provider option. The default
device=0behavior is preserved. The TensorRT path is unchanged.Reverting the commit restores the previous provider configuration.
Migrations / external effects
None.
Screenshots / preview
Not applicable; this changes CUDA device routing in the command-line benchmark.