Skip to content

Latest commit

 

History

History
761 lines (509 loc) · 161 KB

File metadata and controls

761 lines (509 loc) · 161 KB

Changelog

All notable changes to RF-DETR are documented here.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Added

  • Documented the Apple Neural Engine fallback boundary for CoreML exports in docs/learn/export.md: how to choose compute_units, why fp32 never reaches the ANE, what fp16 costs in accuracy, which ops RF-DETR leaves on the CPU, and the measured latency per compute unit. New tests pin that boundary and the ExecuTorch CoreML delegate's whole-graph lowering. (#1024)

  • CocoDetection, YoloDetection and the WebDataset shard reader now decode JPEG files with simplejpeg (libjpeg-turbo straight into a NumPy buffer) when it is installed, which the [train] extra now includes. Both decoders wrap libjpeg-turbo and produce the same pixels with the PyPI wheels tested in CI, but simplejpeg bundles its own copy of libjpeg-turbo, separate from the one in Pillow's wheels, so two decoder builds ship with [train] and installing it can introduce small rounding differences in decoded pixels; draft_size applies the same power-of-two reduction on both decoders. The gain depends on what the consumer wants: at the decode stage, a reader taking the array directly, as _LazyYoloDetectionDataset does, measured 1.8x faster on smooth content and 1.3x on detailed content (i7-8750H, Pillow 12.3.0, simplejpeg 1.9.0; 640x480 2.38 ms to 1.30 ms, 1920x1440 21.19 ms to 11.78 ms). YoloDetection, CocoDetection and the WebDataset reader hand a PIL image to the transform pipeline, so each wraps that array with Image.fromarray, and that wrap costs about what the faster decode saves: YoloDetection.__getitem__ keeps a smaller end-to-end gain because the previous YOLO path copied through np.array as well, while for CocoDetection and the WebDataset reader the net effect is hardware-dependent, and PIL-out consumers may see a slight regression on some platforms until the CPU pipeline consumes arrays directly; they gain the shared decoder policy rather than speed. Non-JPEG files, an environment without simplejpeg, and JPEGs it rejects still decode through Pillow, with the same errors as before, and the simplejpeg path enforces Pillow's decompression-bomb limit (PIL.Image.MAX_IMAGE_PIXELS) before allocating. (#1391, #1473)

  • Added ModelConfig.cuda_graphs, an opt-in CUDA graph replay path for single-GPU detection training with BF16 precision. The registered detector stays unchanged for optimizer, EMA, and checkpoint ownership; the variable-length criterion remains eager, while each static input signature captures the model forward and its backward once and replays it on later batches. The enabled path logs once at train start and once per captured input shape, so an active graph run is distinguishable from eager in the console. Unsupported devices, distributed runs, segmentation, keypoints, non-BF16/non-FP8 precision (FP16, FP32), and gradient checkpointing stay eager with a warning (FP8 gained its own Transformer Engine capture route, described below), and a failed capture stops training with a process-restart instruction. On an NVIDIA L4 with RF-DETR Nano, BF16, batch 4, deterministic synthetic detection batches, and a fixed 8-resolution multi-scale set (expanded_scales=False; the default expanded_scales=True resolves to 11 resolutions for this model and was not benchmarked), the median public Lightning training batch fell from 149.8 ms eager to 101.7 ms graphed, a 32.4% median of the five paired per-run reductions (range 31.4-33.5%); all eight signatures captured on every run and none fell back. The speed costs memory: peak allocated CUDA memory rose from 2,395 MiB to 3,466 MiB and peak reserved memory from 2,754 MiB to 14,550 MiB for those 8 graph pools, because each resolution retains a private graph pool. Final-parameter and loss differences stayed inside an eager-vs-eager control. On real data at the other end of the range — Nano on an RTX PRO 6000, BF16, batch 64, resolution 384, multi_scale=False, COCO train2017 — graph replay matched eager at 3.54 it/s while compile=True cut the epoch from about 8 min 40 s to 7 min 10 s: graphs remove launch gaps and pay at small batch, compilation fuses kernels and pays at large batch. The advanced training guide has a decision rule. Full-dataset accuracy and larger variants were not measured. (#1410, #1468)

  • cuda_graphs=True can now be combined with compile=True: the model is compiled with Inductor's triton.cudagraphs option, so cudagraph trees record and replay the compiled forward and backward kernels, training_step marks each step for the graph-tree allocator, and the eager graph runner stays off. Measured on an RTX PRO 6000 (Nano, BF16, resolution 384, synthetic batches) the combination is 1.20× faster than compile=True alone at batch 4 (1.47× vs eager) and matches it at batch 64 (1.32× vs eager, +0.9%, inside noise); an A100 gave 1.70× over compile at batch 4. Scope is single-GPU detection training without gradient accumulation; segmentation, keypoints, gradient checkpointing, grad_accum_steps > 1, and multi-device runs fall back to compile-only with a warning, because cudagraph trees allocate gradient outputs inside the graph pool and cannot accumulate across replays. Only BF16 was measured. When amp_dtype="fp8" is combined with cuda_graphs=True and compile=True, RF-DETR warns and keeps the run compile-only — it does not wrap the compiled module in the Transformer Engine graph helper, and the two capture runtimes are never nested; the Transformer Engine capture route below needs compile=False. (#1410, #1479)

  • Added a Transformer Engine FP8 CUDA-graph capture route (cuda_graphs=True, amp_dtype="fp8", compile=False): the fixed-shape training step is captured through Transformer Engine's make_graphed_callables under the active Lightning FP8 recipe, with returned gradients cloned before Transformer Engine reclaims its buffers. Scope is single-GPU detection training with grad_accum_steps=1, multi_scale=False and square_resize_div_64=True; unsupported combinations stay eager with a warning, and a capture failure stops training with the original exception attached, matching the process-restart guidance above. The capture call needs a Transformer Engine release providing the 2.19 make_graphed_callables API (clone_param_grads_on_return); an older release raises instead of silently falling back. Tested against Transformer Engine 2.19.0, installed through the existing cuda extra (transformer-engine[pytorch]>=2.19,<3 on Linux x86-64). On a synthetic RF-DETR Nano run (384 px, batch 4, RTX PRO 6000 Blackwell, 20 warm-up plus 50 measured steps), eager averaged 78.075 ms per step versus 40.066 ms with Transformer Engine-aware graphs (about 1.95x), one capture serving 70 calls; this is a single fixed-shape synthetic measurement, not COCO parity, accuracy, or large-batch evidence. See the advanced training guide's CUDA graph training section for the full compatibility matrix. (#1481)

  • DDP training (strategy='ddp'/'auto'/'ddp_spawn'/'ddp_notebook') now also sets static_graph=True and gradient_as_bucket_view=True on DDPStrategy, alongside the existing unconditional find_unused_parameters=True. PyTorch's own DDP documents static_graph=True as skipping the per-iteration autograd-graph search find_unused_parameters=True otherwise repeats every step, as long as the set of parameters that can go unused across the run stays fixed; an empirical probe (real RFDETRNano/RFDETRSegNano forward+backward, several batch/target combinations including all-empty-target images) found that set is a single, constant parameter, backbone.0.encoder.encoder.embeddings.mask_token (a DINOv2 masked-pretraining token untouched in ordinary supervised training), not the data-dependent set the comment on this flag used to describe — so the precondition holds. gradient_as_bucket_view=True is bundled in as a complementary, lower-risk flag measured alongside it. Scope is limited to what was measured: automatic-optimization models (detection, segmentation) with grad_accum_steps<=1 (reading the effective accumulate_grad_batches, including a caller's trainer_kwargs override); keypoint models (manual optimization) and grad_accum_steps>1 (DDP's no_sync() across multiple backward calls per optimizer step) keep the previous behavior (static_graph=False), since neither was exercised by the measurement below. On 2x NVIDIA L4 (driver 580.178.04, torch 2.12.1+cu130, real NCCL, RF-DETR Nano, BF16, batch 4/GPU, real SetCriterion, real multi_scale cycling, real AdamW step, 25 measured steps after 8 discarded warm-ups, median of 5 independent process-group runs): static_graph=True+gradient_as_bucket_view=True alone gave a median -8.70% full training step (range -5.06% to -8.93%); combined with reusing the two-stage selection's already-computed classification logits instead of recomputing them (see the next entry) gave a median -10.56% (range -10.37% to -11.41%, tight and consistent across all 5 runs), zero errors in any run. Gradient correctness was verified under real NCCL (per-parameter gradients bit-identical between baseline and candidate across several steps including an all-empty-target batch) in addition to an earlier 2-rank gloo/CPU proof. world_size>2 and real (non-synthetic) COCO data were not exercised. (#1489)

  • Transformer._two_stage_group_selection's batched two-stage top-k pass now also gathers enc_out_class_embed's already-computed per-position output at the selected positions and returns it, instead of the caller (LWDETR.forward) re-running enc_out_class_embed a second time, per group, on the same gathered hidden state to produce enc_outputs["pred_logits"]. enc_out_class_embed is a plain per-position nn.Linear, so gathering its output is what re-running it on the same gathered subset would produce, within the same float32 GEMM-accumulation-order tolerance already documented for the sibling batched selection; a new test (test_two_stage_group_selection_class_logits_reuse_matches_recomputed_loop) builds a real production RFDETRNano and checks this against the untouched fallback loop directly. Only the fast path (eligible group_detr>1 configs, i.e. every shipped default) is changed; the rare fallback loop (custom/heterogeneous group modules) is untouched and still recomputes. On its own this removed an entire redundant per-group forward+backward and measured a median -2.75% full training step on 2x L4 (range -0.78% to -2.82% across 5 runs) — below this project's 3% GPU-training bar alone — but composes with the static_graph/gradient_as_bucket_view change above to a reliable -10.56% combined median; see that entry for the full measurement. (#1489)

  • Added RFDETR.export(format="litert"), a direct PyTorch → LiteRT .tflite route through litert-torch (torch.export capture, no ONNX or TensorFlow step), installed with pip install "rfdetr[litert]" and implemented as a LiteRTExporter alongside the other exporter classes. It writes one float32 file per export (quantization other than None/"fp32" and dynamic_batch=True raise NotImplementedError), runs on LiteRT's CPU (XNNPACK) delegate, and, on the pretrained Nano / Seg-Nano checkpoints, tracks eager PyTorch to about 1e-7 (boxes) and 3e-5 (class logits, mask probabilities) over the confident queries and to about 2e-3 at worst over all 300 raw mask logits (the opt-in e2e_litert suite gates on looser regression bounds); keypoint models are not supported on litert-torch 0.9.4. The single-level deformable-attention core no longer emits a one-output split in export graphs — the one op litert-torch could not lower — which leaves every other export route's numbers unchanged. (#1024, #1459)

  • Added TrainConfig.eval_backend, selecting the COCO evaluator used for validation and test mAP. Both options now ship with rfdetr[train]; "faster_coco_eval" restores the previous evaluator. Keypoint OKS evaluation is unaffected, and the ONNX/TensorRT benchmark evaluator in rfdetr.evaluation.coco_eval continues to use faster-coco-eval directly.

  • Added "ufcoco" as a third value of TrainConfig.eval_backend, selecting ultrafast-pycocotools, a Rust COCO evaluator (BSD-2-Clause, numpy as its only runtime dependency) that reproduces pycocotools' precision, recall and score arrays byte for byte. It is added to the train extra alongside the other two backends; the default stays "hotcoco". The selection reaches the validation, train-split and EMA metrics and the explicit distributed state merge alike, and the parity tests hold the backend to the same exact equality against faster_coco_eval as hotcoco, for box-only and box-plus-mask evaluation at eval_max_dets 100 and 500. Two places where ufcoco follows pycocotools more literally than the other two backends are adapted in rfdetr.training.coco_map: aggregate AP is summarized at the configured eval_max_dets rather than pycocotools' fixed 100, and the boolean masks TorchMetrics encodes are converted to uint8 first. Keypoint OKS evaluation and the ONNX/TensorRT benchmark evaluator are unaffected. (#1449)

  • Added "vernier" as a fourth value of TrainConfig.eval_backend, selecting vernier, a Rust COCO evaluator (MIT OR Apache-2.0). It ships with the train extra; the default stays "hotcoco". It runs in vernier's corrected parity mode, which corrects some pycocotools quirks similarly as hotcoco and faster_coco_eval.

  • Added python -m rfdetr.cli.webdataset as the dedicated packing entry point.

  • RFDETR.inference() now accepts compile_backend="inductor" as an opt-in backend for long-running inference at a fixed batch size and resolution; the default remains the existing TorchScript path. On CUDA, Inductor completes its two setup invocations and synchronizes inside inference() before the first public predict() call. Runtime benefit and compatibility depend on the workload, CUDA device, operators, and installed PyTorch version.

  • Added dataset_file="webdataset", an opt-in training input path that streams a split from pre-packed .tar shards instead of opening one file per image, together with python -m rfdetr.cli.webdataset to pack a COCO split into ~100 MB shards and a small JSON index. Shards are written with the standard library, so packing needs no extra dependency; reading them installs with pip install "rfdetr[data]". Image bytes are copied verbatim, and a decoded sample goes through the same ConvertCoco conversion, the same make_coco_transforms CPU pipeline (Albumentations included), the same collate function and the same pin_memory hand-off to the Kornia GPU stage as the loose-file loaders — verified by comparing a packed split against the directory it came from, tensor by tensor, on 500 real COCO images: identical pixels, boxes, labels and label space, including when a shard decodes through PIL's draft mode, which now rescales annotations by the actual draft-vs-full-resolution ratio to match. coco, roboflow and yolo are unaffected. Because a streaming dataset has no index to sample from, the training loader gives every worker a fixed sample count floored to a whole number of accumulation windows (batch_size × grad_accum_steps) — the streaming counterpart of the padding GradAccumAlignedDataset gives the map-style loader — while validation, test and predict instead let every worker drain its shards once, so a split is scored exactly once and those loaders report no length; a packed test split is used when present, falling back to val with a log line otherwise. Training refuses to start when world_size × num_workers exceeds the shard count, warns when an uneven shard split leaves the worst-served worker more than 5% short of the samples its epoch asks for, and now raises instead of continuing silently past 30% — a config that trained yesterday at that skew fails fast today — all measured from each shard's real sample count when the packer recorded them; the resolved plan (samples per worker × slots vs. total) is logged at INFO on every run, not only when a threshold is crossed. Each rank gets a fixed shard split assigned once at loader-build time rather than one that re-resolves RANK/WORLD_SIZE per process, so eval assignment stays deterministic even when those env vars are absent or inconsistent across workers, and shard order plus an in-stream reservoir buffer are both reshuffled each epoch — reseeded from a per-worker epoch counter so a persistent_workers=True loader (the DataModule's own default whenever num_workers > 0) reshuffles too instead of replaying its first epoch's order, a local shuffle rather than the global permutation shuffle=True gives a map-style loader. Measured on a 32-vCPU instance with an NVMe-attached persistent disk, streaming COCO 2017 shards gave the same loader throughput as the loose files across six configurations (0.92x–1.03x; page cache cold and warm, 8 and 32 workers, default and Albumentations augmentation) and is not I/O-bound there, though construction is faster (median 19.9 s down to 0.012 s per DDP rank) because the annotation file is never parsed — packing should pay off where per-file access is genuinely the constraint (network filesystems, object-store mounts, millions of small files), which that disk was not. Accuracy is not established either way: over five seeds of a 3-epoch, 2,000-image fine-tune, the streaming median landed about 0.011 mAP@50:95 below the map-style loader at 39 shards and matched it at 290 (the shard-count skew above), with 2–3x the seed-to-seed spread in both cases; parity on a full-length run over a real dataset was not measured. Supports detection and segmentation splits — keypoint training rejects this format explicitly, since its label space is inferred from a whole parsed COCO annotation file that a shard index does not carry. num_classes auto-detects straight from the train shard index — max(category_id) + 1 under category_ids="raw" (matching dataset_file="coco"'s convention), the filtered cat2label count under "remap" — same as class_names; shard paths resolve against the local dataset_dir, so streaming straight from object storage is not wired up here. (#1392, #1396)

  • Added TrainConfig.pad_targets_to, which pads every training image's targets to a fixed row count so the loss keeps one tensor shape across batches. XLA compiles per shape and the detection loss is shaped by the ground-truth box count, so a TPU run recompiles whenever a batch presents a new per-image count; as reported in issue #1433 on a v5litepod-4, a new count costs roughly 47 s and four fresh compilations while a repeated one runs in 0.18 s, and on data with a realistic spread the run never reaches steady state at all. Measured for this fix on a Cloud TPU v6e-1 (RFDETRNano, resolution 256, batch size 4, 15 steps): 78 uncached compiles and 1453.9 s unpadded versus 9 uncached compiles and a median 159.0 s padded. With padding the same data pays a bounded warm-up and then holds steady. The default None keeps the existing variable-length path, which is what CUDA wants. Setting pad_targets_to also requires augmentation_backend to stay off Kornia/GPU (setup("fit") raises ValueError otherwise), and an image whose real box count exceeds pad_targets_to has its extra boxes dropped, logged as a warning. Padding is applied to the training dataloader only: the evaluation loaders keep their real targets, since padded rows would otherwise be counted as ground truth by COCO matching. It composes with pack_targets (#1399): padding runs first, so every sample the packer sees already shares one row count, and the packed batch it produces is bit-identical to packing the same padded targets directly. It is semantically transparent — the padded columns carry a query-independent cost so the Hungarian assignment on real targets is unchanged in both of HungarianMatcher's paths (the batched compact path, and the full cartesian path a batch of one image or a mask/keypoint target always takes), the box losses are masked, a query matched to a filler target keeps its background weighting in the IoU-aware BCE branch, and class_error is reduced with the mask. The three classification branches whose padded pairs are not masked yet (position-supervised, varifocal, plain focal), the mask loss for segmentation models, and the keypoint loss all raise rather than reporting a quietly wrong loss. (#1433, #1450)

  • Added a live opt-in end-to-end CI job (roboflow-deploy-e2e, -m e2e_roboflow) that generates a fresh dataset version in a dedicated Roboflow test project, deploys a real model with version omitted, and independently polls the server-side trained-model status — catching silent server-side upload failures that deploy_to_roboflow()'s return value cannot surface. (#1116)

  • RFDETR.export(backbone_only=True) now exports the encoder and feature projector instead of calling the full detector and failing with an AttributeError. ONNX exports retain every configured feature-pyramid level and support dynamic batches.

  • pack_targets correctness for segmentation targets (the masks field) is now covered by a dedicated regression test through the real RFDETRDataModule collate path, closing a parity gap #1399 shipped without. (#1451)

Changed

  • training_config.json is now written when training starts, before the model, the dataset grids and the trainer are built, and rewritten with the final values once train() returns. It was previously written only after trainer.fit() returned, so a run stopped with Ctrl-C or killed by a crash left checkpoints in output_dir but no record of how it had been configured — the reproducibility record the file was added for was exactly the one a failed run did not get. Resumed runs write it too, recording the resumed run's own configuration. The start-of-run copy carries the same five keys as the final one, so its presence no longer implies the run reached training: class_names comes from TrainConfig.class_names when set, otherwise it is read straight off the dataset directory (COCO train/_annotations.coco.json or YOLO data.yaml), and a layout those readers do not understand — dataset_file="webdataset" among them — records class_names: null with num_classes: 0 until the post-fit write fills them in. A run into an output_dir that already holds a copy overwrites it as soon as it starts rather than when it ends, so a new run that dies early now replaces an earlier run's complete record instead of leaving it in place. The start-of-run write is skipped on any rank the launcher does not identify as rank 0 of node 0, through a private rfdetr.utilities.distributed._is_launcher_main_process() helper that defers the rank lookup to PyTorch Lightning and adds NODE_RANK, LOCAL_RANK and MPI-rank checks on top — the dataset-grid write now shares it, which incidentally fixes its own guard, previously LOCAL_RANK alone, letting one process per node through on a multi-node run and every process through under srun, which sets neither LOCAL_RANK nor NODE_RANK — since torch.distributed is not initialized before fit() and every DDP subprocess would otherwise race on the same path; the post-fit write keeps its global is_main_process() guard. The configuration is now serialized in full before the file is opened, so a serialization failure can no longer truncate an existing copy, and either write failing is logged as a warning instead of ending the run — previously only OSError was tolerated, so a configuration value json refused to serialize ended a training run that had already completed. (#1493)

  • The coreml extra no longer pins torch<2.12, so pip install "rfdetr[coreml]" resolves the current torch. The three [tool.uv] conflicts that existed only to support that pin (coreml against executorch, the typing group and ci-executorch-pin) are gone with it, so uv can now resolve coreml and executorch into one environment. The pin was attributed to a coremltools MIL lowering regression; it was not one. RF-DETR's two-stage encoder selects its queries with a torch.topk over the encoder tokens' class scores, and with untrained weights neighbouring scores sit about 1e-6 apart, inside the legitimate fp32 rounding difference between eager and the CoreML CPU runtime (measured at up to 1e-5 across the parity matrix). A near-tied pair swaps rank, two reference points trade queries, and self-attention spreads that into every logit (0.5-1.5 abs diff), which is what the parity suite was seeing; the ExportedProgram matches eager bit-exactly, the same .mlpackage passes on CPU_AND_GPU, and torch 2.11 fails identically on another seed. The e2e_coreml parity exports now select 5 queries, where the smallest measured top-6 gap is 3.1e-4 against a worst-case CoreML drift of 1e-5, and assert that margin as a precondition before every raw comparison of a two-stage graph instead of trusting it; the shipped query counts stay covered for every variant by output shape and finiteness, plus a structural check that their graph reaches no MIL operation the parity graph never executes — op types cannot discriminate numeric drift, so that check narrows the coverage gap rather than closing it. Raw-tensor divergence from a rank swap is a property of the ranking rather than of the conversion, and docs/learn/export.md now says so and points users at post-processed detections for validation. The coreml-parity CI job returns to the project's 3.10 Python floor, and the macOS legs of the regular CPU suite — which append the coreml extra — now resolve the current torch instead of being held at 2.11 by this pin, bringing them in line with the Linux and Windows legs. A removed inline test comment also blamed the "np.matmul overflow" convert-time RuntimeWarnings coremltools logs while constant-folding a weights-only linear op ("divide by zero encountered in matmul" / "invalid value encountered in matmul") for the flakiness; that was investigated and found causally irrelevant, refuted by the same underlying fact as the coremltools-bug theory above — a bad baked constant from a genuinely overflowing fold would fail on every compute unit, yet the same .mlpackage passes on CPU_AND_GPU. Lifting the pin pulls a two-minor torch jump (2.11 to the current 2.14 at time of writing) into rfdetr[coreml] installs, worth noting for macOS users on a pinned or otherwise constrained torch environment. The deleted comment's seed-scan rationale is preserved here rather than lost with it: the original flaky seed=7 default was replaced after scanning seed_all(0..12), where seed=0 passed structured-plus-real-image detection parity on 4 of 4 independent fresh-process re-runs (commits 5297c7cf, 491b86eb, 7e814a38). (#1024)

  • On CUDA, compile=True now also compiles the matcher's L1 box cost with dynamic shapes, independently of model CUDA graphs. The compiled helper bypasses eager target chunking to allow a fused reduction; compile=False keeps the existing memory-bounded eager path. Under BF16/FP16 AMP the fused compiled L1 cost serves the decoder (and auxiliary) layers; encoder-stage matching stays on the eager torch.cdist path because its predicted boxes are emitted in the reduced precision while targets stay float32. Target packing, assignment, mixed-dtype fallback, and the remaining losses stay eager. End-to-end GPU throughput and memory improvements remain unverified.

  • HungarianMatcher builds its L1 box cost with a broadcast reduction over the four box coordinates instead of torch.cdist(..., p=1), through the new rfdetr.utilities.box_ops.pairwise_box_l1_cost. torch.cdist is a general Minkowski-distance routine whose CUDA kernel cannot exploit a 4-wide feature dimension, which made it the most expensive single operator in the matcher: a torch.profiler trace of RFDETR Medium training (batch 8, resolution 576, multi_scale, BF16) attributed 37.6 ms/step in 5 calls, 15.3% of all device time, to aten::_cdist_forward. Results are bit-identical, not merely close: the reduction runs over the feature axis only, so the target axis is chunked under a fixed element budget (_L1_COST_ELEMENT_BUDGET, 32M elements) to bound the broadcast intermediate, and torch.equal holds against torch.cdist on 2-D, batched, chunked and CUDA matcher-scale shapes. That guarantee is scoped to an equal-dtype operand pair: a bfloat16/float16 pair is reduced in float32 first, which is the form torch.cdist is handed under autocast and refuses outside one. Mismatched input dtypes still delegate to torch.cdist so its rejection of a float32/float64 mix — which the matcher's dtype gate relies on to keep achieved precision independent of batch ordering — is preserved. Measured on an NVIDIA RTX 5060 Ti (torch 2.14.0+cu130): the operation itself is 15-29x faster across microbenchmark shapes spanning the operator's range ([40, 3900, 30]: 21.6 ms -> 1.13 ms; [40, 3900, 300]: 195 ms -> 13.0 ms; [8, 3900, 12]: 1.81 ms -> 0.07 ms — the shape the matcher's per-layer compact route actually produces in production, since _STACKED_COST_ELEMENT_LIMIT keeps the larger two shapes out of reach of the stacked route), and end-to-end training throughput on a real 3,000-image person/face subset rose 12.88 -> 14.81 img/s (+15.0%) on a math-SDPA configuration and 23.13 -> 24.36 img/s (+5.4%) on a fused-attention one. Peak memory at the largest shape grows from 538 MiB to 612 MiB (the chunked intermediate); no accuracy change, since the cost matrix is unchanged bit for bit on the equal-dtype pairs this path handles.

  • The training-only group_detr ranking/selection block now batches each group's projections, normalization, class ranking, top-k, and box MLP. The later encoder-output classification and GroupPose keypoint loops remain unchanged. On an NVIDIA L4 (RFDETRNano, batch 4, BF16, multi-scale, deterministic synthetic detection batches with real shapes/model/loss), median steady-state step time fell from 141.13 ms to 119.09 ms, with a 15.76% median paired reduction across five runs (15.48-16.10%). Custom heads and eval/export retain the generic path. A separate three-seed, four-epoch COCO-subset screen produced mixed mAP50-95 deltas (+0.0050, -0.0103, +0.0008) and mixed whole-train() timing; neither convergence nor public end-to-end speedup is claimed.

  • CUDA segmentation matching now computes class/bbox/GIoU through the compact per-image route once it can avoid at least 728,000 cross-image entries, while the BCE + Dice mask cost keeps its established full-batch calculation. The gate measures the work actually removed (batch_size * num_queries * (sum(sizes) - max(sizes))), so an imbalanced batch where one image owns nearly every target stays on the full-cartesian path; CPU/MPS, keypoint targets, and fixed-row target padding also retain that path. On an NVIDIA L4 (RFDETRSegNano, BF16, steady-state forward + criterion + backward, median of 5 repeats), batch size 8 with 10 targets/image improves 7.98% (7.79-9.03% range), and batch size 20 with target counts from a real COCO training run improves 12.28% (11.89-12.49%). The measured batch-size-4 point is neutral (-0.04%, -1.29% to +0.39%) and remains below the gate. Peak CUDA memory is unchanged at batch size 8 and within 0.01% at batch size 20. Differential tests cover heterogeneous and empty-target batches, group_detr > 1, projected mask-head outputs, the CUDA assignment branch, and non-finite fallback behavior with one preserved random point sample.

  • XLA validation, test, and the optional train-split (compute_train_metrics=True) evaluation callbacks now materialize each model forward once before COCO metric code reads individual tensors on the host, avoiding repeated compilation of overlapping lazy-graph fragments. On a Cloud TPU v5e-1 (RFDETRNano, resolution 384, batch size 2, five training batches with train-split metrics disabled and two validation batches), median end-to-end fit time across five fresh-process pairs fell from 310.2 s to 232.0 s (-25.2%); validation time fell from 105.8 s to 28.1 s (-73.4%), while metrics, model tensors, and checkpoint tensors remained identical. The train-split callback shares the same barrier but was not separately benchmarked. (#1058, #1467)

  • Multi-GPU keypoint training with grad_accum_steps > 1 now synchronizes gradients once per optimizer step instead of once per microbatch, avoiding redundant DDP reductions.

  • On Python 3.14+, dice_loss_jit, sigmoid_ce_loss_jit, batch_dice_loss_jit, and batch_sigmoid_ce_loss_jit are plain aliases of their eager functions because TorchScript is unsupported there; on Python 3.10–3.13 they remain torch.jit.script products for backward compatibility. As a result, import rfdetr no longer emits the unsupported-TorchScript warning on Python 3.14+. Python 3.14 joins the CPU CI matrix and the package classifiers.

  • The ONNX Runtime CPU inference session built by RFDETR.export(format="onnx")'s inference helper no longer lets its intra-op thread pool busy-spin between calls. Spinning previously contended for CPU with any other work sharing the process — including this same helper's own torchvision-based preprocessing step — for as long as the session was alive.

Deprecated

  • ModelConfig.amp is deprecated, removal in v1.14, superseded by TrainConfig.amp_dtype, which now accepts None to disable autocast. The two settings previously split one decision across both configs: the boolean gated AMP on the model config while the dtype was chosen on the train config, and amp=False silently voided any amp_dtype. amp_dtype is now the single authority — an explicitly set value always wins, including "fp8", which reaches the Transformer Engine hardware checks instead of being turned off. The legacy toggle still applies while amp_dtype is left at its default "auto", emitting a FutureWarning; "left at its default" is evaluated by value rather than by which fields were passed, so a config reloaded from training_config.json keeps honoring it. Replace amp=False with amp_dtype=None. One behavior change is not covered by that fallback: amp=False combined with an explicit amp_dtype="fp8" previously raised amp_dtype='fp8' requires model_config.amp=True and now runs FP8.

  • TrainConfig.fp16_eval is deprecated, removal in v1.14. It has had no runtime consumer since the PyTorch Lightning migration — evaluation precision follows amp_dtype — so it is warned about rather than migrated. Setting it to True emits a FutureWarning; the default stays silent, including on a dumped-config reload. Use amp_dtype="fp16" to evaluate in FP16.

Fixed

  • Multi-GPU validation and test metrics no longer count the images Lightning's DistributedSampler repeats to pad the split to a multiple of world_size. The sampler extends the index list with its leading indices, so up to world_size - 1 images were forwarded twice and COCOEvalCallback scored both copies: on a 21-image split over 2 GPUs the merged mAP state held 22 images, and larger world_size values repeat more. COCOEvalCallback now reads the sampler's own rank, num_replicas and total_size at each evaluation epoch start (shuffle=False, as Lightning configures it) and skips accumulation for the positions at or past the dataset length; the forward still runs on those samples, so every rank keeps an equal batch count and the epoch-end collectives stay symmetric. Applies to the mAP/mAR accumulators, the macro-F1 sweep and the keypoint OKS metric on the validation and test loops alike; single-process runs, drop_last samplers and the WebDataset loaders (which never pad) are unchanged. Measured with fixed pretrained RF-DETR Nano weights at eval_batch_size=1 on that 21-image split (2x RTX 4090, strategy="ddp"): two GPUs reported val/mAR 0.1286 and val/mAP_50_95 0.0286 against 0.1299 and 0.0287 on one GPU, and now report the one-GPU values exactly.

  • run_test=True no longer hangs multi-GPU training at the end of fit. BestModelCallback.on_fit_end returned early on every rank but the main one and then called trainer.test() from the main process alone; under strategy="ddp" (the default for devices>1) that call is collective, so the main process blocked in the first test-loop collective while the other ranks had already left fit, and the run never finished (reproduced on 2x RTX 4090 with RF-DETR Nano: rank 1 returned after 9 s, rank 0 was still waiting when killed at 600 s). Every rank now enters trainer.test(): the main process still does all file work (best-checkpoint selection, checkpoint_best_total.pth, last_ema.pth), a barrier orders those writes before the other ranks read, and the decision to test plus the winning source (EMA or regular) are broadcast from the main process so no rank can take a different branch; each rank then loads checkpoint_best_total.pth and the EMA swap suppression applies on every rank rather than only the main one. Single-device runs are unchanged. Spawn-based launchers (strategy="ddp_spawn" / "ddp_notebook") cannot host a nested trainer.test() at all, since the spawn launcher has no notion of already running inside a worker and previously failed with DistNetworkError: EADDRINUSE after spawning a second set of processes; those workers now log a warning and skip the fit-end test, pointing at RFDETR.from_checkpoint("checkpoint_best_total.pth").evaluate(split="test"), instead of crashing.

  • Detection and segmentation training with grad_accum_steps > 1 no longer divides the loss by the accumulation count twice. RFDETRModelModule.training_step returned loss / accumulate_grad_batches on the automatic-optimization path, but Lightning already divides the returned loss by accumulate_grad_batches in ClosureResult.from_training_step_output before calling backward(), so the gradient handed to the optimizer after an accumulation window of N microbatches was 1/N of the mean gradient (1/N**2 of the summed one) rather than the mean. Measured with a real Trainer.fit() on a one-parameter model whose loss has gradient exactly 1.0 per microbatch: the accumulated gradient was 0.5 at N=2, 0.25 at N=4 and 0.125 at N=8, and is 1.0 for every N after this change. AdamW is largely invariant to gradient scale, so the visible effects were on gradient clipping and on any scale-sensitive optimizer: clip_max_norm (default 0.1) was applied to a gradient N times smaller than the single-large-batch gradient it is meant to emulate, so clipping engaged N times less often (an effective threshold of 0.1 * N), and SGD-family optimizers trained with an effective learning rate of lr / N. batch_size="auto" selects grad_accum_steps > 1 on its own, so runs that never asked for accumulation were affected too. The logged train/loss was already the unscaled value and is unchanged; keypoint models (manual optimization, which scales its own backward loss) are unchanged; grad_accum_steps=1 is unchanged. Introduced in 1.8.0 (#1117).

  • python -m rfdetr.export.benchmark no longer crashes when COCO evaluation is enabled. main() passed the annotation file path to CocoEvaluator, which needs a loaded COCO object, so every run without --disable_eval raised AttributeError: 'str' object has no attribute 'dataset' before any inference ran; the annotations are now loaded with COCO(...) first. Only the evaluation path was affected — --disable_eval never constructs an evaluator and worked throughout.

  • fp16 native CoreML exports of an untrained model now compile for the Apple Neural Engine. The two-stage selection gathered the output of enc_output_norm for the CPU-resident topk gather while the same tensor fed the class head on the ANE; when that norm still carries its identity affine (weight all ones, bias all zeros) and the encoder token count is a multiple of 32, the ANE compiler rejected the whole program, so the model ran entirely on CPU under ComputeUnit.CPU_AND_NE and failed to load under the default ComputeUnit.ALL. Trained checkpoints have a non-identity affine and were never affected — RFDETRNano, RFDETRSmall, RFDETRSegSmall and RFDETRKeypointPreview all compiled before this change — so this reaches models exported before training, including the ones the export tests build. The selection now gathers the pre-norm rows and normalizes only the selected tokens, which is the same computation because LayerNorm acts per token; eager outputs are unchanged. Measured before and after on a pretrained RFDETRNano (Apple M3 Pro, macOS 27.0, COCO val2017, all 5000 images): mAP is unchanged for eager (48.024), CoreML fp32 (48.023) and ExecuTorch xnnpack (48.022), and moves 45.054 -> 45.059 for CoreML fp16; latency is unchanged within noise on every compute unit. (#1024)

  • Compiled training keeps positional-embedding interpolation eager as a compatibility mitigation for PyTorch's symbolic antialiased-bicubic backward assertion, preserving interpolation outputs and gradients. RF-DETR no longer globally suppresses compiler errors. Failed CUDA graph capture now stops with the original exception and a process-restart instruction instead of retrying against potentially invalid CUDA state.

  • Explicit amp_dtype="bf16", and the default amp_dtype="auto", now select XLA's bf16-true precision instead of silently falling back to FP32 when training on TPU.

  • Multi-device TPU/XLA training now disables EMA with a runtime warning because per-step EMA weight reads can corrupt subsequent optimizer updates on four TPU cores under BF16. One-device XLA keeps EMA after the shipped callback completed 100 updates on the patched public training path and 350 updates on the commit before #1476, both on a Cloud TPU v6e-1 with finite live and EMA checkpoint tensors. Configurations that resolve to several devices or nodes, and device values that cannot be proven to select one chip, stay on the disabled path. CPU and CUDA EMA behaviour is unchanged. (#1058, #1476)

  • Segmentation validation/test mAP no longer risks CUDA OOM in _compute_mask_iou's boolean-to-float32 mask conversion, which previously materialized every matched prediction of a class at once (N x H x W, full image resolution by default) and every ground truth at once (M x H x W) — a densely annotated class can make M comparable to N, since GT count is bounded only by the image's own annotations, not by eval_max_dets. Both sides are now converted 32 rows at a time. For any nonzero prediction and ground-truth count, output is unchanged — bit-identical to the previous implementation; for a zero count on either side, the previous implementation raised an ambiguous-reshape RuntimeError instead of returning a value, and now returns an empty result. (#1460, #1463)

  • WebDataset validation/test-only runs retain the training index's class names even when evaluation categories are a subset, for both raw and remapped labels.

  • WebDataset training keeps shard permutations consistent across ranks with real DataLoader workers, aligns accumulation at rank level, and sizes raw-label heads from all declared categories. Repacking is documented as offline-only; path and shard-helper doctests are portable and executable.

  • Distributed (DDP) training now preserves the minimum five optimizer steps per epoch for short datasets instead of losing the replacement sample count when Lightning injects its distributed sampler.

  • Installing the [onnx] extra from a source checkout with uv on Python 3.10, 3.11 or 3.13 now brings in ml-dtypes again; the ml-dtypes==0.5.1 override, scoped to Python 3.12 for the TFLite stack, was dropping the requirement on every other interpreter and left import onnx failing with ModuleNotFoundError.

  • Kornia Affine now applies scalar translate_percent to both axes and reads scalar scale as a fixed range, avoiding silent horizontal-translation loss and construction failures. Scalar translation emits a warning because Kornia samples signed offsets while Albumentations applies the scalar as a fixed positive offset.

  • TFLite INT8 documentation and warnings now reflect that dynamic-range quantization needs no calibration data. (#1363, #1364)

  • RFDETR.export(format="tensorrt", fp16=True) now actually builds an FP16 engine on strongly typed TensorRT (11+) instead of silently falling back to FP32, by casting the ONNX graph to FP16 first (rfdetr[tensorrt] now also pulls onnx and onnxconverter-common). This graph cast raises ImportError if those two packages are missing — previously such a setup silently produced an FP32 engine reported as FP16. (#1453, #1454)

Breaking Changes

  • Removed RFDETR.optimize_for_inference(), deprecated since v1.9.0. Call RFDETR.inference(); the signature is unchanged.

  • Removed TrainConfig.lr_drop and TrainConfig.lr_min_factor, deprecated since v1.9.0, together with the validator that folded them into lr_scheduler_kwargs. Pass lr_scheduler_kwargs={"lr_drop": ..., "min_factor": ...}; the managed "step" / "cosine" presets fall back to lr_drop=100 and min_factor=0.0 when a key is absent. TrainConfig rejects unknown fields, so a training_config.json written by v1.9 or v1.10 must have the two keys removed before it is passed back to TrainConfig(**...); the migrated values are already present in its lr_scheduler_kwargs.

  • Removed rfdetr.datasets.synthetic (generate_coco_dataset, generate_synthetic_sample, draw_synthetic_shape, calculate_boundary_overlap, DatasetSplitRatios, SYNTHETIC_SHAPES, SYNTHETIC_COLORS). The module only ever fed RF-DETR's own test fixtures and is replaced by the fuse-augmentations package, whose fuse_augmentations.data module generates the same shape datasets in COCO or YOLO layout for detection, segmentation, and OBB. Callers migrate to pip install fuse-augmentations plus from fuse_augmentations.data import generate_dataset; note that it writes dense COCO category ids where the removed generator wrote sparse ones, and its shape set adds rectangle.

  • Renamed the optional installation extra from webdataset to data, without a compatibility alias. Use pip install "rfdetr[data]"; dataset_file="webdataset" is unchanged.

  • Restructured the export internals into one Exporter class per format, each built from its own configuration dataclass and reached through a registry that maps a format name to the module defining it. RFDETR.export() is unchanged — same signature, same accepted formats, same return value — but the modules behind it moved: rfdetr.export.main (including main() and make_infer_image, now rfdetr.export.prepare.make_infer_image) and rfdetr.export.protocols.ExporterProtocol (now rfdetr.export.base.Exporter) are removed, the per-format export_onnx/export_openvino/export_coreml/export_executorch/export_tflite functions are replaced by OnnxExporter/OpenVINOExporter/CoreMLExporter/ExecuTorchExporter/TFLiteExporter, rfdetr.export._tensorrt.build_engine becomes TensorRTExporter.build_engine in rfdetr.export._tensorrt.exporter, and rfdetr.export.benchmark.TRTInference moves to rfdetr.export._tensorrt.inference. Every removed path except rfdetr.export.main and rfdetr.export.protocols already carried a leading underscore and no stability guarantee. The format-independent graph preparation every format shared now runs once in rfdetr.export.prepare, and requesting a capability a format lacks — dynamic_batch on CoreML, ExecuTorch or OpenVINO — is refused from the registry's own data before that format's optional dependency is imported, rather than after paying for it. The contract each format implements, and the steps a new one takes, are documented in the new Exporter Blueprint page under Export Model in the docs.

  • RFDETR.export(format="tensorrt", backbone_only=True, output_name=...) now writes {output_name}-backbone.trt instead of {output_name}.trt, matching every other format and what export()'s own documentation already described. Without the marker a backbone engine silently overwrote a full-detector engine exported under the same name; scripts that rebuilt the engine path from output_name need the suffix added.

  • TrainConfig.multi_scale is now a MultiScale enum (rfdetr.config.MultiScale) and absorbs the removed do_random_resize_via_padding flag: "per-batch" (the new default) draws one random scale per batch in RFDETRLightningModule.on_train_batch_start, "per-sample" draws a scale per sample inside the dataset transforms and pads at collate, and "off" trains at a fixed resolution. Booleans stay accepted as input — True normalizes to "per-batch", which is exactly what the old multi_scale=True + do_random_resize_via_padding=False default did, and False to "off" — but the stored value is always the enum member and model_dump emits its string. The old do_random_resize_via_padding=True becomes multi_scale="per-sample". TrainConfig and .train(**kwargs) reject do_random_resize_via_padding, and the dataset builders normalize whatever multi_scale value the config namespace carries through MultiScale.from_value.


[1.10.1] — 2026-09-07

Fixed

  • Reduced peak CUDA memory in segmentation loss: matched boolean ground-truth masks are now sampled one image at a time on CUDA instead of concatenating a batch-wide float mask tensor. (#1437)
  • point_sample(mode="nearest") no longer falls back to a host op on MPS/XLA — routed through a backend-agnostic gather path instead of F.grid_sample. CUDA/CPU are unaffected. Measured on a Cloud TPU v6e-1 with RFDETRSegNano: aten::grid_sampler_2d host fallbacks went from 50 to 0 per 5-step fit. (#1432, issue #1058)
  • SetCriterion.loss_masks no longer reads its normalizing denominator back to the host on every call — dice_loss/sigmoid_ce_loss now accept Union[Tensor, float, int] and loss_masks passes the Tensor straight through. Side effect: dice_loss_jit/sigmoid_ce_loss_jit — reachable only through lwdetr.py's backward-compat re-exports, not part of the public API — now reject most NumPy scalar denominators (np.float64 still works, np.float32/np.int64 and similar now raise RuntimeError); the eager dice_loss/sigmoid_ce_loss functions are unaffected. (#1428, issue #1058)
  • build_trainer now selects XLAStrategy for multi-device XLA/TPU training when strategy="auto" — previously this crashed at Trainer construction (DDPStrategy built before Lightning's XLA-first auto selection could apply). Also routes single-device accelerator="auto" runs on an XLA-available host through Lightning's XLAPrecision plugin instead of a plain precision= kwarg. Keypoint models are excluded from the strategy promotion. (#1427, issue #1058)
  • XLA-marked tests now pass on real TPU hardware. (#1426, issue #1058)
  • compile=True now takes effect on CUDA with the default multi_scale=True, instead of logging a notice and training eagerly. (#1436; #1411 made compilation reachable in the first place)

[1.10.0] — 2026-09-04

Added

  • TrainConfig.pack_targets (default True) concatenates each batch's per-sample target dicts into one tensor per field before the DataLoader worker-to-main boundary, rebuilding them losslessly on the other side: a batch of 16 crosses as 9 objects, not 114, with bit-identical values. Loaders yield PackedTargets when packing is lossless, else the original tuple of dicts. (#1399)
  • TrainConfig.eval_batch_size decouples the validation/test/predict dataloaders from the training batch_size. Default None inherits batch_size; unlike batch_size it accepts no "auto". (#1378)
  • TrainConfig.best_model_metric ("map" or "mar", default "map") ranks checkpoints and early-stopping by mAR instead of mAP. (#1305)
  • Training progress bar restored/extended:
    • Restored peak GPU memory (max_mem), dropped during the PyTorch Lightning migration (#794). (#974)
    • Live free/total GPU memory (free_mem) alongside max_mem (trainer.fit() only). (#1314)
    • Restored train/lr, including per-group learning rates. (#1310)
  • deploy_to_roboflow():
    • version is now optional; when omitted, the highest existing dataset version resolves automatically via the Roboflow API. (#1116)
    • Accepts ROBOFLOW_HOME as an alias for the RF_HOME weights cache directory. (#1264)
  • Experimental, undocumented XLA/TPU training path, not announced in the release notes and not exercised by any 1.10.0 benchmark: build_trainer() routes accelerator="xla"/"tpu" through an XLAPrecision("bf16-true") plugin, with a new xla optional extra (torch_xla==2.9.*, Linux only, py3.10-3.13). (#1257, #1256, #1254)
  • Kornia GPU augmentation backend gains seven ops: ToGray, Blur, Sharpen, Equalize, CLAHE, Perspective, ShiftScaleRotate. Params Kornia cannot express are warned about, not silently dropped; HueSaturationValue remains unsupported. (#1249, #1277, #1330, #1370)
  • GPU batched linear-assignment solver (rfdetr.models._assignment) wraps torch_linear_assignment (Triton-backed), folding every decoder layer's assignment problem into one solve. SciPy's linear_sum_assignment remains the CPU/fallback path, and wherever the Triton backend cannot run (non-Linux, compute capability < 8.0, old torch) it falls back internally to that same SciPy solve. New [train]-extra dependency torch-hungarian, pinned to the 0.1.0rc0 pre-release on PyPI pending a stable 0.1.0, imported lazily so inference-only installs are unaffected. (#1368)

Changed

  • Detection and segmentation COCO evaluation now runs on hotcoco by default — a Rust COCO evaluator under MIT with numpy as its only runtime dependency, added to the train extra. Reported metrics do not change: the parity tests compare every aggregate, per-class and class-ID output of both backends for box-only and box-plus-mask evaluation and require exact equality, which they reach. Set TrainConfig.eval_backend="faster_coco_eval" to restore the previous evaluator, which remains installed and is still required — torchmetrics resolves its COCO helpers from a closed backend-name enum with no hotcoco member, so the adapter constructs it with the supported name and replaces the resolved modules. What changes is the cost of compute(), not the validation forward pass that usually dominates a validation epoch: on synthetic COCO-val-shaped state (5,000 images, 36.6k ground-truth boxes, 300 detections per image, 80 classes, eval_max_dets=500) one macOS-CPU compute() took 6.2 s before and 1.1 s after, measured against hotcoco 1.0.0. Most of that is not the evaluator: for box-only evaluation the prediction dataset is now handed to the backend as one detection array instead of the million-plus annotation dictionaries TorchMetrics materializes, which on that state builds in 0.4 s where the dictionary path the other backend still takes costs 1.8 s. Segmentation, the faster_coco_eval backend, and states without stored boxes keep the dictionary path. This is a single-machine CPU measurement on generated detections, not a trained-model or multi-hardware figure. One hotcoco behavior is a silent wrong answer rather than an error and is handled in the adapter, with a test that fails if the handling is dropped: its dataset getter returns a copy, so field-level mutation is discarded — which would leak one IoU type's annotation areas into the other's COCO size buckets, doubling bbox_map_small in the shipped regression fixture. Installing hotcoco also puts a generically-named coco console script on PATH.

  • RFDETR.predict() performance work, none of it changing detections: every entry measured byte-identical or checksum-identical against the previous path.

    • Skips the recursive eval() reassignment when the module tree is already in eval mode, saving ~0.4-0.5 ms/call on RTX 4060/L4 in the common repeated-inference case. (#1419)
    • Transfers PIL/uint8 NumPy inputs to device in their original byte storage and widens to float on-device, not on host, cutting host-to-device transfer size 4x. (#1415)
    • Converts PIL/uint8 NumPy inputs to contiguous CHW float storage in one fused allocation, not a separate dtype/layout pass. (#1390)
    • include_source_image=True converts CUDA float images to uint8 source bytes on-device before the host transfer, not on CPU. CPU tensors and unsupported CUDA dtypes (e.g. bfloat16) keep the previous path. (#1388)
    • Skips the deferred [0, 1] pixel-range scan (from #1341) for PIL/uint8 NumPy inputs, since to_tensor already guarantees that range for them; tensor and non-uint8 NumPy inputs are unaffected. (#1387)
  • Single-feature-level fast paths reuse tensors instead of re-materializing them (current Nano/Small/Medium/Large models; legacy RFDETRLargeDeprecatedConfig unaffected where noted); outputs bit-identical:

    • Eager forward pass skips rebuilding the sine position embedding, padding masks, and padded batch tensor when a batch carries no padding, tracked via NestedTensor.no_padding; position embeddings are served from a small cache in eval mode. Batches with real padding are unaffected. (#1416)
    • Deformable attention reuses its sampled tensor directly for single-level inputs instead of stack+flatten over a one-element list, mainly benefiting keypoint cross-attention. (#1385)
    • Transformer.forward reuses flattened tensors instead of torch.cat over a one-element list. (#1377)
    • Decoder's grouped self-attention reuses the regrouped query tensor as the key, not materializing the same grouping twice. (#1371)
  • Evaluation:

    • New TrainConfig.eval_base_model (default False) restores base+EMA validation comparison when only one model is evaluated (see Breaking Changes). TrainConfig.eval_ema_only is deprecated, removal in v1.13. (#1380)
    • COCO mAP computation consolidated into a new rfdetr.training.coco_map.OnePassCocoMeanAveragePrecision adapter: base and EMA share one evaluation pass, and each image's detection scores convert once, not once per detection. Narrows the torchmetrics[detection] pin to >=1.8.2,<1.9.0, which validates a TorchMetrics-internal contract this adapter relies on. (#1375, #1379)
    • Shares bbox IoU per image with a unified tie-break contract, plus C=1/no-crowd fast paths. (#1373)
    • mAP metric state kept on CPU, restricted to consumed metrics only; the train hot path is gated on eval epochs. (#1356)
    • Detection validation converts each batch's ground-truth targets once and shares the result between base and EMA mAP accumulators. Segmentation still converts twice, because per-head mask grids can differ. (#1381)
  • Segmentation postprocessing, both bit-identical to the previous output:

    • Reads each image's mask resize target once per batch, not per image, cutting CUDA syncs; same fix applied to COCOEvalCallback._convert_targets. (#1369)
    • Writes thresholded interpolation chunks directly into a preallocated buffer instead of torch.cat-ing a list. Small CUDA selections keep the prior path, for lower peak memory at shipped num_select=100 defaults. (#1374)
  • SetCriterion.loss_masks samples matched ground-truth mask labels via direct tensor indexing instead of point_sample, under size/contiguity/dtype guards; CUDA keeps the previous path. Measured 6.7-7.1x faster on a single-thread CPU microbenchmark of the full loss_masks call, labels bit-identical either way. (#1367)

  • HungarianMatcher batches host transfers instead of issuing them per problem. (#1361)

  • Oversized JPEGs, including 1080p sources, are draft-decoded while preserving draft geometry. (#1389)

  • Torch-free NumPy export kernels: bilinear resize made separable, top-k selection partitioned. (#1394, #1393)

  • Kornia GaussianBlur.sigma default changed (0.1, 2.0)(0.5, 3.0) and GaussNoise.std_range default changed (0.01, 0.05)(0.2, 0.44), 4-9x stronger, matching Albumentations' defaults. Silently changes augmentation strength for any config that omits these params on the Kornia/GPU backend (e.g. AUG_INDUSTRIAL reaches the blur default); pin explicit values if you rely on the old strength. (#1395)

  • Training skips PyTorch Lightning's pre-training sanity validation batches by default; num_sanity_val_steps restores it. Per-microbatch training-loss metrics are compacted, 17 → 9 keys on default RFDETRSmall, and compact_train_metrics=False restores per-layer keys. LR metrics emit only on optimizer updates, not every microbatch: a no-op at the new grad_accum_steps=1 default, but ~75% fewer log calls at grad_accum_steps=4, the 1.9.x default. (#1360)

Deprecated

  • rfdetr.datasets.aug_config compatibility shim now has a concrete removal target: deprecated since 1.9.0, removal in v1.12.0. Use rfdetr.datasets.aug_configs (plural) instead; constants unchanged. (#1103, #1037)
  • TrainConfig.eval_ema_only is deprecated, removal in v1.13, superseded by eval_base_model. Legacy True/False still migrate to the equivalent eval_base_model value with a FutureWarning; it still requires use_ema=True and conflicts with eval_base_model=True. (#1380)

Fixed

  • Packed targets materialize directly into per-sample device tensors instead of clone-after-move, removing a transient CUDA allocation equal to the mask field's size. (#1405)
  • Empty COCO targets keep iscrowd/area dtypes matching populated targets, enabling lossless packed-target transport for mixed empty/populated batches. (#1404)
  • Fixed compile=True aborting training on supported PyTorch versions, including 2.2. spatial_shapes is now built from Python ints under compilation instead of torch._shape_as_tensor, which Dynamo could not trace. Eager, torch.jit.trace, and the ONNX/TensorRT export path (#1155) are unaffected. (#1411)
  • Kornia CLAHE reads a scalar clip_limit as a range, matching Albumentations, and rejects the same sequences Albumentations rejects. (#1350)
  • Corrupt COCO zip downloads are retried, size validated against Content-Length, up to 3 attempts with linear backoff, instead of failing the dataset build outright. (#1306)

Breaking Changes

  • TrainConfig.grad_accum_steps now defaults to 1 (was 4), changing the default effective batch size from 16 to 4 — a training-semantics change, not just throughput. Set grad_accum_steps=4 explicitly to restore prior behavior. batch_size="auto" runs are unaffected, since the auto-batch probe overwrites grad_accum_steps. Measured 27% faster/epoch on one L4 (batch_size=16, grad_accum_steps=1 vs. the old 4/4), mAP equal within noise. (#1378)
  • Validation now evaluates one model per epoch, EMA when use_ema=True (the default) and base otherwise, instead of both, removing a full validation pass worth ~5% epoch time in one measured L4 run. Metric keys move: val/mAP_*, val/mAR, per-class val/AP/<class>, and val/loss report whichever model was evaluated, the EMA model by default, instead of always the base model — changing what a ReduceLROnPlateau scheduler, ModelCheckpoint(monitor=...), or early stopping watching those keys tracks. val/ema_* remains available for explicit EMA consumers. checkpoint_best_regular.pth is no longer written when the base model is not evaluated. Set TrainConfig.eval_base_model=True to restore the previous base+EMA comparison; use_ema=False runs are unaffected. (#1380)
  • Optimizer parameter groups are now one per distinct learning-rate/weight-decay combination instead of one per parameter (rfdetr-nano: 465 → 28 groups), letting fused/foreach AdamW batch properly. AdamW steps are bit-identical and old checkpoints auto-regroup on load, but an explicit lr_scheduler_kwargs list sized to the old per-parameter group count, e.g. LambdaLR's per-group lr_lambda, must be resized to the new group count. (#1409)
  • Dataset builders (build_roboflow_from_coco, build_roboflow_from_yolo, build_o365_raw) now require seven image-pipeline options (square_resize_div_64, segmentation_head, multi_scale, expanded_scales, do_random_resize_via_padding, patch_size, num_windows; build_o365_raw takes no segmentation_head) instead of silently substituting contradictory defaults when called with an incomplete config namespace, which could previously train with multi-scale off and the wrong crop scales without warning. Callers passing a complete TrainConfig/ModelConfig are unaffected; callers assembling a partial namespace by hand must supply every field. (#1413)
  • TrainConfig.log_per_class_metrics now defaults False (was True), so per-class AP keys are no longer emitted by default. TrainConfig.compute_val_loss now defaults "auto" (was True), so val/loss is computed only when a scheduler/callback consumes it. Set either explicitly to restore the prior unconditional behavior. (#1372)

[1.9.4] — 2026-08-24

Fixed

  • ONNX and TFLite reference inference helpers accept an explicit background_class_id: -1 preserves the existing final-background default, None retains every exported logit slot for sparse-ID COCO checkpoints, and 0 supports legacy background-first keypoint checkpoints. (#1397)
  • Fixed the TFLite reference inference helper assuming a lone rank-4 output is a segmentation mask. ONNX output names rarely survive the conversion — RF-DETR's own TFLite files arrive as StatefulPartitionedCall:N — so a keypoint export's pred_keypoints tensor was indistinguishable from a mask by name and was silently upsampled into Detections.mask. rank4_output now defaults to None, decoding only named masks; pass "masks" explicitly for a name-stripped segmentation export. (#1397)
  • Fixed the torchvision-native non-square training pipeline resampling crop-branch outputs twice. _build_train_resize_transforms(square=False) resizes each crop directly to a randomly selected target scale, matching the square and Albumentations paths. This changes the augmented pixel distribution for non-square training by avoiding the fixed 384x384 intermediate and its extra resampling step. Square training, the released default for every shipped model config, is untouched, as are validation, prediction, and export preprocessing. (#1383)
  • Fixed custom Albumentations configs treating TimeReverse as a pixel-only transform, which flipped images while leaving boxes and keypoints unchanged. TimeReverse now shares the geometric-transform and replay-based keypoint handling used by HorizontalFlip. The keypoint safety filter disables both TimeReverse and SquareSymmetry when keypoint_flip_pairs=[]; detection-only pipelines (keypoint_flip_pairs=None) retain them, and configured pairs enable their keypoint-slot swapping. SquareSymmetry already had geometric and replay handling as the alias of D4; this fix extends the no-pairs safety filter to it. The default torchvision pipeline is unchanged, as are configs already using the canonical HorizontalFlip/D4 names.
  • Fixed TFLite export failing when onnx2tf could not resolve the installed onnxsim console script from a non-activated virtual environment. onnx2tf invokes the bare onnxsim name; when that lookup raises FileNotFoundError it logs Failed to optimize the onnx file, a warning that also appears in working runs, and a stock RFDETRSmall() export then failed with RuntimeError: onnx2tf conversion failed: Output tensors of a Functional model must be the output of a TensorFlow Layer. RF-DETR now temporarily adds the running interpreter's script directory to PATH during conversion. (#1365)
  • Fixed the default torchvision-native training pipeline silently corrupting keypoint annotations when keypoint_flip_pairs is empty on a schema with genuine left/right pairs. RandomHorizontalFlip on this backend always mirrored keypoint x-coordinates when a flip was drawn, but relabeled left/right joints only if self.keypoint_flip_pairs: — with an empty list, the pydantic default and one possible outcome when automatic flip-pair inference from dataset metadata misses an asymmetric schema, affected training samples got their keypoints mirrored in position while keeping their original left/right label, with no warning. _build_torchvision_pipeline now drops the flip entirely for an empty-but-not-None keypoint_flip_pairs, logging the warning the Albumentations backend already emits via filter_keypoint_hflip_augmentations, worded for this backend's lack of an editable aug_config, matching the annotation-safety behavior that backend has had since #1122. An empty list can also legitimately mean the schema has no left/right pairs at all, e.g. a single midline keypoint; the unpatched flip was harmless there since nothing needed relabeling, but this fix disables it there too, for consistency with the Albumentations backend's contract, at the cost of a now-unavailable-by-default augmentation for that narrower case. Detection-only pipelines (keypoint_flip_pairs=None) and keypoint pipelines with real pairs are unaffected.
  • Fixed BestModelCallback treating PyTorch Lightning's pre-training sanity-check validation pass as a real epoch's result. Its EMA-checkpoint tracking and the smooth_alpha smoothing accumulator are custom bookkeeping sitting outside ModelCheckpoint's own trainer.sanity_checking guard, which the regular-checkpoint path already inherits, so a positive sanity-check score — common when starting a new run initialized with pretrain_weights from a checkpoint pretrained on a different dataset — could be written out as the permanent "best" checkpoint_best_ema.pth before a single real epoch ran, and real training could then never surpass it. This is distinct from PTL's own resume/ckpt_path restart, which PTL itself skips the sanity check for (not val_loop.restarting). (#1357, fixes #1348)

[1.9.3] — 2026-08-17

Changed

  • HungarianMatcher's compact-path safety gate computes its target-side half, the box/label finiteness checks, once per training step rather than once per matcher() call. SetCriterion.forward invokes matcher() separately for the final layer, each auxiliary decoder layer, and the encoder layer with the same targets, so the target-side precheck is precomputed once and reused across all of them, keyed on targets object identity plus pred_boxes dtype/device and num_classes; a mismatch triggers a fresh computation. Matching results are unchanged. Callers must not mutate targets in place between precompute and reuse — the identity check cannot detect that. (#1340)
  • Per-class confidence-threshold sweeps in evaluation are O(N log N), not O(T·N): one stable ascending sort per class plus np.searchsorted into precomputed suffix sums replaces a full rescan per threshold. NaN scores are explicitly masked so they never count as "above threshold". Results are unchanged. (#1339)
  • RFDETR.predict() no longer blocks the host on a per-image CUDA sync for its [0, 1] pixel-range validation. The range-check tensors are collected unsynced across all images and resolved to Python booleans once, after every image's conversion, range check, and transfer have been queued, so later images' GPU work can overlap the sync. Error-message precedence per image is unchanged. A malformed-rank input combined with include_source_image=True now raises a public ValueError with a shape message, where it previously surfaced an internal RuntimeError from permute(). (#1341)
  • Transformer.forward's two-stage query selection gathers the torch.topk-selected rows before running the bbox-delta MLP (enc_out_bbox_embed), not after: the MLP is pointwise with no cross-token mixing, so it needs at most the num_queries rows that survive selection, not every one of the sum(H*W) encoder positions. (#1334)
  • PostProcess box/mask/keypoint selection is deterministically tie-broken: torch.argsort(..., stable=True) plus a slice replaces torch.topk, so ties resolve by descending score then ascending flattened query/class index — the rule now shared with the torch-free export decoders, both sides changed together in this PR. Output ordering may differ from 1.9.2 when scores tie (same detections, different order; detections[0] may change), but ordering among equal scores was never contractual. PostProcess(num_select=<negative>) now raises ValueError at construction instead of being silently accepted. (#1320)

Fixed

  • Fixed evaluate(split="test") on YOLO-format datasets silently evaluating valid/ instead of the real test/ split. When no resolvable test split exists, evaluation falls back to valid/ with a logged warning rather than failing; a new YoloSplitUnavailableError, a FileNotFoundError subclass, drives that fallback and is catchable by callers. If a test path is declared in data.yaml but unresolvable, or the images directory exists but is empty, or the labels directory is missing, evaluation raises instead of silently relabeling the split as validation. COCO-format Roboflow exports have no such fallback and still raise FileNotFoundError; COCO and Objects365 datasets never attempt a test split. (#1329, #1343)
  • Fixed metrics.csv training history being wiped by a resumed run. build_trainer() reconstructs a fresh CSVLogger(version="") on every start, and PyTorch Lightning's _ExperimentWriter deletes any pre-existing metrics.csv the first time .experiment is accessed, removing every pre-resume row. The file is now snapshotted before that access and restored after, with the writer's column cache seeded so the next save() appends instead of overwriting. This is gated on resume being set, so reusing an output_dir for a fresh, non-resumed run still resets the file instead of appending onto an unrelated run's history. (#1325, closes #1321)
  • Fixed SegmentationHead's skip_blocks branch skipping the learned spatial_features_proj 1×1 convolution; it is now applied before computing mask logits, matching the non-skip branch. This affects the encoder-branch aux mask supervision during training only (sparse_forward, skip_blocks=True); the export path (forward_export) already applied the projection unconditionally, and the main decoder path was already projected, so predict() outputs and exported models are unchanged. Custom deployment decoders consuming sparse_forward's spatial_features dict entry must not re-apply the projection themselves, since it is now applied upstream. (#1331)
  • Fixed non-finite keypoint predictions poisoning the shared box head's gradients, in both the decoder and encoder branches. compute_l1_keypoint_loss already guarded its own inputs, but could not zero the local backward pass of a multiply feeding ref_wh, shared with the box head, letting a NaN delta propagate through 0.0 * nan == nan; deltas are now sanitized at the source with torch.nan_to_num(..., 0.0) before the reference is composed. The keypoint loss also masks out non-finite predicted keypoints and non-finite target areas rather than letting them poison the loss. Not yet covered: the matcher's own keypoint cost (compute_keypoint_matching_cost) still lacks the equivalent guard. (#1336)
  • Fixed batch_size="auto" probing ignoring AdamW's optimizer-state memory (exp_avg/exp_avg_sq); it now accounts for it via a shadow optimizer, where previously the probed batch size overshot what real training could fit, causing an out-of-memory error on the first optimizer step. A warning is logged when a non-AdamW optimizer is configured, since the estimate no longer directly applies. The search loop starts from candidate=2/lower_ok=1, not 1/0. (#1342)
  • Fixed the ONNX Runtime export benchmark ignoring the requested device: the inference session is built with providers=[("CUDAExecutionProvider", {"device_id": device})] instead of the bare provider name, which previously always bound to GPU 0 regardless of --device N. (#1346)
  • Fixed training metric plots drawing a legend only on the subplot titled "Loss"; every subplot now gets one. (#1335)
  • Fixed the ONNX and TFLite reference decoders taking a per-query argmax, which silently dropped legitimate detections whenever a query scored above threshold on more than one class; both now mirror PostProcess's multi-label selection. Both paths flatten (Q, C) scores into Q·C query/class pairs and take the top-scoring pairs before thresholding, via a shared _select_topk_multiclass helper using the same deterministic tie rule as PostProcess. The selection cap defaults to the exported model's query count; custom exports can pass an explicit value. Empty, zero, negative, and NaN inputs are handled correctly during debug logging. (#1320)
  • Fixed EMA training performing an extra averaged-model update at epoch boundaries after the final optimizer step, which let one update per epoch bypass ema_update_interval and change the EMA trajectory. (#1319)
  • Fixed model.export(format="tflite") hanging forever at the ONNX → TFLite conversion step. onnx's C extension and TensorFlow both statically link Abseil and export its symbols as weak definitions, which the dynamic loader coalesces onto whichever library loads first. The TFLite route runs a full ONNX export before reaching onnx2tf, so ONNX won that race and supplied Abseil's synchronization primitives to TensorFlow, whose executor then blocked forever in absl::Notification::WaitForNotification() while restoring the SavedModel bundle: no traceback, no error, 0% CPU, no .tflite. TensorFlow is now imported before the ONNX export (rfdetr.export._backend.preload_tensorflow_before_onnx), and a warning is logged when the calling process had already imported onnx before TensorFlow, e.g. a direct export_tflite() call, since that order cannot be repaired in-process. Importing onnx after TensorFlow is safe and does not warn. (#1322, #1323)

Breaking Changes

  • Exported artifact filenames encode precision or backend for variant-derived/default names: TFLite {stem}_float32.tflite / {stem}_float16.tflite{stem}_fp32.tflite / {stem}_fp16.tflite; ExecuTorch {variant}.pte{variant}_{backend}.pte (or {variant}_qnn_{soc}.pte); CoreML {variant}.mlpackage{variant}_fp32.mlpackage / {variant}_fp16.mlpackage; TensorRT {stem}.trt{stem}_fp16.trt / {stem}_fp32.trt. ONNX filenames are unchanged. Update scripts that hardcode or glob these artifact filenames; explicit output_name overrides are unchanged.

[1.9.2] — 2026-08-11

Changed

  • HungarianMatcher's detection-only cost matrix is built padded to each batch's max(T_i) target count and diagonal-extracted, not padded to the cross-image sum(T_i), whenever the batch's targets and predictions pass a fast eligibility check; ineligible batches fall back to the previous full-cartesian computation with identical results. The matcher runs inside the training-step criterion under torch.no_grad(), so this is a training-time, not inference-time, saving: on real COCO batches matcher time drops ~51% and peak CUDA memory ~73-76%, and the measured end-to-end training step goes from 288.364 ms to 232.457 ms on an A100. The saving scales with target-count evenness r = sum(T_i) / max(T_i), capped at the batch size, with a 1 - 1/r ceiling, so a batch where one image holds nearly all the targets (r close to 1) sees little to no improvement. The compact path also copies only the diagonal cost blocks to CPU before assignment instead of the full-size matrix, and its safety gate batches its box/label finiteness sweeps into one synchronization, not one per image. (#1297, #1281, #1312)
  • seed_all() escalates to torch.use_deterministic_algorithms(True, warn_only=True) after setting the cuDNN flags, so every op with a deterministic kernel uses it; ops without one, some scatter / grid_sample CUDA kernels, warn at execution time instead of raising, and a failure to enable determinism is caught and logged rather than propagating out of seed_all. This is user-visible as new runtime warnings and a possible slight performance cost. (#1307)
  • RFDETR.predict() pins CPU image tensors before the CUDA transfer. (#1313)
  • Two-stage query selection avoids materialising repeated top-k gather indices. (#1278)
  • Evaluation matching counts labels on the host, not the device. (#1276)
  • Keypoint decode skips redundant CUDA presence checks in postprocessing. (#1282)

Fixed

  • Fixed loading a detection checkpoint published before keypoint support warning that _kp_active_mask is a "model parameter not in checkpoint (left at random init)". The key is a deterministic schema buffer the model always rebuilds from the configured keypoint schema, empty for detection-only variants, not a learned parameter, so its absence never affected the loaded weights. Affects Nano, Small, Large (2026) and SegSmall. The filter matches the exact terminal key, so a similarly-named real parameter still warns, and an unexpected _kp_active_mask in a checkpoint still warns; the filtered key is recorded at debug level. (#1302)
  • Fixed resuming training from one of BestModelCallback's four lightweight checkpoints (checkpoint_best_regular.pth, checkpoint_best_ema.pth, checkpoint_best_total.pth, last_ema.pth) silently restarting per-callback state cold; it now restores. Those files intentionally omit optimizer/LR-scheduler state, and a warning says so explicitly, distinguishing them from checkpoints that predate callback-state persistence entirely, where best-score tracking, EMA, and early-stopping all restart cold too. Best-score restore additionally requires the original output_dir to match. (#1318)
  • Fixed training-time log calls corrupting or duplicating the completed Rich epoch progress bar when RichProgressBar(leave=True) is active. A new stream handler tracks the log target by name and re-resolves stdout/stderr on every emit, following Rich's redirect proxies instead of capturing the pre-redirect stream once at import time. (#1316)
  • Fixed an index-less torch.device("cuda") never matching an indexed device like cuda:0 in the deferred-move guard, which re-moved every parameter on every call; it is now normalised to the current device index before the comparison. (#1311)
  • Fixed the legacy query-embedding fallback warning on every load; it now warns only when it actually truncates weights. (#1301)
  • Fixed eval_ema_only runs logging no validation output at all when the base metric was empty. EMA metrics are now computed and logged in that case (val/ema_mAP_50_95, val/ema_mAP_50, val/ema_mAR, per-class AP, and a val (ema) summary table), and val/F1 is no longer silently dropped. The eval_ema_only contract is now: val/mAP_50_95 stays unpopulated, so point monitor_ema at val/ema_mAP_50_95 — a prior comment claiming otherwise has been corrected. (#1289)
  • Fixed ModelContext.reinitialize_detection_head() raising AttributeError: 'NoneType' after RFDETR.inference(inplace=True) cleared the weights; it now raises a clear RuntimeError, and does so before args.num_classes is mutated so a rejected call cannot leave the context half-updated. (#1283)
  • Fixed evaluate() not building its datamodule from the resolution-override config. (#1280)

Breaking Changes

  • COCO datasets containing an unannotated grouping category no longer spend a model output slot on it. Roboflow COCO exports prepend a synthetic root category (id 0, supercategory: "none", named after the project) that every real class then lists as its own supercategory; it carries no annotations, but previously took label index 0 and an extra class channel. CocoDetection.cat2label, the auto-detected num_classes and RFDETR._load_classes() now share one filter (rfdetr.datasets.coco.filter_parent_categories), so training such a dataset builds an N-class head instead of N+1 and every real class shifts down one label index. A parent category that owns annotations keeps its slot, and flat datasets are unaffected. Checkpoints trained before this change keep their N+1-class head — evaluating one against the same dataset now misaligns per-class metrics, firing the existing class-count UserWarning; retrain. Passing num_classes explicitly preserves the checkpoint's N+1-class head width so the weights still load, but does not restore the old label indices: CocoDetection drops the grouping category whenever remap_category_ids=True, so every real class still shifts down one slot and the pretrained head is misaligned against the new labels. The keypoint remapping path (_build_keypoint_cat2label) is unchanged, so keypoint datasets still include the grouping category. For hierarchical datasets, the train/valid/test splits now share one label mapping, always derived from the train split, so a grouping category annotated in only some splits no longer shifts that split's label indices out from under the others. (#1303)

[1.9.1] — 2026-08-03

Changed

  • PostProcess selects boxes, masks, and keypoints with index_select/expand instead of materialising a repeated int64 gather index, an allocation reaching 21–84 MiB per image for the segmentation mask head. Mask post-processing at head resolution is 2.6–3.0× faster; the output is bit-for-bit identical. (#1268)
  • RFDETR.predict() no longer upsamples segmentation masks whose scores fall below the caller's threshold before discarding them; on typical COCO images only a few of the num_select masks survive threshold=0.5. End-to-end predict() is ~20% faster at 1080p, the saving scaling with image area and neutral at 640 px; the output is unchanged. (#1265)
  • ExecuTorch export lowers the addmm operations the XNNPACK partitioner leaves undelegated back into aten.linear via AddmmToLinearTransform, which runs ~100× faster for those shapes. RFDETRNano on XNNPACK / Apple silicon is ~2.5× faster (119.9 → 48.3 ms median); outputs match the previous lowering to ~1e-4. (#1262)

Fixed

  • Fixed keypoint_flip_pairs silently disabling horizontal-flip augmentations (HorizontalFlip, Flip, D4) on detection-only datasets when a custom aug_config is supplied. AlbumentationsWrapper.from_config treats an empty keypoint_flip_pairs as "keypoint pipeline with no flip pairs defined" and drops flip transforms for annotation safety; detection pipelines must pass None instead of [] to keep flips enabled. (#1248)
  • Fixed export inference and INT8 calibration resizing through PIL's antialiased BILINEAR/BICUBIC filters, which diverge from predict() on downscale and shift exported-model confidence scores and INT8 calibration ranges. The ONNX inference, TFLite inference, INT8 TFLite calibration, and benchmark/traced-example paths now resize with RFDETR.predict()'s exact convention: bilinear, half-pixel centers, antialias=False. A shared torch-free _bilinear_resize_half_pixel NumPy kernel (rfdetr/export/_resize.py) mirrors the convention wherever torchvision is unavailable. Re-export any INT8 TFLite model to recalibrate against the corrected pixel distribution. (#1269)
  • Fixed pip install 'rfdetr[onnx]' on Python 3.10 and pip install 'rfdetr[executorch]' on Python 3.14 failing during install. Each extra previously resolved to a version (onnxruntime, executorch) shipping no wheel for that interpreter and with no source distribution to fall back on; the extras are now gated to interpreters that publish wheels. (#1267)
  • Fixed the Kornia augmentation builders (GaussianBlur, GaussNoise) rejecting scalars for range parameters; they now accept either a scalar or a (min, max) pair, matching the Albumentations path. A custom aug_config valid under Albumentations no longer raises a bare TypeError when augmentation_backend="cpu"/"auto" resolves to Kornia, i.e. Kornia installed and CUDA available. (#1255)
  • Fixed uv sync failing to create .venv; an executorch/tflite extra conflict previously blocked resolution of the development environment. (#1253)

Documentation

  • Corrected RF-DETR Keypoint Preview's parameter count (126.4 M → 40.7 M), added deployment parameter-count columns to the keypoint benchmark tables, and clarified that the new SAM 3 RF100-VL result is author-reported rather than measured in SAB. (#1258, #1261)
  • Documented ONNX Runtime raw-output decoding and expanded the LLM keypoint task/model/benchmark/API reference. (#1251, #1260)

[1.9.0] — 2026-07-27

  • Default dataset augmentations use torchvision-native transforms unless Albumentations is installed, in which case augmentation_backend="auto"/"cpu", the default, auto-selects Albumentations instead — identical user code can therefore resolve to a different resize backend, and slightly different pixel values / mAP, purely based on whether rfdetr[augment] is installed. Pass augmentation_backend="torchvision" to pin torchvision regardless of what is installed. Non-empty custom aug_config dictionaries use the optional Albumentations integration and Kornia GPU backend, both via pip install 'rfdetr[augment]'. The [train] extra no longer installs Albumentations or Kornia. See the migration guide's "Upgrade 1.8 → 1.9" section for remediation steps. (#1112)

Added

  • Native CoreML export: format="coreml" on RFDETR.export() produces a .mlpackage (mlprogram, iOS 16+) directly from torch.export, with no ONNX intermediary — distinct from ExecuTorch's format="executorch", backend="coreml" .pte path. Install with pip install 'rfdetr[coreml]' (macOS only; coremltools>=8.0,<10.0). (#1235)
  • Multi-GPU / multi-node keypoint (pose) training under DistributedDataParallel. Keypoint models (RFDETRKeypointPreview) previously raised NotImplementedError for any distributed strategy, num_nodes > 1, or devices > 1; they now train with strategy="ddp" / strategy="auto" on multiple GPUs and nodes, launched with torchrun exactly like detection models. Because keypoint models use manual optimization, gradients synchronize on every microbatch — keep grad_accum_steps=1 on multi-GPU for best throughput (grad_accum_steps > 1 is correct but performs redundant all-reduces). Sharded strategies (FSDP / DeepSpeed) remain unsupported for keypoint models and raise a clear error. See the "Keypoint / Pose models" note in docs/learn/train/advanced.md. (#1232)
  • scale_jitter: bool = True on TrainConfig — independent control for the resize → crop → resize branch (Option B) in the training resize pipeline. Disabling this branch previously required passing aug_config={}, which also disabled the entire Albumentations augmentation stack; aug_config now controls only that stack. Set scale_jitter=False to use direct resize only, with annotations near image borders never clipped.
  • AugmentationBackend.TV (augmentation_backend="torchvision") — forces the torchvision-native default pipeline. Unlike "cpu"/"auto", which auto-select the best installed backend (Albumentations > Kornia > torchvision) and can therefore resolve differently across environments, "torchvision" always resolves to torchvision regardless of what optional packages are installed. AugmentationBackend now holds only concrete, directly-usable backends (TV, ALBU, KORNIA); "cpu"/"auto" remain accepted augmentation_backend input strings, resolved lazily at dataset-build time to keep saved configs portable across environments, but are no longer enum members. AugmentationBackend.TV/.ALBU values changed from "tv"/"albu" to "torchvision"/"albumentations"; the old "tv"/"albu"/"gpu" strings are still accepted as legacy input aliases.
  • TrainConfig.optimizer (str | Callable) and optimizer_kwargs — configurable training optimizer. optimizer="adamw", the default, keeps RF-DETR's built-in fused torch.optim.AdamW path unchanged. A bare short name selects a native torch.optim optimizer only (e.g. "sgd", "adam"); any other optimizer, including third-party ones such as pytorch-optimizer (install separately), is selected by a full dotted import path ("pytorch_optimizer.Lion") or a callable / functools.partial called with the RF-DETR parameter groups. optimizer_kwargs forwards constructor arguments, ignored for callables, which bake their own arguments in. (#1006)
  • TrainConfig.lr_scheduler (str | Callable) plus lr_scheduler_kwargs, lr_scheduler_interval, and lr_scheduler_monitor — configurable LR scheduler, mirroring optimizer. lr_scheduler="step"/"cosine", the managed presets, keep RF-DETR's built-in warmup-aware schedules unchanged; any other scheduler is selected by a full dotted import path ("torch.optim.lr_scheduler.OneCycleLR") or a callable / functools.partial called with the optimizer. Explicit schedulers are built from lr_scheduler_kwargs only, with no total_steps/T_max injected, are auto-wrapped in a SequentialLR linear warmup when warmup_epochs>0, and step at lr_scheduler_interval ("step"/"epoch"). ReduceLROnPlateau is supported end-to-end: it steps once per epoch on the metric named by lr_scheduler_monitor (default "val/loss"), in both the automatic and manual (keypoint) optimization paths.

Deprecated

  • TrainConfig.lr_drop and lr_min_factor — pass them through lr_scheduler_kwargs instead ({"lr_drop": ...} / {"min_factor": ...}). Deprecated since v1.9.0, removal in v1.11.0. The fields still work meanwhile and are folded into lr_scheduler_kwargs for the managed presets with a FutureWarning; default values, e.g. on config reload, do not warn. Set with an explicit, non-managed scheduler they are inert and emit a FutureWarning.

Fixed

  • Fixed the keypoint L1-loss helper (compute_l1_keypoint_loss) returning detached new_zeros on its out-of-schema class-index guard; it now returns graph-connected zeros. A detached zero left the keypoint-head parameters without a gradient path on that batch, which desyncs DistributedDataParallel's gradient reducer across ranks (hang or "parameter did not receive grad") when the guard fires on some ranks but not others. This is a prerequisite for the multi-GPU keypoint training above.
  • Fixed non-square Albumentations training resize (aug_config set, augmentation_backend resolving to "albumentations") silently inflating every image's longest side to max_size, 1333 by default. SmallestMaxSizeLongestMaxSize always forces an exact resize in Albumentations, not a conditional cap; a new CappedLongestMaxSize internal transform only shrinks, never upscales, matching torchvision's RandomResize semantics.
  • Fixed explicit augmentation_backend="albumentations" resolving successfully without Albumentations installed and failing later, deep in dataset construction; it now raises a clear ImportError immediately.
  • Fixed RFDETR.from_checkpoint(..., trust_checkpoint=True) having no effect. It previously bypassed the safe-load check only for the checkpoint's own metadata read; model construction then silently reloaded the same file through load_pretrain_weights() with the unsafe-load default, so the flag did nothing for checkpoints that genuinely needed it and raised the same RuntimeError it was supposed to bypass. (#1239)
  • Fixed segmentation evaluation resizing ground-truth masks to each image's original resolution before comparison, a lossy round trip vs. the mask head's native grid; GT masks now resize directly to each prediction's own pixel grid, so segm mAP is computed on consistent pixel grids. (#1241)
  • Fixed pip install 'rfdetr[onnx]' (and [tflite]) hanging while building onnxsim from source on CPython 3.11/3.13 and Linux aarch64. The previous onnxsim<0.6.0 pin resolved to 0.5.0, which ships no wheels for those targets, so pip compiled onnxsim's bundled onnxruntime/onnx from source. The constraint is now onnxsim>=0.7.0, which publishes prebuilt wheels across CPython 3.10–3.13 on Linux x86_64/aarch64, Windows x86_64, and macOS arm64. (#1242)

Deprecated

  • RFDETR.optimize_for_inference() renamed to RFDETR.inference(), same signature. The old name is kept as a deprecated alias that forwards to inference() and emits a FutureWarning. Deprecated since v1.9.0, removal in v1.11.0.

Changed

  • Matched-pair IoU targets in the classification/matching losses compute via elementwise_box_iou/elementwise_generalized_box_iou, new public helpers in rfdetr.utilities.box_ops, instead of torch.diag(box_iou(...)). The old path built the full NxN pairwise IoU matrix just to read its diagonal; the new one computes only the N matched pairs directly, reducing peak GPU memory during loss calculation. Both new helpers raise ValueError on mismatched-length inputs instead of silently broadcasting. (#1245)
  • The [tensorrt] extra no longer installs pycuda, needed only for TRTInference's async benchmarking mode, which now requires the separate [tensorrt-bench] extra (pip install 'rfdetr[tensorrt-bench]'); the standard export→engine path (polygraphy, no pycuda) is unaffected. (#1246)

Security

  • RFDETR.from_checkpoint() uses safe deserialization by default (weights_only=True) instead of always running full pickle deserialization. Checkpoints containing custom Python objects beyond argparse.Namespace or types.SimpleNamespace need the new keyword-only trust_checkpoint: bool = False parameter set to True to opt into the old, unsafe behavior; resume-from-checkpoint during training honors the same flag. (#1179)
  • TensorRT export no longer shells out to the trtexec CLI — engines are built in-process through the polygraphy Python API, removing the subprocess/shell-injection surface entirely. (#853)

Removed

  • [kornia] extra removed — GPU-side augmentation installs via [augment] (pip install 'rfdetr[augment]') instead. There is no [kornia] alias extra; pip install 'rfdetr[kornia]' will fail.
  • rfdetr.util.* and rfdetr.deploy import paths, deprecated since v1.6.0 with remove_in="1.9.0". Use rfdetr.utilities.*, rfdetr.assets.coco_classes, rfdetr.training.drop_schedule, rfdetr.training.param_groups, rfdetr.visualize.data, rfdetr.models.heads.segmentation, and rfdetr.export instead.
  • rfdetr._namespace.build_namespace(model_config, train_config), deprecated since v1.7.0 with remove_in="1.9.0". Use rfdetr.models.build_model_from_config and build_criterion_from_config instead.
  • The train_config argument to load_pretrain_weights(nn_model, model_config, train_config), deprecated since v1.7.0 with remove_in="1.9.0". Call it with just (nn_model, model_config).
  • The start_epoch, do_benchmark, and callbacks keyword arguments to .train()/.evaluate(), deprecated since v1.7.0 with remove_in="1.9.0". PTL resumes automatically via resume=; use the rfdetr.export.benchmark module for benchmarking; pass PTL Callback objects directly instead of a callbacks dict.
  • TrainConfig.group_detr, TrainConfig.ia_bce_loss, TrainConfig.segmentation_head, TrainConfig.num_select, and ModelConfig.cls_loss_coef, deprecated since v1.7.0 with remove_in="1.9.0". group_detr, ia_bce_loss, segmentation_head, and num_select now live only on ModelConfig; cls_loss_coef now lives only on TrainConfig.
  • RFDETRLarge's automatic silent fallback to RFDETRLargeDeprecatedConfig on checkpoint/config incompatibility errors. Loading legacy deprecated-Large weights through RFDETRLarge now raises the original error instead of retrying; use RFDETRLargeDeprecated directly to load those checkpoints.

[1.8.3] — 2026-06-27

Added

  • optimize_for_inference(inplace=True) — new keyword-only argument on RFDETR.optimize_for_inference(); skips the deep-copy of the base model for memory-constrained inference-only deployments, ~0.5× model-weight peak memory reduction. Requires compile=False. After inplace optimization, export() raises RuntimeError and remove_optimized_model() issues a UserWarning and returns cleanly instead of silently clearing state. New RFDETR.is_optimized_inplace property returns True after a successful inplace optimization. (#1089)
  • CocoKeypointSchema.keypoint_flip_pairs and YoloKeypointSchema.keypoint_flip_pairs fields — horizontal-flip swap pairs inferred automatically from keypoint names (left/right naming convention) for COCO schemas, and from flip_idx permutation for YOLO schemas. Auto-populated by infer_coco_keypoint_schema and infer_yolo_keypoint_schema respectively. (#1164)
  • infer_coco_keypoint_schema and infer_yolo_keypoint_schema re-exported from rfdetr.datasets, previously only accessible from rfdetr.datasets._keypoint_schema. (#1164)

Changed

  • Horizontal flip detection in AlbumentationsWrapper uses Albumentations ReplayCompose replay metadata instead of heuristic bbox-center mirroring, eliminating false positives on non-flip transforms that shift box centers. Falls back to alb.Compose with a UserWarning when albumentations <1.3 is detected. (#1164)
  • Keypoint schema inference supports native COCO format (dataset_file="coco") in addition to "roboflow" and "yolo". (#1164)
  • _keypoint_schema_cache key changed from dataset_dir (string) to (dataset_file, dataset_dir) tuple, preventing cross-format cache collisions when the same directory is used with different dataset formats. (#1164)

Fixed

  • Fixed unbounded box regression producing negative or out-of-frame coordinates: predicted bounding boxes are clamped to image bounds [0, width] × [0, height] in PostProcess._postprocess_boxes(). scale_fct is also cast to boxes.dtype before multiplication, preventing dtype mismatch when boxes are float16. (#1168)
  • Fixed SegmentationTrainConfig.cls_loss_coef default of 5.0, corrected to 1.0 to restore the pre-v1.7 effective classification loss weight. The 5.0 value was present since v1.6 but dead code until the v1.7 TrainConfig ownership migration activated it, silently over-penalising classification relative to mask losses during segmentation fine-tuning. To reproduce pre-fix behaviour, pass cls_loss_coef=5.0 explicitly. (#1165)
  • Fixed KeypointTrainConfig.keypoint_nll_loss_coef, restored to 1.0 to align with the other keypoint loss terms (keypoint_l1_loss_coef, keypoint_findable_loss_coef, keypoint_visible_loss_coef). The previous default of 0.5 was set to dampen OKS@75 oscillation but under-weighted the NLL loss relative to other terms in practice. (#1165)

[1.8.2] — 2026-06-25

Added

  • YOLO pose keypoint dataset support: load Ultralytics YOLO pose datasets (.yaml with kpt_shape) directly for keypoint fine-tuning. Schema is inferred automatically via infer_yolo_keypoint_schema. (#1156)
  • is_bg_first_schema, to_active_first, to_bg_first, schemas_semantically_equal utilities in rfdetr.utilities.keypoints, re-exported from rfdetr.utilities, for schema-aware keypoint processing. (#1160)
  • amp_dtype field on TrainConfig ("auto" / "bf16" / "fp16"): pin the mixed-precision autocast dtype instead of relying on device-capability auto-detection. "auto", the default, preserves the historical behaviour — bf16-mixed on Ampere+ CUDA, 16-mixed otherwise. Invalid values degrade gracefully to "auto" with a UserWarning. (#1143)
  • Instance segmentation fine-tuning cookbook (docs/cookbooks/fine-tune_segmentation.ipynb) — end-to-end walkthrough using RFDETRSegSmall across seven diverse segmentation datasets. (#1159)
  • Inference latency benchmark cookbook (docs/cookbooks/inference-latency-benchmark.ipynb) — benchmarks CPU/GPU throughput across model sizes with reproducible measurement methodology. (#1152)

Changed

  • Default num_keypoints_per_class in RFDETRKeypointPreviewConfig changed from [0, 17] (background-first) to [17] (active-first). Legacy bg-first checkpoints auto-align on load via _kp_active_mask. (#1160)

Fixed

  • Fixed RFDETR.from_checkpoint() misreading num_classes as shape[0], i.e. num_classes + 1 including the background class, causing load_state_dict shape mismatches or a silent extra output class on every load. It now infers num_classes and num_keypoints_per_class from checkpoint weights, class_embed.weight.shape[0] - 1 and _kp_active_mask respectively. BestModelCallback._serialize_model_config is also fixed to persist the correct foreground-only num_classes. (#1158)
  • Fixed HungarianMatcher.forward() hardcoding 0.25 in the focal classification matching cost, silently ignoring any non-default focal_alpha passed to the constructor or build_matcher; it now uses the configured value. This had misaligned the bipartite matching cost with the focal classification loss in criterion.py, which correctly used self.focal_alpha. (#1147)
  • Fixed spatial_shapes in Transformer.forward() being built by torch.empty + in-place index assignment, which emitted a ScatterND feeding a shape tensor (level_start_index) that TensorRT rejected with "IScatterLayer cannot be used to compute a shape tensor". It now uses symbolic Shape ops, torch.stack of per-level torch._shape_as_tensor slices. Required to export any RF-DETR model to a TensorRT engine. (#1155)
  • Fixed keypoint model inference returning the wrong class_name field in predictions. (#1151)
  • Fixed silent train-mode inference after the first prediction: predict() re-asserts eval mode before each call for unoptimized models. (#1146)
  • Fixed TFLite inference preprocessing and mask decoder diverging from PyTorch predict() behaviour. (#1131)
  • Fixed a Python version mismatch in optional-dependency version overrides. (#1137)

[1.8.1] — 2026-06-19

Changed

  • Config path parameters, e.g. dataset_dir, output_dir, pretrain_weights, accept pathlib.Path objects in addition to strings. Paths are coerced to str automatically via the expand_paths validator. No API changes required; existing string usage unaffected. (#1124)
  • Keypoint training disables horizontal flip augmentation until keypoint flip-pair swapping is implemented. Flipping was previously applied without reordering keypoint pairs, producing incorrect labels. (#1122)
  • Training metric plots improved with optional seaborn error bands, AP@0.75 metric grouping, and custom AP metric group configuration. (#1122)

Fixed

  • Fixed the keypoint encoder in eval mode splitting num_queries queries across all group heads, because group_detr = len(self.enc_out_keypoint_embed); an if self.training else 1 guard now routes all queries through head 0. (#1135)
  • Fixed config.use_return_dict, deprecated in transformers, replaced with config.return_dict in the DINOv2 windowed attention backbone. (#1135)
  • Fixed epoch metric tables rendering incorrectly when a Rich progress bar callback is active. Tables print through the progress bar's owned Rich console, preventing cursor conflicts with active live displays. (#1128)
  • Fixed spurious keypoint fine-tuning checkpoint switches on noisy OKS metrics: selection is stabilised with smoothed (EMA) best-metric comparison, and smoothing state is correctly restored on training resume. (#1122)
  • Fixed Group DETR train-time metric evaluation crashing on non-tensor mask outputs from auxiliary decoder layers; it now evaluates only the primary query group. (#1122)
  • Fixed _detect_horizontal_flip in the Albumentations transform pipeline using not bboxes, which mishandles Albumentations 2.x where bboxes is a NumPy array, falsy even when non-empty; it now uses len(bboxes) == 0. (#1126)
  • Fixed a crash inside _log_hyperparams when tensorboard is installed alongside a NumPy-2.0-incompatible tensorflow; the TensorBoard logger is now disabled gracefully and training degrades to CSV-only logging with a clear warning. (#1123)

[1.8.0] — 2026-06-13

Added

  • RFDETRKeypointPreview — keypoint detection model variant with GroupPose-style head, covariance-based uncertainty (precision-Cholesky parameterization), and COCO keypoint AP evaluation. Public config classes: KeypointTrainConfig, RFDETRKeypointPreviewConfig (from rfdetr.config). Utility: precision_cholesky_to_pixel_covariance (from rfdetr.utilities). Schema helpers infer_coco_keypoint_schema, CocoKeypointSchema, active_keypoint_counts accessible via rfdetr.datasets._keypoint_schema. (#1099)
  • RFDETR.export_for_roboflow(output_dir) — writes a Roboflow upload bundle (weights.pt + class_names.txt) without a network call; extracted from deploy_to_roboflow, which now delegates to it. (#1086)
  • Keypoint fine-tuning cookbook (docs/cookbooks/fine-tune_keypoints.ipynb) — end-to-end walkthrough: dataset download, schema inference, KeypointTrainConfig, training metrics, and inference with covariance uncertainty. (#1104)
  • MetricKeypointOKS — reusable OKS metric facade over CocoEvaluator, exported from rfdetr.evaluation. Supports arbitrary keypoint counts, per-category OKS sigma values, DDP-safe evaluation with first-rank-wins deduplication, and an OKSKey enum (mAP, mAP@50, mAP@75, mAR) for standardised metric keys. (#1107)

Changed

  • DDP strategy enables find_unused_parameters=True for all detection, keypoint, and segmentation models under strategy='ddp' or strategy='auto' with a distributed launcher, previously segmentation only. Opt out via trainer_kwargs={"strategy": DDPStrategy(find_unused_parameters=False)}. (#1094)
  • rfdetr.datasets.aug_config module renamed to rfdetr.datasets.aug_configs (plural). Direct imports from rfdetr.datasets.aug_config must be updated; the augmentation preset constants (AUG_AGGRESSIVE, etc.) are unchanged. (#1103)

Removed

  • RFDETR.export(simplify=..., force=...) — both kwargs removed from the signature. Deprecated since v1.6.0 with remove_in="1.8.0"; both were no-ops during the deprecation window. Callers passing these args must remove them before upgrading. (#1102)

Fixed

  • Fixed RFDETR.from_checkpoint() treating num_classes loaded from the checkpoint as a user-supplied override, which silently refused fine-tuning on a dataset with a different class count — the head refused to re-initialise and trained against the stale class count. An explicit num_classes kwarg from the caller still wins over both the checkpoint value and the dataset. (#1106)
  • Fixed scale jitter missing from the non-square training crop: RandomCrop in the option_b branch replaced with RandomSizedCrop, restoring the scale-augmentation behaviour lost during the Albumentations migration. (#1088)
  • Fixed a multi-GPU validation deadlock in COCO mAP synchronization; _merge_metric_state_across_ranks is now safe across zero-batch ranks. (#1085)
  • Fixed import rfdetr failing on NumPy 2.x when a transitive dependency references the removed np.complex_ alias. (#1064)
  • Fixed the rfdetr_plus module availability check giving a false-positive hit when the package was partially installed. (#1083)
  • Fixed a spurious "Keypoint class-logit boost has N classes but detection head has M" warning on custom, non-Roboflow keypoint datasets: _align_num_classes_from_dataset now zero-pads num_keypoints_per_class when auto-adjusting num_classes beyond the schema length. (#1113)
  • Fixed loss scaling for keypoint training under gradient accumulation (accumulate_grad_batches > 1). Keypoint models use manual optimization to normalize losses by the accumulated box count across the effective batch; detection and segmentation remain on Lightning's automatic-optimization path. Optimizer-step scheduling, LR warmup/decay, and epoch-boundary flushing are correctly handled in both paths. (#1117)
  • Fixed device auto-detection assigning a CUDA device on a machine with CUDA headers but no GPU driver, which then failed at first use; it now verifies accelerator runtime availability first (PyTorch ≥ 2.4: torch.accelerator.current_accelerator; older builds: torch.cuda.is_available()). (#1111)
  • Fixed RFDETR.from_checkpoint() and related APIs silently treating an explicit num_classes as unset when its value equals the model default, e.g. 80 for COCO, which refused fine-tuning on a different class count. (#1109)
  • Fixed RFDETR.from_checkpoint() raising an error or silently loading the wrong model class for starter-like checkpoints without an explicit pretrain_weights entry; it now infers the model variant from the checkpoint filename when pretrain_weights is absent or unset-like — empty string, None, whitespace. (#1065)

[1.7.0] — 2026-04-29

Added

  • augmentation_backend field on TrainConfig ("cpu" / "auto" / "gpu"): opt-in GPU-side augmentation via Kornia, applied in RFDETRDataModule.on_after_batch_transfer once the batch is on the GPU. The CPU path is unchanged and remains the default. Install with pip install 'rfdetr[augment]'. (#1003)
  • Kornia GPU augmentation supports instance segmentation: images, boxes, and per-instance masks augmented in sync on the GPU, where augmentation_backend="gpu"/"auto" was previously ignored silently. New public helper collate_masks; build_kornia_pipeline gains with_masks: bool = False; unpack_boxes gains an optional masks_aug tensor. Note: the mask buffer is [B, N_max, H, W] float32, roughly 500 MB at B=8, N_max=50, H=W=560; use augmentation_backend="cpu" on cards with limited VRAM. (#1003, closes #997)
  • BuilderArgs — a @runtime_checkable typing.Protocol documenting the minimum attribute set consumed by build_model(), build_backbone(), build_transformer(), and build_criterion_and_postprocessors(). Enables static type-checker support for custom builder integrations. Exported from rfdetr.models. (#841)
  • build_model_from_config(model_config, train_config=None, defaults=MODEL_DEFAULTS) — config-native alternative to build_model(build_namespace(mc, tc)); accepts Pydantic config objects directly and constructs the internal namespace automatically. Exported from rfdetr.models. (#845)
  • build_criterion_from_config(model_config, train_config, defaults=MODEL_DEFAULTS) — config-native alternative to build_criterion_and_postprocessors(build_namespace(mc, tc)); returns a (SetCriterion, PostProcess) tuple. Exported from rfdetr.models. (#845)
  • ModelDefaults dataclass — exposes the 35 hardcoded architectural constants previously buried inside build_namespace(). Pass a dataclasses.replace(MODEL_DEFAULTS, ...) override to the new config-native builders to customise individual constants. Note: fields may be promoted to ModelConfig/TrainConfig in future phases. Exported from rfdetr.models. (#845)
  • MODEL_DEFAULTS — the canonical ModelDefaults singleton with production defaults. Exported from rfdetr.models. (#845)
  • RFDETR.predict(include_source_image=...) — opt-out flag, default True, to skip storing the source image in detections.metadata["source_image"]; set False to reduce memory use when the image is not needed for annotation. (#912)
  • model_name is stored in checkpoint files during training, so RFDETR.from_checkpoint() resolves the model class from the checkpoint without a caller-supplied hint. strip_checkpoint() preserves it; checkpoints without it still resolve via pretrain_weights filename matching. (#895)
  • rfdetr_version is stored in checkpoint files during training for provenance and compatibility hints. strip_checkpoint() preserves it; the key is omitted gracefully when the package version cannot be resolved, and checkpoints without it load normally. (#918)
  • notes parameter on RFDETR.train() and RFDETR.export() — embed arbitrary JSON-serialisable provenance metadata (labeller, date, class names, etc.) into best-model .pth checkpoints, under checkpoint["args"]["notes"], and ONNX files, under the "rfdetr_notes" metadata property. String values are stored verbatim; all other types are JSON-encoded. (#1025, closes #1021)
  • RF_HOME environment variable controls where pretrained weights are cached, default ~/.roboflow/models. Bare filenames passed as pretrain_weights, e.g. "rf-detr-base.pth", resolve relative to it; paths with a directory component are used as-is, parent directories created automatically. (#130)
  • Grayscale and multispectral imagery support: models accept any channel count, not just 3, with pretrained DINOv2 patch-embedding weights adapted to it at construction time and no extra dependencies. (#180, closes #75)
  • Training configuration is saved to training_config.json in the output directory after training, capturing the full TrainConfig, ModelConfig, effective training parameters, class names, and class count. (#194)
  • dinov2_registers_windowed_small backbone is available as a config option in ModelConfig.encoder. (#236)
  • rfdetr.from_checkpoint(path) — new top-level convenience function that loads a checkpoint and infers the correct model subclass automatically, without the caller specifying a class. Equivalent to RFDETR.from_checkpoint(path) but importable directly from the rfdetr package. (#664)
  • ONNX export filenames include the model variant name, e.g. rfdetr-medium.onnx, instead of the generic inference_model.onnx. Exporting multiple variants to the same directory no longer overwrites previous exports. (#910)
  • Background images, those without a matching label file, are included in YOLO detection datasets as empty-detection samples instead of being dropped; detection and segmentation both use _LazyYoloDetectionDataset. (#915)
  • TFLite export via model.export(format="tflite"). Converts through ONNX using onnx2tf; FP32 and FP16 outputs are always produced, INT8 quantization is available with a calibration image directory: model.export(format="tflite", quantization="int8", calibration_data="path/to/images/"). Requires pip install 'rfdetr[onnx,tflite]'. (#920)
  • PyTorch Lightning .ckpt files are accepted as pretrain_weights; keys are normalized from PTL format automatically (state_dict with model.-prefixed keys, hyper_parametersargs), so weight loading, class-name extraction, and compatibility checks need no manual conversion. (#951)
  • skip_best_epochs parameter for RFDETR.train() and TrainConfig: the first N epochs are excluded from best-checkpoint selection and early-stopping comparison, preventing strong pretrained weights or resumed checkpoints from locking in a suboptimal early score. (#1000, closes #789)
  • TFLite inference decodes segmentation mask outputs into sv.Detections.mask, upsampled to source size with Pillow bilinear resampling and thresholded at zero, matching PostProcess.forward. The mask tensor is detected by output name, "masks" substring, with a rank-4 shape fallback. (#1053)
  • PretrainWeightsCompatibilityWarning — new warning class emitted when a ModelConfig override, e.g. custom encoder, num_queries, or num_feature_levels, risks breaking pretrained weight loading. Importable as from rfdetr.config import PretrainWeightsCompatibilityWarning for targeted filtering. (#1017)

Changed

  • peft is no longer installed as part of the default rfdetr package; it moved to the [lora] and [train] optional extras. For LoRA fine-tuning, install with pip install 'rfdetr[lora]'. (#838)
  • Native RLE annotation support in the COCO segmentation pipeline: convert_coco_poly_to_mask explicitly detects and decodes both compressed (string counts) and uncompressed (int-list counts) RLE formats alongside existing polygon support. Malformed annotations now raise instead of being silently swallowed. (#897)
  • Pinned PyTorch Lightning to exclude known-compromised versions. (#1020)

Deprecated

  • build_namespace(model_config, train_config) — no longer used internally and deprecated in this release; use build_model_from_config, build_criterion_from_config, or _namespace_from_configs directly. Removal in v1.9; emits a DeprecationWarning on use. (#845)
  • load_pretrain_weights(nn_model, model_config, train_config) — the train_config positional argument is deprecated, removal in v1.9, and is no longer used internally. Omit it: load_pretrain_weights(nn_model, model_config). Passing a non-None value emits a DeprecationWarning. (#845)
  • TrainConfig.group_detr, TrainConfig.ia_bce_loss, TrainConfig.segmentation_head, TrainConfig.num_selectModelConfig; ModelConfig.cls_loss_coefTrainConfig. Each emits DeprecationWarning when set on the wrong config object and will be removed in v1.9. SegmentationTrainConfig users: remove the num_select override, the model config value is always used. (#841)
  • RFDETRBase — use RFDETRNano, RFDETRSmall, RFDETRMedium, or RFDETRLarge instead. Emits FutureWarning on instantiation; scheduled for removal in v2.0. (#900)
  • RFDETRSegPreview — use RFDETRSegNano, RFDETRSegSmall, RFDETRSegMedium, or RFDETRSegLarge instead. Emits FutureWarning on instantiation; scheduled for removal in v2.0. (#900)
  • rfdetr.util and rfdetr.deploy sub-modules are deprecated, removal in v1.9. A __getattr__ hook on the rfdetr package emits a clear ImportError with migration guidance when these legacy paths are accessed. (#839)

Fixed

  • Fixed TFLite export (format="tflite") producing detection scores that collapse to ~0.02, vs ~0.62 from ONNX; cause was an onnx2tf GridSample lowering bug (PINTO0309/onnx2tf#274) compounding through RF-DETR's per-decoder-layer F.grid_sample. The converter now passes onnx2tf's pseudo-GridSample replacement kwarg, logging a warning when it is absent. (#1041)
  • Fixed WindowedDinov2WithRegistersEmbeddings.forward() failing silently under -O when input spatial dimensions are not divisible by patch_size * num_windows; it now raises ValueError with a clear message identifying the divisor and actual shape. (#167)
  • Fixed _namespace.py: num_select in the builder namespace always reads from ModelConfig, where TrainConfig.num_select (default 300) silently overrode model-specific values of 100–200 for segmentation variants. (#841)
  • Fixed models/weights.py: load_pretrain_weights auto-aligns the model head when the checkpoint has fewer classes than the configured default, preventing a silent mismatch when the caller did not set num_classes. (#845)
  • Fixed models/weights.py: load_pretrain_weights slices refpoint_embed.weight and query_feat.weight per-group when reshaping checkpoint queries; the previous flat slice scrambled groups 1+ when num_queries decreased with group_detr > 1, corrupting training-resume. Inference, which reads group 0 only, was unaffected. (#1019)
  • Fixed YOLO segmentation training on large datasets hitting OS out-of-memory, caused by supervision.DetectionDataset.from_yolo(force_masks=True) eager-rasterising every image's masks at construction time. A new _LazyYoloDetectionDataset stores polygons and defers rasterisation to __getitem__, keeping RAM proportional to annotation count. (#851)
  • Fixed ONNX/TRT dynamic batch inference: the tracer baked the training batch size as a compile-time constant, so TRT engines built with smaller --minShapes failed with Reshape: reshaping failed. Six call sites in gen_encoder_output_proposals and Transformer.forward now use ONNX-symbolic equivalents, keeping the batch dimension dynamic. (#950, closes #949)
  • Fixed training failure when square_resize_div_64=False: the non-square resize pipeline did not guarantee dimensions divisible by patch_size * num_windows, raising ValueError. A PadIfNeeded step is appended after the resize pair in the train and val/test pipelines. (#991, closes #983)
  • Fixed non-square batch padding: block_size rounding is applied in the DataLoader collator as well as the transform-level PadIfNeeded, so divisibility by patch_size * num_windows survives Compose reordering and applies to custom evaluation harnesses. (#992)
  • Fixed RFDETRModelModule.on_load_checkpoint crashing with RuntimeError when resuming from a checkpoint saved at a different image resolution; DINOv2 positional embeddings are bicubic-interpolated to model_config.positional_encoding_size first. (#1002, closes #998)
  • Fixed RFDETRLarge initialization showing two conflicting ValueErrors, for patch_size=14 and patch_size=16, when the deprecated-config fallback retry also fails; the fallback re-raises the original error without chained context. (#975)
  • Fixed RFDETRModelModule.__init__ crashing with RuntimeError: size mismatch for backbone.0.encoder.encoder.embeddings.position_embeddings when training segmentation models at a custom resolution, e.g. RFDETRSegLarge(resolution=1008); the training entry path delegates to load_pretrain_weights, which interpolates the positional embeddings. (#1040, closes #1038, #1023)
  • Fixed TFLite detection scores collapsing for all queries when GridSample was used as an onnx2tf pseudo-operator; the node is rewritten to Gather-based integer-index arithmetic before conversion. Supersedes the runtime-kwarg approach in #1041. (#1054)
  • Fixed class_name lookup for pretrained COCO models: sparse COCO category IDs, 1–90 for 80 classes, made flat 0-based indexing return the wrong name. Detection uses a coco_id → class_name mapping built from COCO_CLASSES; fine-tuned models keep direct 0-based indexing. (#1051)

[1.6.5] — 2026-04-22

Breaking Changes

  • predict() stores the source image in detections.metadata["source_image"], not detections.data["source_image"], which supervision indexed per-detection and raised IndexError on. Update any code that reads detections.data["source_image"]. (#972, #968)

Fixed

  • Fixed segmentation training crash on T4 and P100 GPUs, caused by cuDNN engine selection for depthwise convolution backward on some CUDA stacks. A custom autograd.Function disables cuDNN in forward and backward. (#967)
  • Fixed ema_segm_mAP_50_95 and ema_segm_mAP_50 being computed from the base, non-EMA, metric accumulator instead of the EMA accumulator, producing misleading validation scores for segmentation models. (#980)
  • Fixed BestModelCallback losing the best EMA score on training resume, because _best_ema was not persisted in state_dict(). (#973)
  • Fixed positional_encoding_size not updating when resolution is set at construction time, e.g. RFDETRLarge(resolution=640), causing shape mismatches during forward. A model validator now auto-syncs PE size. (#956)
  • Fixed a pretrained weight loading crash with custom resolution: DINOv2 positional embeddings are bicubic-interpolated to match the target grid before load_state_dict. (#964)
  • Fixed validate_checkpoint_compatibility producing a cryptic RuntimeError on patch_size mismatch when the checkpoint lacks explicit args.patch_size; it now infers patch_size from the DINOv2 projection weight shape and raises a descriptive ValueError. (#971)
  • Fixed predict() storing detections.data["source_shape"] as a Python tuple, which raised TypeError whenever sv.Detections was iterated. The value is now an np.ndarray of shape (N, 2) and dtype int64. (#966, #963)
  • Fixed predict() emitting a misleading "class_id out of range" warning for the background/no-object class, class index num_classes. Background-class detections map data["class_name"] to "__background__" without any warning. (#970)

[1.6.4] — 2026-04-10

Changed

  • predict() includes class_name in detections.data, mapping each detection's 0-indexed class ID to its human-readable name. (#914)

Fixed

  • Fixed segmentation multi-GPU DDP training crashing with RuntimeError: It looks like your LightningModule has parameters that were not used in producing the loss, because the segmentation head's sparse_forward() leaves parameters unused on some steps: build_trainer() wraps strategy="ddp" with DDPStrategy(find_unused_parameters=True) when segmentation_head=True. Non-segmentation DDP and other strategies are unchanged. (#942, #947)
  • Fixed fused AdamW crashing under FP32 multi-GPU training with RuntimeError: params, grads, exp_avgs, and exp_avg_sqs must have same dtype, device, and layout: configure_optimizers() and clip_gradients() gate fused AdamW on the trainer's actual precision, not GPU capability, which reports BF16 support on Ampere+ even at precision="32-true". (#942, #947)
  • Fixed multi-GPU DDP training crashing in Jupyter notebooks and Kaggle: the fork-based ddp_notebook strategy is replaced with a spawn-based one, avoiding OpenMP thread pool corruption after fork(). (#928)
  • Fixed RFDETR.train(resolution=...) being silently ignored; the kwarg is applied to model_config before training begins, with validation that the value is divisible by patch_size * num_windows. (#933)
  • Fixed save_dataset_grids being silently a no-op; DatasetGridSaver is wired into the training loop, saving sample grids to {output_dir}/dataset_grids/ when enabled. Grid save failures are caught without interrupting training. (#946)
  • Fixed partial gradient-accumulation windows at the tail of training epochs: the training dataset is padded to an exact multiple of effective_batch_size * world_size, so every optimizer step uses a full gradient window. Workaround for pytorch-lightning#19987. (#937)
  • Fixed torch.export.export failing on the transformer decoder, by threading spatial_shapes_hw through all decoder layers. (#936)
  • Fixed download_pretrain_weights() overwriting fine-tuned checkpoints that share a filename with a registry model, e.g. rf-detr-nano.pth, where an MD5 mismatch silently restored the original COCO checkpoint. It now returns early whenever the file exists and redownload=False, warning when the hash differs; pass redownload=True to force a fresh download. (#935)

[1.6.3] — 2026-04-02

Changed

  • predict() stores the original image and its shape on returned sv.Detections objects — detections.data["source_image"] (NumPy array) and detections.data["source_shape"] (NumPy array of shape (N, 2), each row [height, width]) let you annotate results without loading the image separately. (#892)
  • RFDETR.train() auto-detects num_classes from the dataset directory when not explicitly set, reinitializing the detection head to the correct class count automatically. A warning is emitted when the configured value differs from the dataset count. (#893)
  • optimize_for_inference() accepts dtype as a string name, e.g. "float16", in addition to a torch.dtype object; invalid dtype inputs uniformly raise TypeError. (#899)

Fixed

  • Fixed models/lwdetr.py: reinitialize_detection_head replaces nn.Linear modules instead of mutating .data in place, keeping out_features consistent with the weight shape, so ONNX export and torch.jit.trace no longer emit stale class counts for fine-tuned models. (#904)
  • Fixed RFDETR.optimize_for_inference() leaking a CUDA context on multi-GPU setups: the deep-copy, export, and JIT-trace steps run inside torch.cuda.device(device) to pin the context to the correct device. (#899)
  • Fixed optimize_for_inference() leaving inconsistent state on failure: prior optimized state is reset and flags are committed only after a successful build/trace; temp download files use unique per-process paths to avoid parallel worker collisions.
  • Fixed deploy_to_roboflow failing with FileNotFoundError after the PyTorch Lightning migration: class_names.txt is written to the upload directory and args.class_names is populated before saving the checkpoint. (#890)

[1.6.2] — 2026-03-27

Added

  • RFDETR.predict(shape=...) — optional (height, width) tuple overrides the default square inference resolution; useful when matching a non-square ONNX export. Both dimensions must be positive integers divisible by patch_size × num_windows as determined by the model configuration. (#866)

Changed

  • ModelConfig.device and RFDETR.train(device=...) accept torch.device objects and indexed device strings such as "cuda:0". Values are normalized to canonical torch-style strings. RFDETR.train() warns when an unmapped device type is passed to PyTorch Lightning auto-detection. (#872)

Fixed

  • Fixed ONNX export ignoring an explicit patch_size argument: export() and predict() resolve patch_size from model_config by default, validate it strictly (positive integer, not bool), and enforce that (H, W) dimensions are divisible by patch_size × num_windows. (#876)
  • Fixed ONNX export for models with dynamic batch dimensions: H_.expand(N_) replaced with torch.full for Python-int spatial dims, eliminating tracer failures. (#871)

[1.6.1] — 2026-03-25

Deprecated

  • RFDETR.export(..., simplify=..., force=...) — both arguments are now no-ops and emit a DeprecationWarning. RF-DETR no longer runs ONNX simplification automatically; remove these arguments from your calls. Removal in v1.8. (#861)

Fixed

  • Fixed RFDETR.train() raising a bare ModuleNotFoundError on a missing rfdetr[train] install; it now raises an ImportError naming the fix, pip install "rfdetr[train,loggers]". (#858)
  • Fixed AUG_AGGRESSIVE preset: translate_percent (0.1, 0.1) was a degenerate range forcing Affine to always translate right/down by exactly 10%, corrected to (-0.1, 0.1). (#863)
  • Fixed the PTL training path: latest.ckpt and per-interval checkpoints (checkpoint_interval_N.ckpt) are written and restored on resume. (#847)
  • Fixed BestModelCallback and checkpoint monitor raising MisconfigurationException on non-eval epochs when eval_interval > 1; monitor key absence is handled gracefully. (#848)
  • Fixed the protobuf version constraint in the loggers extra, guarding against the TensorBoard descriptor crash (TypeError: Descriptors cannot be created directly) with protobuf ≥ 4. (#846)
  • Fixed duplicate ModelCheckpoint state keys when checkpoint_interval=1; last.ckpt is omitted in that configuration to avoid collision. (#859)

[1.6.0] — 2026-03-20

Added

  • PyTorch Lightning training building blocks: RFDETRModelModule, RFDETRDataModule, build_trainer(), and callbacks (RFDETREMACallback, COCOEvalCallback, BestModelCallback, DropPathCallback, MetricsPlotCallback) — standard PTL components, swap/subclass/extend any piece. Level 3: rfdetr fit --config CLI, zero Python required. (#757, #794)
  • Multi-GPU DDP via model.train(): strategy, devices, and num_nodes added to TrainConfig; single-GPU behaviour unchanged when omitted. (#808)
  • batch_size='auto': CUDA memory probe finds the largest safe micro-batch size, then recommends grad_accum_steps to reach a configurable effective batch target, default 16 via auto_batch_target_effective. (#814)
  • ModelContext promoted from _ModelContext to a public, exported API — inspect class_names, num_classes, and related metadata via model.context after training. (#835)
  • backbone_lora and freeze_encoder added as first-class fields in ModelConfig. (#829)
  • generate_coco_dataset(with_segmentation=True) produces COCO polygon annotations alongside bounding boxes for segmentation fine-tuning with synthetic data. (#781)
  • set_attn_implementation("eager" | "sdpa") on the DINOv2 backbone — switch attention implementation at runtime. (#760)
  • eval_max_dets, eval_interval, and log_per_class_metrics added to TrainConfig.
  • python -m rfdetr entry point alongside the rfdetr console script.
  • py.typed marker — RF-DETR is now PEP 561–compliant.

Changed

  • Breaking: Minimum transformers version bumped to >=5.1.0,<6.0.0. The DINOv2 windowed-attention backbone uses the transformers v5 API (BackboneMixin._init_transformers_backbone(), removed head_mask plumbing). Projects still on transformers v4 must pin rfdetr<1.6.0. (#760)
  • Breaking: PyPI install extras renamed — rfdetr[metrics]rfdetr[loggers], rfdetr[onnxexport]rfdetr[onnx].
  • draw_synthetic_shape returns Tuple[np.ndarray, List[float]], not np.ndarray. The second element is a flat COCO-style polygon list [x1, y1, x2, y2, …]. Any caller that did img = draw_synthetic_shape(...) must be updated to img, polygon = draw_synthetic_shape(...). (#781)
  • Albumentations version constraint broadened to >=1.4.24,<3.0.0; RandomSizedCrop configs using height/width kwargs are adapted automatically to the 2.x size=(height, width) API. (#786)
  • Current learning rate is shown in the training progress bar alongside loss. (#809)
  • supervision, pytorch_lightning, and other heavy dependencies are imported lazily, on first use, rather than at module load, reducing cold-import time in inference-only environments. (#801)

Deprecated

  • rfdetr.deploy.* — redirects to rfdetr.export.* with a DeprecationWarning. Migrate before v1.7.
  • rfdetr.util.* — redirects to rfdetr.utilities.* with a DeprecationWarning. Migrate before v1.7.

Fixed

  • Fixed a cryptic RuntimeError / tensor-size mismatch when a checkpoint is incompatible with the current model architecture; a descriptive ValueError is raised instead, covering segmentation_head mismatch and patch_size mismatch. (#810)
  • Fixed class_names not reflecting dataset labels on model.predict() after training; class names are synced from the dataset so inference always uses the correct label list. (#816)
  • Fixed detection head reinitialization overwriting fine-tuned weights when loading a checkpoint with fewer classes than the model default. The second reinitialize_detection_head call fires only in the backbone-pretrain scenario. (#815, #509)
  • Fixed grid_sample and bicubic interpolation silently falling back to CPU on MPS (Apple Silicon); both run natively on the MPS device. (#821)
  • Fixed early_stopping=False in TrainConfig being silently ignored; the setting propagates correctly. (#835)
  • Fixed an AttributeError crash in update_drop_path when the DINOv2 backbone layer structure does not match any known pattern.
  • Added warning when drop_path_rate > 0.0 is configured with a non-windowed DINOv2 backbone, where drop-path is silently ignored.
  • Fixed ValueError: matrix entries are not finite in HungarianMatcher when the cost matrix contains NaN or Inf; non-finite entries are replaced with a finite sentinel before linear_sum_assignment, warning emitted at most once per matcher instance. (#787)
  • Fixed YOLO dataset validation rejecting data.yml; both .yaml and .yml are accepted. (#777)
  • Silently dropped degenerate bounding boxes, zero width or height, before Albumentations validation instead of raising ValueError. (#825)

[1.5.2] — 2026-03-04

Added

  • Added peak GPU memory (max_mem in MB) to training and evaluation progress bars on CUDA; omitted on CPU and MPS. (#773)

Fixed

  • Fixed aug_config being silently ignored when training on YOLO-format datasets; build_roboflow_from_yolo never forwarded the value, so transforms always fell back to the default. (#774)
  • Fixed segmentation evaluation metrics not being written to results_mask.json during validation and test runs. (#772)
  • Fixed an AttributeError crash in update_drop_path when the DINOv2 backbone layer structure does not match any known pattern; _get_backbone_encoder_layers returns None for unrecognised architectures. (#762)
  • Fixed drop_path_rate not being forwarded to the DINOv2 model configuration, so stochastic depth was never applied even when explicitly set. Added a warning when drop_path_rate > 0.0 is used with a non-windowed backbone. (#762)
  • Fixed incorrect COCO hierarchy filtering that excluded parent categories from the class list. (#759)
  • Fixed evaluation metric corruption on 1-indexed Roboflow datasets, caused by a flawed contiguity check in _should_use_raw_category_ids. (#755)

[1.5.1] — 2026-02-27

Added

  • Added support for nested Albumentations containers (OneOf, Sequential) inside aug_config. (#752)

Changed

  • Migrated dataset transform pipeline to torchvision-native Compose, ToImage, and ToDtype; Normalize defaults to ImageNet mean/std. (#745)

Fixed

  • Fixed RFDETRMedium missing from the public API; __all__ contained a duplicate RFDETRSmall entry. (#748)
  • Fixed AR50_90 reporting an incorrect value in MetricsMLFlowSink, due to a wrong COCO evaluation index. (#735)
  • Fixed supercategory filtering in _load_classes for COCO datasets with flat or mixed supercategory structures. (#744)
  • Fixed a crash in geometric transforms when a sample contained zero-area or empty masks. (#727)
  • Fixed segmentation training on Colab; DepthwiseConvBlock disables cuDNN for depthwise separable convolutions. (#728)
  • Pinned onnxsim<0.6.0 to prevent pip install from hanging indefinitely. (#749)

[1.5.0] — 2026-02-23

Added

  • Added custom training augmentations via aug_config in model.train() — accepts a dict of Albumentations transforms, a built-in preset (AUG_CONSERVATIVE, AUG_AGGRESSIVE, AUG_AERIAL, AUG_INDUSTRIAL), or {} to disable. Bounding boxes and segmentation masks are transformed automatically. (#263, #702)
  • Added save_dataset_grids=True in TrainConfig to write 3×3 JPEG grids of augmented samples to output_dir before training begins. (#153)
  • Added ClearML logger: set clearml=True in TrainConfig to stream per-epoch metrics to ClearML. (#520)
  • Added MLflow logger: set mlflow=True in TrainConfig to log runs and metrics to MLflow with custom tracking URI support. (#109)
  • Added live progress bar for training and validation with structured per-epoch logs. (#204)
  • Added device field to TrainConfig for explicit device selection. (#687)
  • ModelConfig raises an error on unknown parameters, preventing silent misconfiguration. (#196)

Changed

  • Deprecated OPEN_SOURCE_MODELS constant in favour of ModelWeights enum. (#696)
  • Added MD5 checksum validation for pretrained weight downloads. (#679)

Fixed

  • Fixed Albumentations bool-mask crash during segmentation training. (#706)
  • Fixed UnboundLocalError when resuming training from a completed checkpoint. (#707)
  • Prevented corruption of checkpoint_best_total.pth via atomic checkpoint stripping. (#708)
  • Fixed PyTorch 2.9+ compatibility issue with CUDA capability detection. (#686)
  • Fixed dtype mismatch error when use_position_supervised_loss=True. (#447)
  • Fixed inconsistent return values from build_model. (#519)
  • Fixed positional_encoding_size type annotation (boolint). (#524)
  • Fixed ONNX export output_names to include masks when exporting segmentation models. (#402)
  • Fixed num_select not being updated correctly during segmentation model fine-tuning. (#399)
  • Fixed np.argwherenp.argmax misuse. (#536)
  • Fixed COCO sparse category ID remapping for non-contiguous or offset category IDs. (#712)
  • Fixed segmentation mask filtering when using aggressive augmentations. (#717)

[1.4.3] — 2026-02-16

Changed

  • Pretrained weight downloads validate against an MD5 checksum to detect corrupted files. (#679)

Fixed

  • Fixed deploy_to_roboflow failing for segmentation model exports. (#578)
  • Fixed missing info key in COCO export format. (#681)

[1.4.2] — 2026-02-12

Added

  • Added generate_coco_dataset() utility for generating synthetic COCO-format datasets with configurable class counts, split ratios, and bounding box annotations. (#617)
  • Added run_test=False to TrainConfig — skip test-split evaluation when your dataset has no test set. (#628)

Changed

  • model.predict() accepts image URLs directly, with no need to download images before inference. (#629)
  • Plus models (RFDETRXLarge, RFDETR2XLarge) are distributed as a separate rfdetr_plus package under the Roboflow Model License. (#645)

Fixed

  • Fixed segmentation ONNX export failure. (#626)

[1.4.1] — 2026-01-30

Added

  • Added native YOLO dataset format support alongside COCO. (#74)
  • Added --print-freq CLI argument to control training log frequency. (#603)

Changed

  • Pinned transformers to <5.0.0 to prevent incompatibility with the transformers v5 API. (#599)

Fixed

  • Fixed class count mismatch in train_from_config for Roboflow-uploaded datasets. (#588)
  • Improved num_classes mismatch warning messages to be actionable rather than misleading. (#261)
  • Fixed CLI crash when specifying the device argument. (#246)

[1.4.0] — 2026-01-22

Headline release introducing new pre-trained model sizes — L, XL, and 2XL for object detection, and the full N/S/M/L/XL/2XL range for instance segmentation. Also added YOLO format training support, simplified the dependency footprint by removing several heavy packages (cython, fairscale, timm, einops, and others), and fixed per-class precision/recall/F1 computation. Drops Python 3.9 support.