TBDTracker: Tracking-by-Detection Trackers for Multiple Object Tracking

Notice

🎉🎉🎉Excited to share that our paper "LAKALMANTRACKER: ROBUST LEARNING-AIDED KALMAN FILTERING FOR MULTI-OBJECT TRACKING" has been accepted to ICASSP 2026! 🎉🎉🎉 We will upload all the code once the paper has been accepted.

🎉🎉🎉Excited to share that our paper "Motion Estimation for Multi-Object Tracking using KalmanNet with Semantic-Independent Encoding" has been accepted to Information Fusion! 🎉🎉🎉

We are currently organizing the code and uploading it in stages.

📄 Documentation

Install

If you want to use Learning-aided Kalman filtering (LAKF), you need to install FilterNet FilterNet.

Otherwise, you only need to install the required dependencies:

pip install -e .

Usage

For convenience, we decouple detection from tracking. This allows the detector to run only once, enabling multiple trackers to reuse the same detection results. This approach reduces detector inference time while ensuring consistent inputs, which benefits researchers focusing solely on tracker performance. Of course, detection, tracking, and evaluation can also be performed in a single pipeline. Consequently, the workflow can be organized into the following configurations:

End-to-end: Detection → Tracking → Evaluation
Two-stage: (Detection + Tracking) → Evaluation
Modular: Detection (precomputed) → (Tracking + Evaluation) We consider the third configuration to be best practice: detections are precomputed and saved to a specified path, then used jointly by tracking and evaluation. This avoids the overhead of repeated detection inference while maintaining flexibility across different trackers.

Preprocess

Convert the dataset format to the required format

We default to using the YOLO format, so we need to convert the standard MOT format to YOLO format.

Convert the train dataset to YOLO format:

 python tools/convert_StandardMOT_to_yolo.py --data_root /data/MOT_Datasets/DanceTrack --split train --generate_imgs

Convert the val dataset to YOLO format:

 python tools/convert_StandardMOT_to_yolo.py --data_root /data/MOT_Datasets/DanceTrack --split val --generate_imgs

If half the train dataset:

python tools/half_dataset.py --source /data/MOT_Datasets/DanceTrack/train copy

The half-dataset is stored in /data/MOT_Datasets/DanceTrack-tr50-val50. Then perform the preceding format conversion, but note that --data_root needs to be modified to /data/MOT_Datasets/DanceTrack-tr50-val50.

✨ Best Practices: Detection, Tracking + Evaluation

1. Detect

Modify the DATASET_ROOT parameter in ./configs/data_cfg/dancetrack-val.yaml to your local directory path.

Then run

sh ./scripts/dance_detect.sh

Detection results are stored in ./results_detected/dance/val.

2. Track + Evaluation

Modify the data_root parameter in ./configs/eval_cfg/DanceTrack.yaml to your local directory path (same to DATASET_ROOT in ./configs/data_cfg/dancetrack-val.yaml).

Modify the DETECTED_FOLDER parameter in ./scripts/track_from_fold_dance_sort_kf.sh to detection results path ./results_detected/dance/val.

Modify the tracker parameter in ./configs/tracker_cfg/sort.py to the tracker you want to use.

Then run

sh ./scripts/track_from_fold_dance_sort.sh

You will receive the following outputs

CustomDataset ./results_tracked/dancetrack-val/sort/fromdet-KF/val
TrackEval COMBINED Metrics (All Sequences Merged):CustomDataset-./results_tracked/dancetrack-val/sort/fromdet-KF/val
+---------+--------+--------+--------+--------+--------+--------+
|  Class  |   HOTA |   DetA |   AssA |   MOTA |   IDF1 |   IDSW |
+=========+========+========+========+========+========+========+
|  valid  | 53.072 | 76.447 | 36.998 | 91.964 | 56.685 |   1693 |
+---------+--------+--------+--------+--------+--------+--------+
Eval results saved to: ./results_tracked/dancetrack-val/sort/fromdet-KF/val/trackeval
Evaluation done
Tracked results saved to ./results_tracked/dancetrack-val/sort/fromdet-KF/val
Eval track from ./results_tracked/dancetrack-val/sort/fromdet-KF/val

Tracking results and evaluation results are stored in ./results_tracked/dancetrack-val/sort/fromdet_KF.

Only Evaluation

Modify the parameters in ./configs/eval_cfg/DanceTrack.yaml:

data_root: Same as `DATASET_ROOT` in `./configs/data_cfg/dancetrack-val.yaml`
trackers_folder: The tracked results folder like  `./results_tracked/dancetrack-val/sort/fromdet_KF`

Then run

sh ./scripts/eval.sh

The evaluation results are stored in the tracked results folder ./results_tracked/dancetrack-val/sort/fromdet_KF/trackeval

Benchmark

Notes:The learning-aided Kalman filter (LAKF), such as KalmanNet(KNet), Split-KalmanNet(SKNet), and Semantic-Independent-KalmanNet(SIKNet), were trained on a semi-synthetic dataset, then integrated into the Tracker as an replacement to KF. Therefore, the noise parameters in the semi-synthetic dataset affect performance. Consequently, different versions of HOTA exist, as they originate from two distinct papers.

DanceTrack

NOTES: The evaluation was conducted on the DanceTrack validation set. Oracle detections. The semi-synthetic dataset used to train LAKF was constructed from the first half of the trajectories in the DanceTrack training set.

Tracker	Status	Motion Model	HOTA↑	AssA↑	MOTA↑	IDF1↑	IDSW
botsort	✅
strongsort	✅
bytetrack	✅	KF	49.95	34.80	90.41	56.22	1738
bytetrack	✅	KNet	54.60	38.88	92.10	56.22	1738
bytetrack	✅	SKNet	50.94	35.74	89.99	54.88	1619
bytetrack	✅	SIKNet	56.19	39.97	92.37	57.87	1427
ocsort	✅
imprassoc	✅
deepocsort	✅
hybridsort	❌

Tracker	Motion Model	Status	HOTA↑	DetA↑	AssA↑	MOTA↑	IDF1↑	IDSw↓
SORT	Kalman filter (Original)	✅	53.07	76.45	37.00	91.96	56.69	1693
	KNet	✅	55.92	79.29	39.61	91.90	57.19	1677
	SKNet	✅	55.38	79.27	38.87	91.91	55.84	1670
	SIKNet	✅	57.88	81.00	41.52	92.00	57.35	1680
	PKNet
ByteTrack	Kalman filter (Original)	✅	49.95	71.95	34.80	90.41	56.22	1738
	KNet	✅	56.04	78.56	40.14	92.27	59.76	1424
	SKNet	✅	56.15	78.29	40.45	92.06	58.69	1485
	SIKNet	✅	58.42	79.62	43.03	92.24	59.67	1452
	PKNet	✅
OC-SORT	Kalman filter (Original)	✅	52.04	80.55	33.76	91.70	51.53	2308

SoccerNet

NOTES: The evaluation was conducted on the SoccerNet testing set. Oracle detections. The semi-synthetic dataset used to train LAKF was constructed from the first half of the trajectories in the SoccerNet training set.

Tracker	Status	Motion Model	HOTA↑	AssA ↑	MOTA↑	IDF1↑	IDSW
botsort	✅
strongsort	✅
bytetrack	✅	KF	72.30	62.48	94.62	75.58	5054
bytetrack	✅	KNet	75.49	66.67	94.71	77.71	4354
bytetrack	✅	SKNet	73.82	65.14	94.24	77.43	4171
bytetrack	✅	SIKNet	76.17	67.45	95.43	77.81	3844
ocsort	✅
imprassoc	✅
deepocsort	✅
hybridsort	❌

Tracker	Motion Model	Status	HOTA↑	DetA↑	AssA↑	MOTA↑	IDF1↑	IDSw↓
SORT	Kalman filter (Original)	✅	71.65	86.78	59.26	93.40	70.91	9294
	KNet	✅	76.83	88.41	66.85	94.15	76.02	7418
	SKNet	✅	74.78	90.07	62.14	94.28	72.12	7147
	SIKNet	✅	79.02	91.07	68.61	94.61	76.70	6270
	PKNet
ByteTrack	Kalman filter (Original)	✅	72.30	83.44	62.48	94.62	75.58	5054
	KNet	✅	78.39	88.49	69.54	95.16	78.54	3902
	SKNet	✅	77.90	90.57	67.05	95.92	76.52	3633
	SIKNet	✅	80.83	91.51	71.44	95.82	79.05	3378
	PKNet	✅
OC-SORT	Kalman filter (Original)	✅	70.07	93.10	52.73	90.67	62.94	15405

MIATrack

Maneuver Infrared Anti-UAV Tracking Dataset (MIATrack) Detections from YOLO26m. Datasets from (OpenDataLab).

Tracker	Motion Model	Status	HOTA↑	DetA↑	AssA↑	MOTA↑	IDF1↑	IDSw↓
SORT	Kalman filter (Original)	✅	33.546	35.849	32.223	46.501	39.788	754
ByteTrack	Kalman filter (Original)	✅	40.921	43.220	39.688	58.557	49.901	210
OC-SORT	Kalman filter (Original)	✅	22.573	32.157	18.042	35.318	23.666	2390
SparseTrack	Kalman filter (Original)	✅	31.912	38.559	26.684	46.455	37.236	2055
ImprAsso	Kalman filter (Original)	✅	44.535	55.747	36.439	68.856	50.732	2659
HybridSORT	Kalman filter (Original)	✅	42.040	48.493	38.810	65.929	49.629	282

Citation

If you find this repo useful, please cite our papers.

@INPROCEEDINGS{11463682,
  author={Song, Jian and Mei, Wei and Xu, Yunfeng and Fu, Qiang and Kou, Renke and Bu, Lina and Long, Yucheng},
  booktitle={ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
  title={LAKALMANTracker: Robust Learning-Aided Kalman Filtering for Multi-Object Tracking}, 
  year={2026},
  volume={},
  number={},
  pages={3821-3825},
  keywords={Feeds;Antennas;Kalman filters;Filters;Videos;Protocols;Video equipment;HTTP;Bot (Internet);Over-the-top media services;Model-based deep learning;Multi-object tracking;Kalman filter;KalmanNet;Motion Estimation},
  doi={10.1109/ICASSP55912.2026.11463682}}

@misc{song2025motionestimationmultiobjecttracking,
      title={Motion Estimation for Multi-Object Tracking using KalmanNet with Semantic-Independent Encoding},
      author={Jian Song and Wei Mei and Yunfeng Xu and Qiang Fu and Renke Kou and Lina Bu and Yucheng Long},
      year={2025},
      eprint={2509.11323},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2509.11323},
}

Acknowledgement

The structure of this repository and much of the code is thanks to the authors of the following repositories.

Yolov7-tracker : A simple multi-object tracker based on YOLO.
BoxMOT : A multi-object tracker for computer vision.

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
configs		configs
scripts		scripts
tools		tools
tracker		tracker
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
README.md		README.md
detect.py		detect.py
pyproject.toml		pyproject.toml
run_custom_dataset_eval.py		run_custom_dataset_eval.py
track.py		track.py

Navigation Menu

Repository files navigation

TBDTracker: Tracking-by-Detection Trackers for Multiple Object Tracking

Notice

📄 Documentation

Preprocess

Convert the dataset format to the required format

✨ Best Practices: Detection, Tracking + Evaluation

1. Detect

2. Track + Evaluation

Only Evaluation

Benchmark

DanceTrack

SoccerNet

MIATrack

Citation

Acknowledgement

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

TBDTracker: Tracking-by-Detection Trackers for Multiple Object Tracking

Notice

📄 Documentation

Preprocess

Convert the dataset format to the required format

✨ Best Practices: Detection, Tracking + Evaluation

1. Detect

2. Track + Evaluation

Only Evaluation

Benchmark

DanceTrack

SoccerNet

MIATrack

Citation

Acknowledgement

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages