eAP Dataset

The largest multi-modal dataset with event cameras for autonomous driving perception.

🌐 Project Page

The eAP 3D Detection Benchmark is officially online (https://www.codabench.org/competitions/16717/#/pages-tab).

You are welcome to rank your algorithm on the benchmark!

If you use this dataset, please cite:

@misc{li2026eap,
  title         = {Toward Deep Representation Learning for Event-Enhanced
                   Visual Autonomous Perception: the eAP Dataset},
  author        = {Li, Jinghang and Li, Shichao and Lian, Qing
                   and Li, Peiliang and Chen, Xiaozhi and Zhou, Yi},
  year          = {2026},
  eprint        = {2603.16303},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2603.16303},
}

release_visualizer/          # Standalone visualization tools
    ├── requirements.txt
    ├── visualize_release_sequence.py
    └── sample_release_frames.py

Each released sequence is a self-contained directory:

$DATASET_ROOT/
└── <sequence_id>/
    ├── annotations.pkl
    ├── frames.pkl
    ├── events.h5
    └── rgb/
        └── *.png

Object-level annotations. Each record contains:

Full-frame index (includes frames with no annotations). Each record contains:

Event stream in HDF5 format. Events are accessed via a millisecond-to-index map (ms_to_idx) and an events group with fields t (timestamp µs), x, y, p (polarity). The visualizer uses a 50 ms window centered on each RGB exposure midpoint.

pip install -r release_visualizer/requirements.txt

First, list available sequence IDs in a release root:

python release_visualizer/visualize_release_sequence.py $DATASET_ROOT --list-sequences

Render a full sequence to MP4:

python release_visualizer/visualize_release_sequence.py $DATASET_ROOT \
  --sequence-id <sequence_id> \
  --output-dir ./outputs

Output: ./outputs/<sequence_id>_release_visualization.mp4
Layout: RGB + event overlay (left panel) · bird's-eye view with TTC annotations (right panel).

Pass $DATASET_ROOT/<sequence_id> directly to skip --sequence-id:

python release_visualizer/visualize_release_sequence.py $DATASET_ROOT/<sequence_id> \
  --output-dir ./outputs

Key options:

Export a small set of randomly sampled rendered frames (PNG) for quick spot-checks:

python release_visualizer/sample_release_frames.py $DATASET_ROOT \
  --sequence-id <sequence_id> \
  --output-dir ./release_checks \
  --samples-per-asset 5

Output: ./release_checks/<sequence_id>_<frame_idx>.png

Key options:

If you encounter OpenMP shared-memory errors, set thread counts explicitly:

env OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 KMP_INIT_AT_FORK=FALSE \
python release_visualizer/visualize_release_sequence.py ...