The official code for our paper: Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting.

This work has been accepted by CVPR 2025 🎉

Jingyi Xu#, Xieyuanli Chen#, Junyi Ma, Jiawei Huang, Jintao Xu, Yue Wang, Ling Pei*.

If you use EfficientOCF in an academic work, please cite our paper:

@inproceedings{xu2025cvpr,
	author = {Jingyi Xu and Xieyuanli Chen and Junyi Ma and Jiawei Huang and Jintao Xu and Yue Wang and Ling Pei},
	title = {{Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting}},
	booktitle = {Proc.~of the IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)},
	year = 2025
}

conda create -n efficientocf python=3.7 -y
conda activate efficientocf

conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge

conda install -c omgarcia gcc-6

pip install mmcv-full==1.4.0
pip install mmdet==2.14.0
pip install mmsegmentation==0.14.1

git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
git checkout v0.17.1 # Other versions may not be compatible.
python setup.py install

pip install timm
pip install open3d-python
pip install PyMCubes
pip install spconv-cu113
pip install fvcore
pip install setuptools==59.5.0

pip install lyft_dataset_sdk # for lyft dataset

git clone git@github.com:BIT-XJY/EfficientOCF.git
cd EfficientOCF

• Please link your nuScenes V1.0 full dataset to the data folder.
• nuScenes-Occupancy, nuscenes_occ_infos_train.pkl, and nuscenes_occ_infos_val.pkl are also provided by the previous work.
• test_ids are also predefined for nuScenes dataset.

• Please link your Lyft dataset to the data folder.
• The required folders are listed below:

EfficientOCF
├── data/
│   ├── nuscenes/
│   │   ├── maps/
│   │   ├── samples/
│   │   ├── sweeps/
│   │   ├── lidarseg/
│   │   ├── v1.0-test/
│   │   ├── v1.0-trainval/
│   │   ├── nuscenes_occ_infos_train.pkl
│   │   ├── nuscenes_occ_infos_val.pkl
│   ├── nuScenes-Occupancy/
│   ├── lyft/
│   │   ├── maps/
│   │   ├── train_data/
│   │   ├── images/   # from train images, containing xxx.jpeg
│   ├── efficientocf
│   │   ├── GMO/
│   │   │   ├── ...
│   │   ├── GMO_lyft/
│   │   │   ├── ...
│   │   ├── test_ids/

Alternatively, you could manually modify the path parameters in the config files instead of using the default data structure, which are also listed here:

occ_path = "./data/nuScenes-Occupancy"
depth_gt_path = './data/depth_gt'
train_ann_file = "./data/nuscenes/nuscenes_occ_infos_train.pkl"
val_ann_file = "./data/nuscenes/nuscenes_occ_infos_val.pkl"
ocf_dataset_path = "./data/efficientocf/"
nusc_root = './data/nuscenes/'

We directly integrate the EfficientOCF dataset generation pipeline into the dataloader, so you can directly run training or evaluate scripts and just wait

For the nuScenes or nuScenes-Occupancy datasets, please run

bash run.sh ./projects/configs/baselines/EfficientOCF_V1.1.py 8

For the Lyft dataset, please run

bash run.sh ./projects/configs/baselines/EfficientOCF_V1.1_lyft.py 8

If you only want to test the performance of occupancy prediction for the present frame (current observation), please set test_present=True in the config files. Otherwise, forecasting performance on the future interval is evaluated.

bash run_eval.sh $PATH_TO_CFG $PATH_TO_CKPT $GPU_NUM
# e.g. bash run_eval.sh ./projects/configs/baselines/EfficientOCF_V1.1.py ./work_dirs/EfficientOCF_V1.1/epoch_15.pth  8

Please set save_pred and save_path in the config files once saving prediction results are needed.

Here is some basic information and key parameters for EfficientOCF.

Our proposed EfficientOCF can still perform well while being trained with partial data. Please try to decrease train_capacity if you want to explore more details with sparser supervision signals.

In addition, please make sure that n_future_frames_plus <= time_receptive_field + n_future_frames because n_future_frames_plus means the real prediction number. We estimate more frames including the past ones rather than only n_future_frames.

We thank the fantastic works Cam4DOcc, OpenOccupancy, PowerBEV, and FIERY for their pioneer code release, which provide codebase for this benchmark.