Skip to content

prepare_dataset.md

Latest commit

 

History

History
75 lines (57 loc) · 3.95 KB

prepare_dataset.md

File metadata and controls

75 lines (57 loc) · 3.95 KB
Raw

💾 Datasets Preparation

KITTI-360

  • Download perspective images, fisheye images, raw Velodyne scans, calibrations, and vehicle poses of KITTI-360, unzip and save them to folder data/KITTI-360.

  • Following BTS and KYN, we preprocess the images with the command below. It rectifies the fisheye views, resizes all images, and stores them in separate folders.

python data_preprocess/preprocess_kitti_360.py
  • For efficient depth evaluation, we preprocess the velodyne_points in data/KITTI-360/data_3d_raw/ directory, convert them into depth maps, and save them in data/KITTI-360/depth_gt/:
python data_preprocess/preprocess_kitti_360_data3draw.py [--save_png]
  • For efficient training, we precompute Pseudo Depths from Depth Anything V2, and save them in data/KITTI-360/depth/. Before doing this, you should download official weights, and place it in data_preprocess/pseudo_depth/checkpoints.
mkdir data_preprocess/pseudo_depth/checkpoints

wget -P data_preprocess/pseudo_depth/checkpoints https://huggingface.co/depth-anything/Depth-Anything-V2-Metric-VKITTI-Large/resolve/main/depth_anything_v2_metric_vkitti_vitl.pth

python data_preprocess/pseudo_depth/gen_pseudo_depth_kitti360.py
  • (Optional) We precompute sampling anchors using visual priors from Grounded Segment Anything, and save them in data/KITTI-360/samples/. First, you should clone the repo Grounded-SAM and set up the environment following their readme file. Then, put the scripts data_preprocess/sampling_anchor/gen_sampling_anchors_kitti360.py and snog_sampler.py in the root directory. Finally, run the following command:
python gen_sampling_anchors_kitti360.py [--save_json] [--save_mask]

You can add --save_json and --save_mask to check the semantic output from Grounded-SAM. When the json files and semantic masks are saved, you can use the following command to visualize the sampling performance of SNOG sampler:

python data_preprocess/sampling_anchor/snog_sampler.py --save_dir 'visualization/sampling_anchors'
  • We use precomputed ground truth from KYN. The data directory is set to data/KITTI-360 by default. Download and unzip the pre-computed GT occupancy maps into data/KITTI-360. Download and unzip the object labels to data/KITTI-360.

The final folder structure should look like:

ViPOcc
  └──data
      └──KITTI-360
          ├── calibration
          ├── data_poses
          ├── data_2d_raw
          │   ├── 2013_05_28_drive_0003_sync
          │   │   ├── image_00
          │   │   │    ├── data_192x640
          │   │   │    └── data_rect
          │   │   ├── image_01
          │   │   ├── image_02
          │   │   │    ├── data_192x640_0x-15
          │   │   │    └── data_rgb
          │   │   └── image_03
          │   └── ...
          ├── data_3d_raw     # not used anymore
          │   ├── 2013_05_28_drive_0003_sync
          │   └── ...
          ├── depth/          # Generated by Depth Anything V2
          ├── depth_gt/       # Generated from data_3d_raw
          ├── GT_Occ/         
          ├── Object_Label/   
          └── samples/        # Generated by SNOG sampler, optional