A comprehensive stereo matching toolbox for efficient development and research.

pip install stereo_toolbox

Dataloader Return:

• left image (color jitter if training)
• right image (color jitter and random mask if training)
• disparity ground-truth (nan if not available)
• non-occucluded mask (nan if not available)
• raw left image (not normalized)
• raw right image (not normalized)

Dataloader Return: Returns a dictionary containing the requested data types:

• ref (torch.Tensor): Reference image in CHW format, values in [0, 255].
• tgt (torch.Tensor): Target image in CHW format, values in [0, 255].
• gt_disp (torch.Tensor): Ground truth disparity map in H*W format, with 0 indicating invalid pixels.
• noc_mask (torch.Tensor): Non-occluded mask in H*W format, with Flase for occluded and True for non-occluded pixels.
• raw_ref (torch.Tensor): Unaugmented reference image in CHW format, values in [0, 255].
• raw_tgt (torch.Tensor): Unaugmented target image in CHW format, values in [0, 255].
• ref_filename (str): Filename of the reference image.
• top_pad (int): Number of pixels padded at the top during testing.
• right_pad (int): Number of pixels padded on the right during testing.

• Unless specified, the maximum search disparity for cost volume filtering methods is set to 192.
• All predictions are output as a list during training, and only the final disparity map is output during inference.
• For all iterative methods, the default training and validation iterations are set to 22 and 32, respectively.
• Due to version dependency, please additionally install timm==0.5.4 and rename it to timm_0_5_4:

  wget https://github.com/huggingface/pytorch-image-models/archive/refs/tags/v0.5.4.zip
  # unzip the zip file
  cd pytorch-image-models-0.5.4
  # replace 'timm' in 'setup.py' with 'timm_0_5_4'
  # replace all the 'import timm' and 'from timm' with 'import timm_0_5_4' and 'from timm_0_5_4', respectively
  pip install .
  

Table 1: Evaluation on SceneFlow finalpass test set.

• ^†w/o occluded mask input
• ^‡employed the foundation model (DepthAnything v2).

Table 2: Generalization evaluation on four real-world training sets. For all datasets, we report the average error (EPE), outlier rates in occluded, non-occluded, and all regions. The outlier thresholds are set to 3, 3, 2, and 1 for KITTI 2015, KITTI 2012, Middlebury Eval3, and ETH3D, respectively.

• ^†trained on extra synthetic/real data.
• ^‡employed the foundation model (DepthAnything v2).
• CroCoStereo was trained with Middlebury and ETH3D datasets.

Table 3: Inference speed (s) and memory (MB) usage. Device: NVIDIA GeForce RTX 4090.

Table 4: Generalization across different weathers. The outlier threshold is set to 3.

We sincerely thank the authors of the models and datasets mentioned above.