S²AM3D: Scale-controllable Part Segmentation of 3D Point Clouds

Part-level point cloud segmentation has recently attracted significant attention in 3D computer vision. Nevertheless, existing research is constrained by two major challenges: native 3D models lack fine-grained part understanding while employing 2D priors leads to inconsistent predictions. To address these challenges, we propose S²AM3D, which incorporates 2D segmentation priors with 3D consistent supervision. We design a point-consistent part encoder that aggregates multi-view information while respecting the 3D consistency of individual points. The decoder architecture introduces a controllable part decoder with a learnable weight generation module, facilitating flexible scaling of segmentation granularity. Extensive experiments demonstrate that S²AM3D achieves leading performance across multiple evaluation settings, exhibiting exceptional robustness and controllability when handling complex structures.

Download the pretrained models from Hugging Face and place them in the ckpt/ folder:

mkdir -p ckpt
# Download the following files and place them in ckpt/
# - Encoder.ckpt
# - S2AM3D_decoder.pt

Your directory structure should look like:

S2AM3D/
├── ckpt/
│   ├── Encoder.ckpt
│   └── S2AM3D_decoder.pt
├── decoder/
├── encoder/
├── demo/
└── ...

conda env create -f environment.yml
conda activate s2am3d

# Create conda environment
conda create -n s2am3d python=3.10
conda activate s2am3d

# Install CUDA
conda install nvidia/label/cuda-12.4.0::cuda

# Install PyTorch
pip install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 --index-url https://download.pytorch.org/whl/cu124

# Install core dependencies
pip install psutil
pip install lightning==2.2 h5py yacs trimesh scikit-image loguru boto3
pip install omegaconf viser

# Install additional packages
pip install mesh2sdf tetgen pymeshlab plyfile einops libigl polyscope potpourri3d simple_parsing arrgh open3d

# Install torch-scatter
pip install torch-scatter -f https://data.pyg.org/whl/torch-2.4.0+cu124.html

# Install system dependencies (Ubuntu/Debian)
apt install libx11-6 libgl1 libxrender1

# Install VTK
pip install vtk

Run the interactive segmentation demo:

cd decoder
bash run_interactive_demo.sh

After the server starts, open your browser and navigate to:

http://localhost:8080

Using an automated data processing pipeline, we collect a dataset of over 100,000 point cloud instances spanning 400 categories, annotated with approximately 1.2 million fine-grained part labels at three granularity levels. This represents the most comprehensive dataset for part-level point cloud segmentation to date.

Download the dataset from Hugging Face Datasets.

• Release training code
• Support more input formats
• Release data preprocessing scripts

If you find this work useful, please consider citing:

@InProceedings{ su2026s2am3d,
    author    = {Su, Han and Huang, Tianyu and Wan, Zichen and Wu, Xiaohe and Zuo, Wangmeng},
    title     = {{S$^2$AM3D}: Scale-controllable Part Segmentation of 3D Point Clouds},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2026},
    pages     = {14357--14366}
}

This project is built upon and inspired by the following excellent works:

• PartField - Neural implicit representation for part segmentation
• OpenShape - Open-vocabulary 3D shape understanding
• Segment Anything - Foundation model for image segmentation

We thank the authors for their outstanding contributions to the community.