Official PyTorch implementation of "SASep: Saliency-Aware Structured Separation of Geometry and Feature for Open Set Learning on Point Clouds".

This work has been accepted to CVPR 2025.

Abstract: Recent advancements in deep learning have greatly enhanced 3D object recognition, but most models are limited to closed-set scenarios, unable to handle unknown samples in real-world applications. Open-set recognition (OSR) addresses this limitation by enabling models to both classify known classes and identify novel classes. However, current OSR methods rely on global features to differentiate known and unknown classes, treating the entire object uniformly and overlooking the varying semantic importance of its different parts. To address this gap, we propose Salience-Aware Structured Separation (SASep), which includes (i) a tunable semantic decomposition (TSD) module to semantically decompose objects into important and unimportant parts, (ii) a geometric synthesis strategy (GSS) to generate pseudo-unknown objects by combining these unimportant parts, and (iii) a synth-aided margin separation (SMS) module to enhance feature-level separation by expanding the feature distributions between classes. Together, these components improve both geometric and feature representations, enhancing the model's ability to effectively distinguish known and unknown classes. Experimental results show that SASep achieves superior performance in 3D OSR, outperforming existing state-of-the-art methods.

SASep/
├─ classifiers/          # training scripts for SN/ModelNet/ScanObjectNN
├─ datasets/             # dataset loaders and splits
├─ models/               # backbones, heads, and loss layers
├─ cfgs/                 # yaml configs for different backbones
├─ utils/                # training utilities, metrics, CAM helpers
├─ supcon/               # contrastive training (optional)
├─ 3D_OS_release_data/   # dataset root (downloaded by script)
└─ outputs/              # checkpoints and logs

• Ubuntu: 18.04.
• Python: 3.9.
• CUDA: 11.7.
• PyTorch: 2.0.1.

conda create -n sasep python=3.9
conda activate sasep
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia
conda install timm==0.5.4 wandb==0.12.16 tqdm==4.66.5 h5py==3.6.0 protobuf==3.20.1 msgpack-numpy==0.4.8 ninja==1.11 scikit-learn==1.5.1 mkl==2024.0 numpy==1.24.4
conda install -c frgfm torchcam==0.4.0

pip install open3d==0.18.0
pip install "git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
pip install "https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl"

We use the script provided by 3DOS to prepare the dataset.

chmod +x download_data.sh
./download_data.sh

This section describes the full reproduction pipeline used in the paper: baseline closed-set training -> CAM generation -> SASep training -> evaluation.

All examples below use ShapeNetCore (SN1). For SN2/SN3, replace --src SN1.

This trains a standard classifier without TSD/GSS/SMS.

python -m torch.distributed.launch --nproc_per_node <NUM_GPUS> --master_port <PORT> \
  classifiers/trainer_ddp_cla.py \
  --config cfgs/dgcnn-cla.yaml \
  --exp_name DGCNN_cosine_SN1_baseline \
  --src SN1 \
  --loss cosine \
  --batch_size 16 \
  --baseline

The checkpoint will be saved under: outputs/DGCNN_cosine_SN1_baseline/models/

CAM is computed from a trained baseline checkpoint and saved to: 3D_OS_release_data/sncore_fps_4096_cam/SN1/cam.pth.

python -m torch.distributed.launch --nproc_per_node <NUM_GPUS> --master_port <PORT> \
  classifiers/trainer_ddp_cla.py \
  --script_mode cam \
  --config cfgs/dgcnn-cla.yaml \
  --exp_name DGCNN_cosine_SN1_cam \
  --src SN1 \
  --loss cosine \
  --batch_size 16 \
  --ckpt_path <BASELINE_CKPT>

Do not pass --baseline in this stage.

python -m torch.distributed.launch --nproc_per_node <NUM_GPUS> --master_port <PORT> \
  classifiers/trainer_ddp_cla.py \
  --config cfgs/dgcnn-cla.yaml \
  --exp_name DGCNN_cosine_SN1_sasep \
  --src SN1 \
  --loss cosine \
  --batch_size 12

Pick the best checkpoint (e.g. model_best_auroc_ep*.pth).

python -m torch.distributed.launch --nproc_per_node <NUM_GPUS> --master_port <PORT> \
  classifiers/trainer_ddp_cla.py \
  --script_mode eval \
  --config cfgs/dgcnn-cla.yaml \
  --src SN1 \
  --loss cosine \
  --ckpt_path <BEST_CKPT>

• ModelNet40 (Synth -> Real): use classifiers/trainer_ddp_cla_md.py
• ScanObjectNN (Real -> Real): use classifiers/trainer_ddp_cla_r2r.py

PN2 uses the same commands above; just switch --config to cfgs/pn2-msg.yaml.

If you find this work useful, please consider citing:

@InProceedings{Xu_2025_CVPR,
    author    = {Xu, Jinfeng and Li, Xianzhi and Tang, Yuan and Han, Xu and Yu, Qiao and Hao, Yixue and Hu, Long and Chen, Min},
    title     = {SASep: Saliency-Aware Structured Separation of Geometry and Feature for Open Set Learning on Point Clouds},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {27295-27304}
}

This project builds upon prior work, including:

• antoalli/3D_OS
• frgfm/torch-cam

We sincerely thank the authors and maintainers of these projects.

If you have any questions, please contact jinfengxu.edu@gmail.com.