Shift the Lens: Environment-Aware Unsupervised Camouflaged Object Detection

CVPR 2025

NKU & PolyU

Abstract

Camouflaged Object Detection (COD) seeks to distinguish objects from their highly similar backgrounds. Existing work has essentially focused on isolating camouflaged objects from the environment, demonstrating ever-improving performance but at the cost of extensive annotations and complex optimizations. In this paper, we diverge from this paradigm and shift the lens to isolating the salient environment from the camouflaged object. We introduce EASE, an Environment-Aware unSupErvised COD framework that identifies the environment by referencing an environment prototype library and detects camouflaged objects by inverting the retrieved environmental features. Specifically, our approach (DiffPro) uses large multimodal models, diffusion models, and vision-foundation models to construct the environment prototype library. To retrieve environments from the library and refrain from confusing foreground and background, we incorporate three retrieval schemes: Kernel Density Estimation-based Adaptive Threshold (KDE-AT), Global-to-Local pixel-level retrieval (G2L), and Self-Retrieval (SR). Our experiments demonstrate significant improvements over current unsupervised methods, with EASE achieving an average gain of over 10% on the COD10K dataset. When integrated with SAM, EASE surpasses prompt-based segmentation approaches and performs competitively with state-of-the-art fully-supervised methods.

1. Create the Virtual Environment

conda create -n EASE python=3.8
conda activate EASE
pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
git clone https://github.com/xiaohainku/EASE.git
cd EASE
pip install -r requirement.txt

2. Install FAISS package for efficient retrieval

Please refer to Installing FAISS for more details.

Install gpu version of faiss:

conda install -c pytorch -c nvidia faiss-gpu=1.7.3

If you have difficulty in installing the gpu version of faiss, try to install the cpu verison:

pip install faiss-cpu==1.7.4

If you have installed the cpu version, remember to change the "--faiss_device" argument in ret-unsupervised.py and ret-prompt-based-seg.py from "cuda" to "cpu".

The predicted maps, test datasets (CHAMELEON/CAMO/COD10K/NC4K) and environment images generated by Stable Diffusion are available at Google | Quark | Baidu.

EASE consists of two components: DiffPro and Retrieval.

Please note that DiffPro requires a large amount of GPU memory (48G is recommended), whereas Retrieval only requires 12G of GPU memory to work. If you want to quickly reproduce our results, we recommend using our pre-generated images from Stable Diffusion to generate prototypes. The pre-generated images could be found in Prepare datasets.

1. Generate environment categories through LLaVA-1.5

   • (Recommend) You can directly use the environment categories we provide in envir-cate.yaml

   • Or you can craft your own environment category set based on gen-envir-cate.py

     To download LLaVA checkpoint from huggingface, we recommend the following scripts:

     # export HF_ENDPOINT=https://hf-mirror.com # If you are in China mainland, try using this mirror site
     # export HF_HUB_ENABLE_HF_TRANSFER=1  # cancel comments for faster download, but stability is not guaranteed.
     huggingface-cli download --resume-download llava-hf/llava-1.5-7b-hf --local-dir path-to-save-checkpoint --local-dir-use-symlinks False

2. Generate environment images through Stable Diffusion V1-5

   • (Recommend) You can directly use the environment images we provide in Prepare datasets

   • Or you can generate your own environment images based on gen-envir-img.py

     To download Stable Diffusion v1-5 checkpoint from huggingface, we recommend the following scripts.:

     # export HF_ENDPOINT=https://hf-mirror.com # If you are in China mainland, try using this mirror site
     # export HF_HUB_ENABLE_HF_TRANSFER=1  # cancel comments for faster download, but stability is not guaranteed
     huggingface-cli download --resume-download stable-diffusion-v1-5/stable-diffusion-v1-5 --local-dir path-to-save-checkpoint --local-dir-use-symlinks False

3. Generate environment prototypes

   python gen-envir-proto.py --category_path envir-cate.yaml --sd_img_path path-of-images-generated-by-stable-diffusion --proto_save_path ./prototype

1. Unsupervised setting

   python ret-unsupervised.py --test_dataset_path your-test-dataset-path --mask_save_path ./pred/unsupervised

2. Prompt-based segmentation

   Prepare HQ-SAM-ViT-H checkpoint

   python ret-prompt-based-seg.py --test_dataset_path your-test-dataset-path --mask_save_path ./pred/prompt-based-seg --sam_ckpt_path your-HQ-SAM-checkpoint-path

Our method builds upon a series of foundation models, including DINOv2, LLaVA, Stable Diffusion, SAM, and HQ-SAM. Thanks for their excellent contributions.

If you find our work interesting, please consider using the following BibTeX entry:

@INPROCEEDINGS{EASE,
    author    = {Du, Ji and Hao, Fangwei and Yu, Mingyang and Kong, Desheng and Wu, Jiesheng and Wang, Bin and Xu, Jing and Li, Ping},
    title     = {Shift the Lens: Environment-Aware Unsupervised Camouflaged Object Detection},
    booktitle = {CVPR},
    year      = {2025}
}