Hadi M. Dolatabadi, Sarah Erfani, and Christopher Leckie 2024

This repository contains the official implementation of the IEEE SatML 2024 paper "The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models." The code is built upon the official repository of DiffPure.

Abstract: Protecting personal data against exploitation of machine learning models is crucial. Recently, availability attacks have shown great promise to provide an extra layer of protection against the unauthorized use of data to train neural networks. These methods aim to add imperceptible noise to clean data so that the neural networks cannot extract meaningful patterns from the protected data, claiming that they can make personal data "unexploitable." This paper provides a strong countermeasure against such approaches, showing that unexploitable data might only be an illusion. In particular, we leverage the power of diffusion models and show that a carefully designed denoising process can counteract the effectiveness of the data-protecting perturbations. We rigorously analyze our algorithm, and theoretically prove that the amount of required denoising is directly related to the magnitude of the data-protecting perturbations. Our approach, called AVATAR, delivers state-of-the-art performance against a suite of recent availability attacks in various scenarios, outperforming adversarial training even under distribution mismatch between the diffusion model and the protected data. Our findings call for more research into making personal data unexploitable, showing that this goal is far from over.

The code has been tested with PyTorch 1.7.1 and CUDA 11.7. To install the entire requirements, run the following:

pip install -r requirements.txt

The overall structure of the code has been given above. In particular, we mostly work with the two files AVATAR_sanitization.py and AVATAR_evaluation.py. These two steps are equivalent to the two steps taken in AVATAR. The first step sanitizes the dataset. In the second step, we train a model on these datasets using different architectures, datasets, augmentations, etc. As the primary data structure, we use .npz for CIFAR-10, CIFAR-100, and SVHN. For ImageNet-100, we use .lmdb for which we have provided a sample script of converting raw data to this format. Finally, for WebFace we use the usual ImageFolder structure of PyTorch.

We will explain the two primary steps below.

In this step, we use various diffusion models to sanitize the (possibly) corrupted data. To this end, first you need to either download the pre-trained diffusion models or train one of your own. For the experiments of this paper, we use the pre-trained models provided with the DiffPure and DDPM-IP libraries as well as our in-house models. Locate these files in their respective subfolder within the pretrained folder. In particular, put the CIFAR-10, CIFAR-100, SVHN, ImageNet-10, and CelebA-HQ into the score_sde folder and the ImageNet and ImageNet-32x32 into the guided_diffusion subfolder. Below, we provide the link for downloading the pre-trained diffusion models used in our experiments.

Note 1: if any of the above download links for external models are not working, please visit their respective source GitHub.

Note 2: if you want to run new experiments with your own diffusion model, you should add its library to this repo and create a runner in the runners folder. In case your new model follows previous architectures such as GuidedDiffusion or ScoreSDE, you can just add a suitable loader to their runner files.

Once the diffusion model is ready, we can defuse availability attacks. To this end, we assume that we have an attack, denoted with UNL_ALG, that has been stored as either a .npz array with keys data (stored as np.uint8) and targets, or a respective .lmdb file. Then, we can run the following:

python AVATAR_sanitization.py \
        --domain <UNL_DOMAIN> \
        --unlearnable_alg <UNL_ALG> \
        --path <UNL_PATH> \
        --config <CONFIG_FILE> \
        --diffusion_type <DIFFUSION_TYPE> \
        --score_type <SCORE_TYPE> \
        --t <DENOISING_STEPS> \
        --save_path <SAVE_PATH>

where the parameters' definition is given below:

You can familiarize yourself with the data structure of different domains by looking at how the data is stored at the end of the AVATAR_sanitization.py step.

We have released some of the availability attacks and their denoised versions for the CIFAR-10 dataset here. The original availability attacks, say AR, have been named as AR_CIFAR10_Sanitized_0.npz. If the data has been denoised with our cifar10 ScoreSDE model for 100 steps, we denote it by AR_CIFAR10_Sanitized_100_cifar10.npz. Following this convention, data that has been defused with the imagenet-32x32 model has been named as AR_CIFAR10_Sanitized_100_in32.npz.

Note: We will keep the data on Google Drive for one year after publication. After that, it is going to be deleted to save space.

After we have created a database of the availability attack and/or their sanitized version using Step 1, we can train a model to see the performance. Besides the usual functionality, in this repo we also include basic and advanced data augmentation techniques for the CIFAR-10 dataset. These augmentations have been added similar to the EMN and ISS libraries. To run the training, we can use (here we use the most important arguments):

python AVATAR_evaluation.py \
        --path <PATH> \
        --arch <ARCH> \
        --use_cutout \
        --use_mixup  \
        --use_cutmix \
        --use_fa \
        --grayscale \ 
        --jpeg <JPEG_Q> \
        --bdr <JPEG_BIT_DEPTH> \
        --domain <UNL_DOMAIN> \
        --unlearnable_alg <UNL_ALG>    

where the parameters' definition is given below:

Note: for ImageNet-100 and WebFace experiments, we used the training settings found on the REMN repo.

This repository is mainly built upon DiffPure library. We have also used the repositories of DDPM-IP, ScoreSDE, AR, EMN, REMN, NTGA, SHR, and ISS to generate the attacks, purify them, or train classifiers for evaluation. We thank the authors of all these repositories and have cited their respective papers.

Now, if you have found our code or paper beneficial to your research in any shape, please consider starring the repo and citing our research paper as:

@inproceedings{dolatabadi2024avatar,
  title={The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models},
  author={Hadi Mohaghegh Dolatabadi and Sarah Erfani and Christopher Leckie},
  booktitle = {Proceedings of the {IEEE} Conference on Secure and Trustworthy Machine Learning ({SatML})},
  year={2024}
}