AnimeGAN improved the CVPR paper CartoonGAN, mainly to solve the over-stylized and color artifact area. For the details, you can refer to the Zhihu article writes by the paper author.Based on the AnimeGAN, the AnimeGANv2 add the total variation loss in the generator loss.

After installing PaddleGAN, you can run python code as follows to generate the stylized image. Where the PATH_OF_IMAGE is your source image path.

from ppgan.apps import AnimeGANPredictor
predictor = AnimeGANPredictor()
predictor.run(PATH_OF_IMAGE)

Or run such a command to get the same result:

python applications/tools/animeganv2.py --input_image ${PATH_OF_IMAGE}

We download the dataset provided by the author from here.Then unzip to the data directory.

wget https://github.com/TachibanaYoshino/AnimeGAN/releases/download/dataset-1/dataset.zip
cd PaddleGAN
unzip YOUR_DATASET_DIR/dataset.zip -d data/animedataset

For example, the structure of animedataset is as following:

animedataset
├── Hayao
│   ├── smooth
│   └── style
├── Paprika
│   ├── smooth
│   └── style
├── Shinkai
│   ├── smooth
│   └── style
├── SummerWar
│   ├── smooth
│   └── style
├── test
│   ├── HR_photo
│   ├── label_map
│   ├── real
│   ├── test_photo
│   └── test_photo256
├── train_photo
└── val

An example is training to Hayao stylize.

1. To ensure the generator can generate the original image, we need to warmup the model.:

python tools/main.py --config-file configs/animeganv2_pretrain.yaml

2. After the warmup, we strat to training GAN.: NOTE： you must modify the configs/animeganv2.yaml > pretrain_ckpt parameter first! ensure the GAN can reuse the warmup generator model. Set the batch size=4 and the learning rate=0.00002. Train 30 epochs on a GTX2060S GPU to reproduce the result. For other hyperparameters, please refer to configs/animeganv2.yaml.

python tools/main.py --config-file configs/animeganv2.yaml

3. Change target style Modify style parameter in the configs/animeganv2.yaml, now support choice from Hayao, Paprika, Shinkai, SummerWar. If you want to use your own dataset, you can modify it to be your own in the configuration file.

NOTE : After modifying the target style, calculate the mean value of the target style dataset at first, and the transform_anime->Add->value parameter in configs/animeganv2.yaml must be modified.

The following example shows how to obtain the mean value of the Hayao style:

python tools/animegan_picmean.py --dataset data/animedataset/Hayao/style
image_num: 1792
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1792/1792 [00:04<00:00, 444.95it/s]
RGB mean diff
[-4.4346957 -8.665916  13.100612 ]

test model on data/animedataset/test/HR_photo

python tools/main.py --config-file configs/animeganv2.yaml --evaluate-only --load ${PATH_OF_WEIGHT}

original image	style image