Skip to content

Commit

Permalink
Merge pull request #8 from podgorskiy/readme
Browse files Browse the repository at this point in the history 
Readme
  • Loading branch information
Stanislav Pidhorskyi authored Apr 21, 2020
2 parents 4a384cd + 49fee13 commit ac6f049
Showing 1 changed file with 219 additions and 4 deletions.
223 changes: 219 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@

## To run the demo

To run the demo, you will need to have a CUDA capable GPU, pytorch >= v1.3.1 and cuda/cuDNN drivers installed.
Install required packages:
To run the demo, you will need to have a CUDA capable GPU, PyTorch >= v1.3.1 and cuda/cuDNN drivers installed.
Install the required packages:

pip install -r requirements.txt

Download pretrained models:
Download pre-trained models:

python training_artifacts/download_all.py

Expand All @@ -74,5 +74,220 @@ You can change the config using `-c` parameter. To run on `celeb-hq` in 256x256

python interactive_demo.py -c celeba-hq256

However, for configs other then FFHQ, you will need to obtain new principal direction vectors for the attibutes.
However, for configs other then FFHQ, you need to obtain new principal direction vectors for the attributes.

## Repository organization

#### Running scripts

The code in the repository is organized in such a way that all scripts must be run from the root of the repository.
If you use an IDE (e.g. PyCharm or Visual Studio Code), just set *Working Directory* to point to the root of the repository.

If you want to run from the command line, then you also need to set **PYTHONPATH** variable to point to the root of the repository.

For example, let's say we've cloned repository to *~/ALAE* directory, then do:

$ cd ~/ALAE
$ export PYTHONPATH=$PYTHONPATH:$(pwd)

![pythonpath](https://podgorskiy.com/static/pythonpath.svg)

Now you can run scripts as follows:

$ python style_mixing/stylemix.py

#### Repository structure


| Path | Description
| :--- | :----------
| ALAE | Repository root folder
| ├  configs | Folder with yaml config files.
| │  ├  bedroom.yaml | Config file for LSUN bedroom dataset at 256x256 resolution.
| │  ├  celeba.yaml | Config file for CelebA dataset at 128x128 resolution.
| │  ├  celeba-hq256.yaml | Config file for CelebA-HQ dataset at 256x256 resolution.
| │  ├  celeba_ablation_nostyle.yaml | Config file for CelebA 128x128 dataset for ablation study (no styles).
| │  ├  celeba_ablation_separate.yaml | Config file for CelebA 128x128 dataset for ablation study (separate encoder and discriminator).
| │  ├  celeba_ablation_z_reg.yaml | Config file for CelebA 128x128 dataset for ablation study (regress in Z space, not W).
| │  ├  ffhq.yaml | Config file for FFHQ dataset at 1024x1024 resolution.
| │  ├  mnist.yaml | Config file for MNIST dataset using Style architecture.
| │  └  mnist_fc.yaml | Config file for MNIST dataset using only fully connected layers (Permutation Invariant MNIST).
| ├  dataset_preparation | Folder with scripts for dataset preparation.
| │  ├  prepare_celeba_hq_tfrec.py | To prepare TFRecords for CelebA-HQ dataset at 256x256 resolution.
| │  ├  prepare_celeba_tfrec.py | To prepare TFRecords for CelebA dataset at 128x128 resolution.
| │  ├  prepare_mnist_tfrec.py | To prepare TFRecords for MNIST dataset.
| │  ├  split_tfrecords_bedroom.py | To split official TFRecords from StyleGAN paper for LSUN bedroom dataset.
| │  └  split_tfrecords_ffhq.py | To split official TFRecords from StyleGAN paper for FFHQ dataset.
| ├  dataset_samples | Folder with sample inputs for different datasets. Used for figures and for test inputs during training.
| ├  make_figures | Scripts for making various figures.
| ├  metrics | Scripts for computing metrics.
| ├  principal_directions | Scripts for computing principal direction vectors for various attributes. **For interactive demo**.
| ├  style_mixing | Sample inputs and script for producing style-mixing figures.
| ├  training_artifacts | Default place for saving checkpoints/sample outputs/plots.
| │  └  download_all.py | Script for downloading all pretrained models.
| ├  interactive_demo.py | Runnable script for interactive demo.
| ├  train_alae.py | Runnable script for training.
| ├  train_alae_separate.py | Runnable script for training for ablation study (separate encoder and discriminator).
| ├  checkpointer.py | Module for saving/restoring model weights, optimizer state and loss history.
| ├  custom_adam.py | Customized adam optimizer for learning rate equalization and zero second beta.
| ├  dataloader.py | Module with dataset classes, loaders, iterators, etc.
| ├  defaults.py | Definition for config variables with default values.
| ├  launcher.py | Helper for running multi-GPU, multiprocess training. Sets up config and logging.
| ├  lod_driver.py | Helper class for managing growing/stabilizing network.
| ├  lreq.py | Custom `Linear`, `Conv2d` and `ConvTranspose2d` modules for learning rate equalization.
| ├  model.py | Module with high-level model definition.
| ├  model_separate.py | Same as above, but for ablation study.
| ├  net.py | Definition of all network blocks for multiple architectures.
| ├  registry.py | Registry of network blocks for selecting from config file.
| ├  scheduler.py | Custom schedulers with warm start and aggregating several optimizers.
| ├  tracker.py | Module for plotting losses.
| └  utils.py | Decorator for async call, decorator for caching, registry for network blocks.


#### Configs

In this codebase [**yacs**](https://github.com/rbgirshick/yacs) is used to handle configurations.

Most of the runnable scripts accept `-c` parameter that can specify config files to use.
For example, to make reconstruction figures, you can run:

python make_figures/make_recon_figure_paged.py
python make_figures/make_recon_figure_paged.py -c celeba
python make_figures/make_recon_figure_paged.py -c celeba-hq256
python make_figures/make_recon_figure_paged.py -c bedroom

The Default config is `ffhq`.

#### Datasets

Training is done using TFRecords. TFRecords are read using [DareBlopy](https://github.com/podgorskiy/DareBlopy), which allows using them with Pytorch.

In config files as well as in all preparation scripts, it is assumed that all datasets are in `/data/datasets/`. You can either change path in config files, either create a symlink to where you store datasets.

The official way of generating CelebA-HQ can be challenging. Please refer to this page: https://github.com/suvojit-0x55aa/celebA-HQ-dataset-download
You can get the pre-generated dataset from: https://drive.google.com/drive/folders/11Vz0fqHS2rXDb5pprgTjpD7S2BAJhi1P

#### Pre-trained models

To download pre-trained models run:

python training_artifacts/download_all.py

In config files, `OUTPUT_DIR` points to where weights are saved to and read from. For example: `OUTPUT_DIR: training_artifacts/celeba-hq256`

In `OUTPUT_DIR` it saves a file `last_checkpoint` which contains path to the actual `.pth` pickle with model weight. If you want to test the model with a specific weight file, you can simply modify `last_checkpoint` file.


## Generating figures

#### Style-mixing

To generate style-mixing figures run:

python style_mixing/stylemix.py -c <config>

Where instead of `<config>` put one of: `ffhq`, `celeba`, `celeba-hq256`, `bedroom`


#### Reconstructions

To generate reconstruction with multiple scale images:

python make_figures/make_recon_figure_multires.py -c <config>

To generate reconstruction from all sample inputs on multiple pages:

python make_figures/make_recon_figure_paged.py -c <config>

There are also:

python make_figures/old/make_recon_figure_celeba.py
python make_figures/old/make_recon_figure_bed.py

To generate reconstruction from test set of FFHQ:

python make_figures/make_recon_figure_ffhq_real.py

To generate interpolation figure:

python make_figures/make_recon_figure_interpolation.py -c <config>

To generate traversals figure:

(For datasets other then FFHQ, you will need to find principal directions first)

python make_figures/make_traversarls.py -c <config>

#### Generations

To make generation figure run:

make_generation_figure.py -c <config>

## Training

In addition to installing required packages:

pip install -r requirements.txt

You will need to install [DareBlopy](https://github.com/podgorskiy/DareBlopy):

pip install dareblopy

To run training:

python train_alae.py -c <config>

It will run multi-GPU training on all available GPUs. It uses `DistributedDataParallel` for parallelism.
If only one GPU available, it will run on single GPU, no special care is needed.

The recommended number of GPUs is 8. Reproducibility on a smaller number of GPUs may have issues. You might need to adjust the batch size in the config file depending on the memory size of the GPUs.

## Running metrics

In addition to installing required packages and [DareBlopy](https://github.com/podgorskiy/DareBlopy), you need to install TensorFlow and dnnlib from StyleGAN.

Tensorflow must be of version `1.10`:

pip install tensorflow-gpu==1.10

It requires CUDA version 9.0.

Perhaps, the best way is to use Anaconda to handle this, but I prefer installing CUDA 9.0 from pop-os repositories (works on Ubuntu):

```
sudo echo "deb http://apt.pop-os.org/proprietary bionic main" | sudo tee -a /etc/apt/sources.list.d/pop-proprietary.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key 204DD8AEC33A7AFF
sudo apt update
sudo apt install system76-cuda-9.0
sudo apt install system76-cudnn-9.0
```

Then just set `LD_LIBRARY_PATH` variable:

```
export LD_LIBRARY_PATH=/usr/lib/cuda-9.0/lib64
```

Dnnlib is a package used in StyleGAN. You can install it with:

pip install https://github.com/podgorskiy/dnnlib/releases/download/0.0.1/dnnlib-0.0.1-py3-none-any.whl

All code for running metrics is heavily based on those from StyleGAN repository. It also uses the same pre-trained models:

[https://github.com/NVlabs/stylegan#licenses](https://github.com/NVlabs/stylegan#licenses)

> inception_v3_features.pkl and inception_v3_softmax.pkl are derived from the pre-trained Inception-v3 network by Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. The network was originally shared under Apache 2.0 license on the TensorFlow Models repository.
>
> vgg16.pkl and vgg16_zhang_perceptual.pkl are derived from the pre-trained VGG-16 network by Karen Simonyan and Andrew Zisserman. The network was originally shared under Creative Commons BY 4.0 license on the Very Deep Convolutional Networks for Large-Scale Visual Recognition project page.
>
> vgg16_zhang_perceptual.pkl is further derived from the pre-trained LPIPS weights by Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The weights were originally shared under BSD 2-Clause "Simplified" License on the PerceptualSimilarity repository.
Finally, to run metrics:

python metrics/fid.py -c <config> # FID score on generations
python metrics/fid_rec.py -c <config> # FID score on reconstructions
python metrics/ppl.py -c <config> # PPL score on generations
python metrics/lpips.py -c <config> # LPIPS score of reconstructions

0 comments on commit ac6f049

Please sign in to comment.