Yuhao Cheng, Zhuo Chen, Xingyu Ren, Wenhan Zhu, Zhengqin Xu, Di Xu, Changpeng Yang, Yichao Yan
CVPR 2024
-
Please refer to eg3d installation guidelines: https://github.com/NVlabs/eg3d.
-
Download pretrained eg3d models ffhqrebalanced512-128.pkl from https://github.com/NVlabs/eg3d/blob/main/docs/models.md, and put it in Codes.
-
Install Mediapipe: https://github.com/google-ai-edge/mediapipe.
-
Download pretrained warp-aware encoder models from https://drive.google.com/file/d/1L576COZyXX0Lwz8U71NbJ2Vtp7SZBZY0/view?usp=sharing, and put it in Codes.
- For one random seed, e.g., 0, you can obtain a facial image with its 3D landmarks.
python generate_3D_landmarks.py --seeds 0-
Then, you will obtain a source_3d_ldms.txt and a target_3d_ldms.txt, then you can edit the locations of ldms in target_3d_ldms.txt.
-
You can obtain warped images with:
python gen_warped_images.py --outdir=out --network=ffhqrebalanced512-128.pkl- You can obtain final edited images with:
python gen_inversion_warped_images.py --outdir=out --network=ffhqrebalanced512-128.pkl
# --edit_shape controls editing facial shape, and --edit_exp controls editing facial expressions.
# We also provide some editing directions in editing_direction, which can be used by --direction.
Some functions or scripts in this implementation are inspired by external sources. We appreciate the authors for their excellent work.
Here are some valuable resources we have benefited from:
- eg3d for the pretrained generative model.
- GOAE for the encoder structure and pretrained models.
- PTI for GAN inversion.
- WarpGAN for triplane Warp.
- Mediapipe for 2D facial landmarks detection.
@inproceedings{cheng20243d,
title={3D-Aware Face Editing via Warping-Guided Latent Direction Learning},
author={Cheng, Yuhao and Chen, Zhuo and Ren, Xingyu and Zhu, Wenhan and Xu, Zhengqin and Xu, Di and Yang, Changpeng and Yan, Yichao},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={916--926},
year={2024}
}