LEARNING ONE-SHOT FACE REENACTMENT To enable realistic shape (e.g. pose and expression) transfer, existing face reenactment methods rely on a set of target faces for learning subject-specific traits. However, in real-world scenario end users often only have one target face at hand, rendering the existing methods inapplicable. In this work, we bridge this gap by proposing a novel one-shot face reenactment learning system. REENACTGAN: LEARNING TO REENACT FACES VIA BOUNDARY TRANSFER We present a novel learning-based framework for face reenactment. The proposed method, known as ReenactGAN, is capable of transferring facial movements and expressions from an arbitrary person’s monocular video input to a target person’s video. Instead of performing a direct transfer in the pixel space, which could result in structural artifacts, we first map the source face onto a LOOK AT BOUNDARY: A BOUNDARY-AWARE FACE ALIGNMENT ALGORITHM Wider Facial Landmarks in-the-wild (WFLW) contains 10000 faces (7500 for training and 2500 for testing) with 98 fully manual annotated landmarks. Apart from landmark annotation, out new dataset includes rich attribute annotations, i.e., occlusion, pose, make-up, illumination, blur and expression for comprehensive analysis of existing algorithms. LOOK AT BOUNDARY: A BOUNDARY-AWARE FACE ALIGNMENT ALGORITHM We present a novel boundary-aware face alignment algorithm by utilising boundary lines as the geometric structure of a human face to help facial landmark localisation. Unlike the conventional heatmap based method and regression based method, our approach derives face landmarks from boundary lines which remove the ambiguities in the landmark definition. MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4 Zhuoqian Yang2 Wayne Wu1y Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change

Loy6

EVERYTHING’S TALKIN’: PAREIDOLIA FACE REENACTMENT Everything’s Talkin’: Pareidolia Face Reenactment Linsen Song1,2* Wayne Wu3,4* Chaoyou Fu1,2 Chen Qian3 Chen Change Loy4 Ran He1,2� 1School of Artificial Intelligence, University of Chinese Academy of Sciences 2NLPR & CRIPAC, CASIA 3SenseTime Research 4S-Lab, Nanyang Technological University songlinsen2018@ia.ac.cn, fwuwenyan,qiancheng@sensetime.com,

WYWU.GITHUB.IO

@inproceedings{kaisiyuan2020mead, author = {Wang, Kaisiyuan and Wu, Qianyi and Song, Linsen and Yang, Zhuoqian and Wu, Wayne and Qian, Chen and He, Ran and Qiao, Yu and Loy, Chen Change}, title = {MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation}, booktitle = {ECCV}, month = Augest, year = {2020} } MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Supplementary Material Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4? Zhuoqian Yang2 Wayne Wu1?? Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change Loy6 Linsen Song1 ;3 4 Zhuoqian Yang2 Wayne Wu1y Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change

Loy6

EVERYTHING’S TALKIN’: PAREIDOLIA FACE REENACTMENT Everything’s Talkin’: Pareidolia Face Reenactment Linsen Song1,2* Wayne Wu3,4* Chaoyou Fu1,2 Chen Qian3 Chen Change Loy4 Ran He1,2� 1School of Artificial Intelligence, University of Chinese Academy of Sciences 2NLPR & CRIPAC, CASIA 3SenseTime Research 4S-Lab, Nanyang Technological University songlinsen2018@ia.ac.cn, fwuwenyan,qiancheng@sensetime.com,

WYWU.GITHUB.IO

@inproceedings{kaisiyuan2020mead, author = {Wang, Kaisiyuan and Wu, Qianyi and Song, Linsen and Yang, Zhuoqian and Wu, Wayne and Qian, Chen and He, Ran and Qiao, Yu and Loy, Chen Change}, title = {MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation}, booktitle = {ECCV}, month = Augest, year = {2020} } MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Supplementary Material Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4? Zhuoqian Yang2 Wayne Wu1?? Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change Loy6 Linsen Song1 ;3 4 Zhuoqian Yang2 Wayne Wu1y Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change

Loy6

EVERYTHING’S TALKIN’: PAREIDOLIA FACE REENACTMENT Everything’s Talkin’: Pareidolia Face Reenactment Linsen Song1,2* Wayne Wu3,4* Chaoyou Fu1,2 Chen Qian3 Chen Change Loy4 Ran He1,2� 1School of Artificial Intelligence, University of Chinese Academy of Sciences 2NLPR & CRIPAC, CASIA 3SenseTime Research 4S-Lab, Nanyang Technological University songlinsen2018@ia.ac.cn, fwuwenyan,qiancheng@sensetime.com,

WYWU.GITHUB.IO

@inproceedings{kaisiyuan2020mead, author = {Wang, Kaisiyuan and Wu, Qianyi and Song, Linsen and Yang, Zhuoqian and Wu, Wayne and Qian, Chen and He, Ran and Qiao, Yu and Loy, Chen Change}, title = {MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation}, booktitle = {ECCV}, month = Augest, year = {2020} } MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Supplementary Material Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4? Zhuoqian Yang2 Wayne Wu1?? Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change Loy6 Linsen Song1 ;3 4 Zhuoqian Yang2 Wayne Wu1y Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change

Loy6

EVERYTHING’S TALKIN’: PAREIDOLIA FACE REENACTMENT Everything’s Talkin’: Pareidolia Face Reenactment Linsen Song1,2* Wayne Wu3,4* Chaoyou Fu1,2 Chen Qian3 Chen Change Loy4 Ran He1,2� 1School of Artificial Intelligence, University of Chinese Academy of Sciences 2NLPR & CRIPAC, CASIA 3SenseTime Research 4S-Lab, Nanyang Technological University songlinsen2018@ia.ac.cn, fwuwenyan,qiancheng@sensetime.com,

WYWU.GITHUB.IO

@inproceedings{kaisiyuan2020mead, author = {Wang, Kaisiyuan and Wu, Qianyi and Song, Linsen and Yang, Zhuoqian and Wu, Wayne and Qian, Chen and He, Ran and Qiao, Yu and Loy, Chen Change}, title = {MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation}, booktitle = {ECCV}, month = Augest, year = {2020} } MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Supplementary Material Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4? Zhuoqian Yang2 Wayne Wu1?? Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change Loy6 Linsen Song1 ;3 4 Zhuoqian Yang2 Wayne Wu1y Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change

Loy6

WYWU.GITHUB.IO

@inproceedings{kaisiyuan2020mead, author = {Wang, Kaisiyuan and Wu, Qianyi and Song, Linsen and Yang, Zhuoqian and Wu, Wayne and Qian, Chen and He, Ran and Qiao, Yu and Loy, Chen Change}, title = {MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation}, booktitle = {ECCV}, month = Augest, year = {2020} } REENACTGAN: LEARNING TO REENACT FACES VIA BOUNDARY TRANSFER ReenactGAN: Learning to Reenact Faces via Boundary Transfer Wayne Wu 1, Yunxuan Zhang , Cheng Li , Chen Qian , Chen Change Loy2 1 SenseTime Research 2 Nanyang Technological University fwuwenyan,zhangyunxuan,chengli,qiancheng@sensetime.com ccloy@ntu.edu.sg Abstract. MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Supplementary Material Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4? Zhuoqian Yang2 Wayne Wu1?? Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change Loy6 Linsen Song1 ;3 4 Zhuoqian Yang2 Wayne Wu1y Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change

Loy6

WYWU.GITHUB.IO

@inproceedings{kaisiyuan2020mead, author = {Wang, Kaisiyuan and Wu, Qianyi and Song, Linsen and Yang, Zhuoqian and Wu, Wayne and Qian, Chen and He, Ran and Qiao, Yu and Loy, Chen Change}, title = {MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation}, booktitle = {ECCV}, month = Augest, year = {2020} } REENACTGAN: LEARNING TO REENACT FACES VIA BOUNDARY TRANSFER ReenactGAN: Learning to Reenact Faces via Boundary Transfer Wayne Wu 1, Yunxuan Zhang , Cheng Li , Chen Qian , Chen Change Loy2 1 SenseTime Research 2 Nanyang Technological University fwuwenyan,zhangyunxuan,chengli,qiancheng@sensetime.com ccloy@ntu.edu.sg Abstract. MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Supplementary Material Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4? Zhuoqian Yang2 Wayne Wu1?? Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change Loy6 Linsen Song1 ;3 4 Zhuoqian Yang2 Wayne Wu1y Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change

Loy6

MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Supplementary Material Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4? Zhuoqian Yang2 Wayne Wu1?? Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change Loy6 Linsen Song1 ;3 4 Zhuoqian Yang2 Wayne Wu1y Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change

Loy6

WYWU.GITHUB.IO

@inproceedings{kaisiyuan2020mead, author = {Wang, Kaisiyuan and Wu, Qianyi and Song, Linsen and Yang, Zhuoqian and Wu, Wayne and Qian, Chen and He, Ran and Qiao, Yu and Loy, Chen Change}, title = {MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation}, booktitle = {ECCV}, month = Augest, year = {2020} } REENACTGAN: LEARNING TO REENACT FACES VIA BOUNDARY TRANSFER ReenactGAN: Learning to Reenact Faces via Boundary Transfer Wayne Wu 1, Yunxuan Zhang , Cheng Li , Chen Qian , Chen Change Loy2 1 SenseTime Research 2 Nanyang Technological University fwuwenyan,zhangyunxuan,chengli,qiancheng@sensetime.com ccloy@ntu.edu.sg Abstract. MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Supplementary Material Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4? Zhuoqian Yang2 Wayne Wu1?? Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change Loy6 Linsen Song1 ;3 4 Zhuoqian Yang2 Wayne Wu1y Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change

Loy6

MEAD: A LARGE-SCALE AUDIO-VISUAL DATASET FOR EMOTIONAL MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation Supplementary Material Kaisiyuan Wang 1 Qianyi Wu 0001 8764 6178] Linsen Song1 ;3 4? Zhuoqian Yang2 Wayne Wu1?? Chen Qian1 Ran He3;4 Yu Qiao5 Chen Change Loy6[0000 0001 REENACTGAN: LEARNING TO REENACT FACES VIA BOUNDARY TRANSFER ReenactGAN: Learning to Reenact Faces via Boundary Transfer Supplementary Material Wayne Wu 1, Yunxuan Zhang , Cheng Li1, Chen Qian , Chen Change Loy2 1 SenseTime Research 2 Nanyang Technological University fwuwenyan,zhangyunxuan,chengli,qiancheng@sensetime.com

ccloy@ntu.edu.sg

TRANSGAGA: GEOMETRY-AWARE UNSUPERVISED IMAGE-TO-IMAGE TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation Wayne Wu 1Kaidi Cao2 Cheng Li Chen Qian1 Chen Change Loy3 1SenseTime Research 2Stanford University 3Nanyang Technological University fwuwenyan, chengli, qiancheng@sensetime.com kaidicao@cs.stanford.edu ONE-SHOT FACE REENACTMENT 4 ZHANG ET AL.: ONE-SHOT FACE REENACTMENT. Capture face movement by a face tracking or using optical flow on either RGB or RGBD camera. 2) Fit the movement of the face into a LOOK AT BOUNDARY: A BOUNDARY-AWARE FACE ALIGNMENT ALGORITHM Look at Boundary: A Boundary-Aware Face Alignment Algorithm Wayne Wu 1,2, Chen Qian2, Shuo Yang3, Quan Wang2, Yici Cai1, Qiang Zhou1 1Tsinghua National Laboratory for Information Science and Technology (TNList), Department of Computer Science and Technology, Tsinghua University 2SenseTime Research 3Amazon Rekognition 1wwy15@mails.tsinghua.edu.cn 1caiyc@mail.tsinghua.edu.cn

WAYNE WU

吳文巖

ABOUT ME

I am currently a visiting scholar in MMLab Singapore, Nanyang Technological University, supervised by Prof. Chen Change Loy . I am a fourth year Ph.D candidate in BNRist Lab

,

Department of Computer Science and Technology, Tsinghua University. Also, I am a Research Intern at SenseTime Research , working closely with Chen

Qian

and Cheng Li

. My

research interests lie at the intersection of Computer Vision, Augmented Reality and Machine Learning, especially Generative Model and Facial Animation. I used to work with Prof. Zhoujun Li

during Sep. 2013 to

Jun. 2014.

NEWS

* _WE ARE RECRUITING INTERNS / FULL-TIME RESEARCHERS IN COMPUTER VISION AT SENSETIME RESEARCH (BEIJING/SHANGHAI/SHENZHEN/HANGZHOU). IF YOU ARE INTERESTED IN, PLEASE SEND YOUR CV TO MY EMAIL._ * _THREE PAPERS_ to appear in ICCV 2019. (2019-07-23) * The _CELEBRITY VIDEO DATASET _ (CelebV) is publicly available now. (2019-06-12) * Our paper _DISENTANGLINGCONSTY_ is accepted to ICLR 2019 Workshop.

(2019-04-20)

* Our paper _TRANSGAGA _ is accepted to CVPR 2019. (2019-03-02) * The _TESTING CODE AND MODELS _ of LAB are publicly available now. (2018-07-26) * The _WIDER FACIAL LANDMARKS IN-THE-WILD DATASET _ (WFLW) is publicly available now.

(2018-07-15)

* Our paper _REENACTGAN _ is accepted to ECCV 2018. (2018-07-10) * Our team won the _WEBVISION CHALLENGE 2018

_ as first

runner-up. (2018-06-20) * Our paper _LAB _ is accepted to CVPR

2018. (2018-02-26)

* Our paper _DVLN _ is accepted to CVPR 2017 Workshop. (2017-05-10)

PUBLICATION

AGGREGATION VIA SEPARATION: BOOSTING FACIAL LANDMARK DETECTOR WITH SELF-SUPERVISED STYLE TRANSITION Shengju Qian, Keqiang Sun, WAYNE WU, Chen Qian, Jiaya Jia. _IEEE International Conference on Computer Vision (ICCV), 2019._ MAKE A FACE: TOWARDS ARBITRARY HIGH FIDELITY FACE MANIPULATION Shengju Qian, Kwan-Yee Lin, WAYNE WU, Yangxiaokang Liu, Quan Wang, Fumin Shen, Chen Qian, Ran He. _IEEE International Conference on Computer Vision (ICCV), 2019._ FAB: A ROBUST FACIAL LANDMARK DETECTION FRAMEWORK FOR MOTION-BLURRED

VIDEOS

Keqiang Sun, WAYNE WU, Tinghao Liu, Shuo Yang, Quan Wang, Qiang Zhou, Zuochang Ye, Chen Qian. _IEEE International Conference on Computer Vision (ICCV), 2019._ TRANSGAGA: GEOMETRY-AWARE UNSUPERVISED IMAGE-TO-IMAGE TRANSLATION WAYNE WU, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy. _IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

2019._

DISENTANGLING CONTENT AND STYLE VIA UNSUPERVISED GEOMETRY DISTILLATION WAYNE WU, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy. _International Conference on Learning Representations (ICLR) Workshop,

2019._

REENACTGAN: LEARNING TO REENACT FACES VIA BOUNDARY TRANSFER WAYNE WU*, Yunxuan Zhang*, Cheng Li, Chen Qian, Chen Change Loy. _European Conference on Computer Vision (ECCV), 2018._ (* indicates equal contribution.)

LOOK AT BOUNDARY: A BOUNDARY-AWARE FACE ALIGNMENT ALGORITHM WAYNE WU, Chen Qian, Shuo Yang, Quan Wang, Yici Cai, Qiang Zhou. _IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

2018._

LEVERAGING INTRA AND INTER-DATASET VARIATIONS FOR ROBUST FACE

ALIGNMENT

WAYNE WU, Shuo Yang. _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Workshop, 2017._

ACADEMIC SERVICES

* Reviewer for CVPR 2019, ICCV 2019

TEACHING

* 2017, Spring. Numrical Analysis, THU * 2016, Spring. Combinatorial Mathematics, THU * 2014, Spring. Discrete Mathematics, BUAA SELECTED HONORS AND AWARDS * Excellent Undergraduate Student in Beijing, 2015 * Outstanding Undergraduate Thesis, 2015 * National Scholarship (top 1%), 2014.

CONTACTS

__ wuwenyan0503 gmail.com

654 Pageviews

Jul. 02nd - Aug. 02nd