StarGAN Based Facial Expression Transfer for Anime Characters

Human facial expression transfer has been well explored using Generative Adversarial Networks. Also, in case of anime style images, several successful attempts have been made to generate high-quality anime face images using GAN approach. However, the task of anime facial expression transfer is not well studied yet due to the lack of a clean labeled anime dataset. We address this issue from both data and model perspectives, by providing a clean labeled anime dataset and leveraging the use of the StarGAN image-to-image translation framework. Our collected dataset consists of about 5k high- quality anime face images including five major emotions collected from online image boards. We preprocessed our dataset by CARN super-resolution technique to improve quality of the images, and applied tuned StarGAN model to learn the mapping of an input anime image with arbitrary expression to the target expression. We evaluate our work by visually comparing the output translated results with the baseline model. Moreover, we provide a quantitative analysis of our proposed approach by computing the confusion matrix of expression transfer accuracy.