PSGAN: Revisit the binary discriminator and an alternative for face frontalization

Generative adversarial network (GAN) based face frontalization is a cheap and convenient way to eliminate the impact of pose variance on face recognition. The sigmoid cross-entropy loss function is usually employed for the discriminator in those GAN based face synthesis methods. There are two disadvantages for this loss function: 1) The discriminator always wins the generator easily at the beginning of training because the convergence of the discriminator and the generator is unbalanced; 2) The training of GANs becomes unstable due to the prediction boundary uncertainty and massive parameters of the traditional binary discriminator. In order to eliminate the impacts caused by the traditional discriminator in the general GANs, a Bayesian induced perceptual self-representation discriminator (i.e. PSD) is proposed, which can also maintain the identity information, and simultaneously reduce the model parameters and training difficulty. There are three key contributions in this work: 1) On the basis of PSD, a perceptual self-representation GAN (i.e. PSGAN) with a new architecture is proposed, which reduces the training difficulty without lowering the synthetic quality; 2) In order to further improve the performance of our method, multiple features extracted from different layers are adopted to constitute a multi-perceptual self-representation discriminator (i.e. MPSD); 3) The proposed PSD discriminator is more lightweight with fewer parameters and can also be easily plugged and played in various GANs. Extensive qualitative and quantitative experiments on both restricted and unrestricted face databases and non-facial datasets demonstrate its superiority.