Reconstruct face from features based on genetic algorithm using GAN generator as a distribution constraint

Face recognition based on deep convolutional neural networks (CNN) shows superior accuracy performance attributed to the high discriminative features extracted. Yet, the security and privacy of the extracted features from deep learning models (deep features) have often been overlooked. This paper proposes the reconstruction of face images from deep features without accessing the CNN network configurations as a constrained optimization problem. Such optimization minimizes the distance between the features extracted from the original face image and the reconstructed face image. Instead of directly solving the optimization problem in the image space, we innovatively reformulate the problem by looking for a latent vector of a generative adversarial networks (GAN) generator, then use it to generate the face image. The GAN generator serves a dual role in this novel framework, i.e., face distribution constraint of the optimization goal and a face generator. To solve this optimization problem, We present an optimization approach based on a Genetic Algorithm. On top of the novel optimization task, we also propose an attack pipeline to impersonate the target user based on the generated face image. Our results show that the generated face images can achieve a state-of-the-art successful attack rate of 99.33% on Labeled Faces in the Wild (LFW) under type-I attack at a false accept rate of 0.1%. Our work sheds light on biometric deployment to meet privacy-preserving and security policies.