FaceEraser: Removing Facial Parts for Augmented Reality

Abstract: Our task is to remove all facial parts (e.g., eyebrows, eyes, mouth and nose), and then impose visual elements onto the blank'' face for augmented reality. Conventional object removal methods rely on image inpainting techniques (e.g., EdgeConnect, HiFill) that are trained in a self-supervised manner with randomly manipulated image pairs. Specifically, given a set of natural images, randomly masked images are used as inputs and the raw images are treated as ground truths. Whereas, this technique does not satisfy the requirements of facial parts removal, as it is hard to obtainground-truth'' images with real blank'' faces. To address this issue, we propose a novel data generation technique to produce paired training data that well mimic theblank'' faces. In the mean time, we propose a novel network architecture for improved inpainting quality for our task. Finally, we demonstrate various face-oriented augmented reality applications on top of our facial parts removal model. The source codes are released at \href{https://github.com/duxingren14/FaceEraser}{duxingren14/FaceEraser} on github for research purposes.