Joint Geometric-Semantic Driven Character Line Drawing Generation (2206.02998v3)
Abstract: Character line drawing synthesis can be formulated as a special case of image-to-image translation problem that automatically manipulates the photo-to-line drawing style transformation. In this paper, we present the first generative adversarial network based end-to-end trainable translation architecture, dubbed P2LDGAN, for automatic generation of high-quality character drawings from input photos/images. The core component of our approach is the joint geometric-semantic driven generator, which uses our well-designed cross-scale dense skip connections framework to embed learned geometric and semantic information for generating delicate line drawings. In order to support the evaluation of our model, we release a new dataset including 1,532 well-matched pairs of freehand character line drawings as well as corresponding character images/photos, where these line drawings with diverse styles are manually drawn by skilled artists. Extensive experiments on our introduced dataset demonstrate the superior performance of our proposed models against the state-of-the-art approaches in terms of quantitative, qualitative and human evaluations. Our code, models and dataset will be available at Github.
- Face photo-sketch synthesis via full-scale identity supervision. Pattern Recognition 124 (2022), 108446.
- Wengling Chen and James Hays. 2018. Sketchygan: Towards diverse and realistic sketch to image synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9416–9425.
- Puppeteergan: Arbitrary portrait animation with semantic-aware appearance transformation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13518–13527.
- CartoonLossGAN: Learning Surface and Coloring of Images for Cartoonization. IEEE Transactions on Image Processing 31 (2021), 485–498.
- Sketchycoco: Image generation from freehand scene sketches. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5174–5183.
- Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2414–2423.
- Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017).
- Multimodal unsupervised image-to-image translation. In Proceedings of the European conference on computer vision (ECCV). 172–189.
- Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125–1134.
- StyleCariGAN: caricature generation via StyleGAN feature map modulation. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–16.
- Alexia Jolicoeur-Martineau. 2018. The relativistic discriminator: a key element missing from standard GAN. In International Conference on Learning Representations.
- Learning to discover cross-domain relations with generative adversarial networks. In International conference on machine learning. PMLR, 1857–1865.
- Deep extraction of manga structural lines. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1–12.
- Face sketch synthesis using regularized broad learning system. IEEE Transactions on Neural Networks and Learning Systems (2021).
- Im2pencil: Controllable pencil illustration from photographs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1525–1534.
- Pd-gan: Probabilistic diverse gan for image inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9371–9381.
- Unsupervised image-to-image translation networks. Advances in neural information processing systems 30 (2017).
- Continuous and diverse image-to-image translation via signed attribute vectors. International Journal of Computer Vision (2022), 1–33.
- A Survey on Deep Learning for Skeleton-Based Human Animation. In Computer Graphics Forum. Wiley Online Library.
- Image-to-image translation: Methods and applications. IEEE Transactions on Multimedia (2021).
- Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network. In 2021 IEEE International Joint Conference on Biometrics (IJCB). IEEE, 1–8.
- Encoding in style: a stylegan encoder for image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2287–2296.
- A coarse-to-fine approach for dynamic-to-static image translation. Pattern Recognition 123 (2022), 108373.
- High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8798–8807.
- Xinrui Wang and Jinze Yu. 2020. Learning to cartoonize using white-box cartoon representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8090–8099.
- Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600–612.
- Towards Vivid and Diverse Image Colorization with Generative Color Prior. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14377–14386.
- IsGAN: Identity-sensitive generative adversarial network for face photo-sketch synthesis. Pattern Recognition 119 (2021), 108077.
- Quality metric guided portrait line drawing generation from unpaired training data. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).
- Unpaired portrait drawing generation via asymmetric cycle mapping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8217–8225.
- Dualgan: Unsupervised dual learning for image-to-image translation. In Proceedings of the IEEE international conference on computer vision. 2849–2857.
- Mingcheng Yuan and Edgar Simo-Serra. 2021. Line art colorization with concatenated spatial attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3946–3950.
- SmartShadow: Artistic Shadow Drawing Tool for Line Drawings. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5391–5400.
- User-guided line art flat filling with split filling mechanism. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9889–9898.
- Generating manga from illustrations via mimicking manga creation workflow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5642–5651.
- Uctgan: Diverse image inpainting based on unsupervised cross-space translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5741–5750.
- Learning to shadow hand-drawn sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7436–7445.
- Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223–2232.