Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Stroke-based Neural Painting and Stylization with Dynamically Predicted Painting Region (2309.03504v2)

Published 7 Sep 2023 in cs.CV

Abstract: Stroke-based rendering aims to recreate an image with a set of strokes. Most existing methods render complex images using an uniform-block-dividing strategy, which leads to boundary inconsistency artifacts. To solve the problem, we propose Compositional Neural Painter, a novel stroke-based rendering framework which dynamically predicts the next painting region based on the current canvas, instead of dividing the image plane uniformly into painting regions. We start from an empty canvas and divide the painting process into several steps. At each step, a compositor network trained with a phasic RL strategy first predicts the next painting region, then a painter network trained with a WGAN discriminator predicts stroke parameters, and a stroke renderer paints the strokes onto the painting region of the current canvas. Moreover, we extend our method to stroke-based style transfer with a novel differentiable distance transform loss, which helps preserve the structure of the input image during stroke-based stylization. Extensive experiments show our model outperforms the existing models in both stroke-based neural painting and stroke-based stylization. Code is available at https://github.com/sjtuplayer/Compositional_Neural_Painter

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Wasserstein generative adversarial networks. In International conference on machine learning. PMLR, 214–223.
  2. Carigans: Unpaired photo-to-caricature translation. arXiv preprint arXiv:1811.00222 (2018).
  3. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248–255.
  4. Synthesizing programs for images using reinforced adversarial learning. In International Conference on Machine Learning. PMLR, 1666–1675.
  5. Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2414–2423.
  6. Generative adversarial nets. NeurIPS 27.
  7. Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013).
  8. Improved training of wasserstein gans. Advances in neural information processing systems 30 (2017).
  9. David Ha and Douglas Eck. 2017. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017).
  10. Paul Haeberli. 1990. Paint by numbers: Abstract image representations. In Proceedings of the 17th annual conference on Computer graphics and interactive techniques. 207–214.
  11. Aaron Hertzmann. 1998. Painterly rendering with curved brush strokes of multiple sizes. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. 453–460.
  12. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33 (2020), 6840–6851.
  13. Xun Huang and Serge Belongie. 2017. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE international conference on computer vision. 1501–1510.
  14. Learning to paint with model-based deep reinforcement learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8709–8718.
  15. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In International Conference on Learning Representations.
  16. Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).
  17. Rethinking style transfer: From pixels to parameterized brushstrokes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12196–12205.
  18. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015).
  19. Peter Litwinowicz. 1997. Processing images and video for an impressionist effect. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques. 407–414.
  20. Paint transformer: Feed forward neural painting with stroke prediction. In Proceedings of the IEEE/CVF international conference on computer vision. 6598–6607.
  21. Adaattn: Revisit attention mechanism in arbitrary neural style transfer. In Proceedings of the IEEE/CVF international conference on computer vision. 6649–6658.
  22. Intelli-Paint: Towards Developing More Human-Intelligible Painting Agents. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVI. Springer, 685–701.
  23. Jaskirat Singh and Liang Zheng. 2021. Combining semantic guidance and deep reinforcement learning for generating human level paintings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16387–16396.
  24. AgileGAN: stylizing portraits by inversion-consistent transfer learning. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–13.
  25. Daniel Teece. 1998. 3d painting for non-photorealistic rendering. In ACM SIGGRAPH 98 Conference abstracts and applications. 248.
  26. Im2Oil: Stroke-Based Oil Painting Rendering with Linearly Controllable Fineness Via Adaptive Sampling. In Proceedings of the 30th ACM International Conference on Multimedia. 1035–1046.
  27. Greg Turk and David Banks. 1996. Image-guided streamline placement. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. 453–460.
  28. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR. 586–595.
  29. Learning to sketch with deep q networks and demonstrated strokes. arXiv preprint arXiv:1810.05977 (2018).
  30. Stylized neural painting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 15689–15698.
Citations (10)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now