Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 144 tok/s
Gemini 2.5 Pro 46 tok/s Pro
GPT-5 Medium 22 tok/s Pro
GPT-5 High 24 tok/s Pro
GPT-4o 84 tok/s Pro
Kimi K2 200 tok/s Pro
GPT OSS 120B 432 tok/s Pro
Claude Sonnet 4.5 37 tok/s Pro
2000 character limit reached

One Noise to Rule Them All: Learning a Unified Model of Spatially-Varying Noise Patterns (2404.16292v1)

Published 25 Apr 2024 in cs.GR, cs.CV, and cs.LG

Abstract: Procedural noise is a fundamental component of computer graphics pipelines, offering a flexible way to generate textures that exhibit "natural" random variation. Many different types of noise exist, each produced by a separate algorithm. In this paper, we present a single generative model which can learn to generate multiple types of noise as well as blend between them. In addition, it is capable of producing spatially-varying noise blends despite not having access to such data for training. These features are enabled by training a denoising diffusion model using a novel combination of data augmentation and network conditioning techniques. Like procedural noise generators, the model's behavior is controllable via interpretable parameters and a source of randomness. We use our model to produce a variety of visually compelling noise textures. We also present an application of our model to improving inverse procedural material design; using our model in place of fixed-type noise nodes in a procedural material graph results in higher-fidelity material reconstructions without needing to know the type of noise in advance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. Adobe. 2023a. Adobe Substance 3D Designer. https://www.adobe.com/products/substance3d-designer.html.
  2. Adobe. 2023b. Adobe Substance 3D Documentation. https://helpx.adobe.com/substance-3d-designer/substance-compositing-graphs/nodes-reference-for-substance-compositing-graphs/node-library.html.
  3. Wasserstein Generative Adversarial Networks. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 214–223. https://proceedings.mlr.press/v70/arjovsky17a.html
  4. Learning Texture Manifolds with the Periodic Spatial GAN. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 469–477. https://proceedings.mlr.press/v70/bergmann17a.html
  5. Blender. 2023. Blender. https://www.blender.org.
  6. Robert L Cook and Tony DeRose. 2005. Wavelet noise. ACM Transactions on Graphics (TOG) 24, 3 (2005), 803–811.
  7. Image Melding: Combining Inconsistent Images using Patch-based Synthesis. ACM Transactions on Graphics (TOG) (Proceedings of SIGGRAPH 2012) 31, 4, Article 82 (2012), 82:1–82:10 pages.
  8. Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis. Advances in neural information processing systems 34 (2021), 8780–8794.
  9. Julie Dorsey and Pat Hanrahany. 2006. Modeling and rendering of metallic patinas. In ACM SIGGRAPH 2006 Courses. 2–es.
  10. Texturing And Modeling. A Procedural Approach (3 ed.). Morgan Kaufmann.
  11. Computer rendering of stochastic models. Commun. ACM 25, 6 (1982), 371–384.
  12. A review of digital terrain modeling. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 553–577.
  13. Texture Synthesis Using Convolutional Neural Networks. In NeurIPS.
  14. Terrain generation using procedural models based on hydrology. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1–13.
  15. MatFormer: A generative model for procedural materials. arXiv preprint arXiv:2207.01044 (2022).
  16. A bayesian inference framework for procedural material parameter estimation. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 255–266.
  17. Nicholas Guttenberg. 2023. Diffusion with Offset Noise. (2023).
  18. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017).
  19. Interactive animation of ocean waves. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation. 161–166.
  20. Denoising diffusion probabilistic models. Advances in neural information processing systems 33 (2020), 6840–6851.
  21. A novel framework for inverse procedural texture modeling. ACM Transactions on Graphics (ToG) 38, 6 (2019), 1–14.
  22. Node graph optimization using differentiable proxies. In ACM SIGGRAPH 2022 conference proceedings. 1–9.
  23. DU-GAN: Generative adversarial networks with dual-domain U-Net-based discriminators for low-dose CT denoising. IEEE Transactions on Instrumentation and Measurement 71 (2021), 1–12.
  24. Michael J Jones and Tomaso Poggio. 1998. Multidimensional morphable models. In Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271). IEEE, 683–688.
  25. UCSG-NET- Unsupervised Discovering of Constructive Solid Geometry Tree. In Advances in Neural Information Processing Systems.
  26. A survey of procedural noise functions. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 2579–2600.
  27. Procedural noise using sparse Gabor convolution. ACM Transactions on Graphics (TOG) 28, 3 (2009), 1–10.
  28. Chuan Li and Michael Wand. 2016. Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks. In Computer Vision – ECCV 2016, Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, Cham, 702–716.
  29. Common diffusion noise schedules and sample steps are flawed. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 5404–5411.
  30. InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation. arXiv:2309.06380 [cs.LG]
  31. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017).
  32. Texture design using a simplicial complex of morphable textures. ACM Transactions on Graphics (TOG) 24, 3 (2005), 787–794.
  33. Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning. PMLR, 8162–8171.
  34. Semantic Image Synthesis with Spatially-Adaptive Normalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  35. Ken Perlin. 1985. An Image Synthesizer. In Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’85). Association for Computing Machinery, New York, NY, USA, 287–296. https://doi.org/10.1145/325334.325247
  36. Ken Perlin. 2002. Improving noise. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques. 681–682.
  37. A survey of exemplar-based texture synthesis. Annals of Mathematical Sciences and Applications 3, 1 (2018), 89–148.
  38. CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 12478–12487.
  39. Carlos Rodriguez-Pardo and Elena Garces. 2023. SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps. IEEE Transactions on Visualization and Computer Graphics 29, 6 (jun 2023), 2914–2925. https://doi.org/10.1109/TVCG.2022.3143615
  40. A u-net based discriminator for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8207–8216.
  41. Omry Sendik and Daniel Cohen-Or. 2017. Deep Correlations for Texture Synthesis. ACM Trans. Graph. 36, 5, Article 161 (jul 2017), 15 pages. https://doi.org/10.1145/3015461
  42. Efficient attention: Attention with linear complexities. In Proceedings of the IEEE/CVF winter conference on applications of computer vision. 3531–3539.
  43. MATch: Differentiable Material Graphs for Procedural Material Capture. ACM Trans. Graph. 39, 6 (Dec. 2020), 1–15.
  44. Ken Shoemake. 1985. Animating rotation with quaternion curves. In Proceedings of the 12th annual conference on Computer graphics and interactive techniques. 245–254.
  45. Denoising Diffusion Implicit Models. arXiv:2010.02502 (October 2020). https://arxiv.org/abs/2010.02502
  46. Yang Song. 2021. Generative Modeling by Estimating Gradients of the Data Distribution. https://yang-song.net/blog/2021/score/.
  47. Procedural phasor noise. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1–13.
  48. Texture Networks: Feed-forward Synthesis of Textures and Stylized Images. In Proceedings of The 33rd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 48), Maria Florina Balcan and Kilian Q. Weinberger (Eds.). PMLR, New York, New York, USA, 1349–1357.
  49. Don’t Play Favorites: Minority Guidance for Diffusion Models. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=3NmO9lY4Jn
  50. Procedural generation of dungeons. IEEE Transactions on Computational Intelligence and AI in Games 6, 1 (2013), 78–89.
  51. Steven Worley. 1996. A cellular texture basis function. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’96). Association for Computing Machinery, New York, NY, USA, 291–294. https://doi.org/10.1145/237170.237267
  52. Yuxin Wu and Kaiming He. 2018. Group Normalization. In Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XIII (Lecture Notes in Computer Science, Vol. 11217), Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). Springer, 3–19. https://doi.org/10.1007/978-3-030-01261-8_1
  53. Tackling the Generative Learning Trilemma with Denoising Diffusion GANs. In ICLR.
  54. CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features. In International Conference on Computer Vision (ICCV).
  55. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR.
  56. PhotoMat: A Material Generator Learned from Single Flash Photos. In ACM SIGGRAPH 2023 Conference Proceedings. 1–11.
  57. Neural Texture Synthesis with Guided Correspondence. In Conference on Computer Vision and Pattern Recognition (CVPR). 18095–18104.
  58. Non-stationary Texture Synthesis by Adversarial Expansion. ACM Transactions on Graphics (Proc. SIGGRAPH) 37, 4 (2018), 49:1–49:13.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Questions

We haven't generated a list of open questions mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 1 tweet and received 0 likes.

Upgrade to Pro to view all of the tweets about this paper:

Start a free 7-day Pro trial