Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Drag-guided diffusion models for vehicle image generation (2306.09935v1)

Published 16 Jun 2023 in cs.LG, cs.CV, and cs.GR

Abstract: Denoising diffusion models trained at web-scale have revolutionized image generation. The application of these tools to engineering design is an intriguing possibility, but is currently limited by their inability to parse and enforce concrete engineering constraints. In this paper, we take a step towards this goal by proposing physics-based guidance, which enables optimization of a performance metric (as predicted by a surrogate model) during the generation process. As a proof-of-concept, we add drag guidance to Stable Diffusion, which allows this tool to generate images of novel vehicles while simultaneously minimizing their predicted drag coefficients.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. Geometrical deep learning for performance prediction of high-speed craft. Ocean Engineering, 258:111716, 2022.
  2. Physical design using differentiable learned simulators. arXiv preprint arXiv:2202.00728, 2022.
  3. Universal guidance for diffusion models. arXiv preprint arXiv:2302.07121, 2023.
  4. Geodesic convolutional shape optimization. In International Conference on Machine Learning, pages 472–481. PMLR, 2018.
  5. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687, 2022a.
  6. Improving diffusion models for inverse problems using manifold constraints. arXiv preprint arXiv:2206.00941, 2022b.
  7. K. Crowson. CLIP guided diffusion, 2021. URL https://colab.research.google.com/drive/12a_Wrfi2_gwwAuN3VvMTwVMz9TfqctNj.
  8. P. Dhariwal and A. Nichol. Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems, 34:8780–8794, 2021.
  9. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  10. Debosh: Deep bayesian shape optimization. arXiv preprint arXiv:2109.13337, 2021.
  11. G. Giannone and F. Ahmed. Diffusing the optimal topology: A generative optimization approach. arXiv preprint arXiv:2303.09760, 2023.
  12. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  13. J. Ho and T. Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
  14. Deep learning for real-time aerodynamic evaluations of arbitrary vehicle shapes. arXiv preprint arXiv:2108.05798, 2021.
  15. DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation. 10 2021. doi: 10.48550/arxiv.2110.02711. URL https://arxiv.org/abs/2110.02711v6.
  16. Fine-tuning can distort pretrained features and underperform out-of-distribution. arXiv preprint arXiv:2202.10054, 2022.
  17. Zero-1-to-3: Zero-shot one image to 3d object. arXiv preprint arXiv:2303.11328, 2023.
  18. F. Mazé and F. Ahmed. Topodiff: A performance and constraint-guided diffusion model for topology optimization. arXiv preprint arXiv:2208.09591, 2022.
  19. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2021.
  20. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models. 12 2021. doi: 10.48550/arxiv.2112.10741. URL https://arxiv.org/abs/2112.10741v3.
  21. F. Permenter and C. Yuan. Interpreting and improving diffusion models using the Euclidean distance function. arXiv preprint arXiv:2306.04848, 2023.
  22. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022.
  23. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  24. A. Rahimi and B. Recht. Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007.
  25. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
  26. Meshsdf: Differentiable iso-surface extraction. Advances in Neural Information Processing Systems, 33:22468–22478, 2020.
  27. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022.
  28. Interactive design of 2d car profiles with aerodynamic feedback. In Computer Graphics Forum, volume 42, 2023.
  29. Exploiting generative models for performance predictions of 3d car designs. In 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pages 1–9. IEEE, 2021.
  30. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.11487, 2022.
  31. Surrogate modeling of car drag coefficient with depth and normal rendering. In Proceedings of the ASME 2023 IDETC/CIE, 2023.
  32. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
  33. Deep learning methods for reynolds-averaged navier–stokes simulations of airfoil flows. AIAA Journal, 58(1):25–36, 2020.
  34. N. Umetani and B. Bickel. Learning three-dimensional flow for interactive aerodynamic design. ACM Transactions on Graphics (TOG), 37(4):1–10, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Nikos Arechiga (23 papers)
  2. Frank Permenter (15 papers)
  3. Binyang Song (7 papers)
  4. Chenyang Yuan (12 papers)
Citations (9)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now