Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards AI-Architecture Liberty: A Comprehensive Survey on Design and Generation of Virtual Architecture by Deep Learning (2305.00510v4)

Published 30 Apr 2023 in cs.HC, cs.CV, and cs.LG

Abstract: 3D shape generation techniques leveraging deep learning have garnered significant interest from both the computer vision and architectural design communities, promising to enrich the content in the virtual environment. However, research on virtual architectural design remains limited, particularly regarding designer-AI collaboration and deep learning-assisted design. In our survey, we reviewed 149 related articles (81.2% of articles published between 2019 and 2023) covering architectural design, 3D shape techniques, and virtual environments. Through scrutinizing the literature, we first identify the principles of virtual architecture and illuminate its current production challenges, including datasets, multimodality, design intuition, and generative frameworks. We then introduce the latest approaches to designing and generating virtual buildings leveraging 3D shape generation and summarize four characteristics of various approaches to virtual architecture. Based on our analysis, we expound on four research agendas, including agency, communication, user consideration, and integrating tools. Additionally, we highlight four important enablers of ubiquitous interaction with immersive systems in deep learning-assisted architectural generation. Our work contributes to fostering understanding between designers and deep learning techniques, broadening access to designer-AI collaboration. We advocate for interdisciplinary efforts to address this timely research topic, facilitating content designing and generation in the virtual environment.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (187)
  1. A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges. Information Fusion 76 (2021), 243–297. https://doi.org/10.1016/j.inffus.2021.05.008
  2. Learning Representations and Generative Models for 3D Point Clouds. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80). PMLR, Stockholm, Sweden, 40–49. https://proceedings.mlr.press/v80/achlioptas18a.html
  3. Generative Adversarial Network: An Overview of Theory and Applications. International Journal of Information Management Data Insights 1, 1 (2021), 100004. https://doi.org/10.1016/j.jjimei.2020.100004
  4. Deep learning in the Construction Industry: A Review of Present Status and Future Innovations. Journal of Building Engineering 32 (2020), 101827. https://doi.org/10.1016/j.jobe.2020.101827
  5. Introduction to Human-Building Interaction (HBI): Interfacing HCI with Architecture and Urban Design. ACM Trans. Comput.-Hum. Interact. 26, 2, Article 6 (mar 2019), 10 pages. https://doi.org/10.1145/3309714
  6. A State-of-the-Art Survey on Deep Learning Theory and Architectures. Electronics 8, 3 (2019). https://doi.org/10.3390/electronics8030292
  7. Mohammad Samiul Arshad and William J. Beksi. 2020. A Progressive Conditional Generative Adversarial Network for Generating Dense and Colored 3D Point Clouds. In 2020 International Conference on 3D Vision (3DV). online, 712–722. https://doi.org/10.1109/3DV50981.2020.00081
  8. Karen El Asmar and Harpreet Sareen. 2020. Machinic Interpolations: A GAN Pipeline for Integrating Lateral Thinking in Computational Tools of Architecture. In Blucher Design Proceedings. Editora Blucher, Medellín, Colombia, 60–66. https://doi.org/10.5151/sigradi2020-9
  9. Floorplan embedding with latent semantics and human behavior annotations. In Proceedings of the 11th Annual Symposium on Simulation for Architecture and Urban Design (Virtual Event, Austria) (SimAUD ’20). Society for Computer Simulation International, San Diego, CA, USA, Article 11, 8 pages. https://doi.org/10.5555/3465085.3465096
  10. Artificial Intelligence and Smart Vision for Building and Construction 4.0: Machine and Deep Learning Methods and Applications. Automation in Construction 141 (2022), 104440. https://doi.org/10.1016/j.autcon.2022.104440
  11. Learning Spatiality - A GAN method for designing architectural models through labelled sections. In Proceedings of the 40st Conference on Education and Research in Computer Aided Architectural Design in Europe (eCAADe). Ghent, Belgium, 611–619. https://doi.org/10.52842/conf.ecaade.2022.2.611
  12. Affective Computing for Generating Virtual Procedural Environments Using Game Technologies. In Proceedings of the 40th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA). Online and Global, 120–129. https://doi.org/10.52842/conf.acadia.2020.1.120
  13. Richard Bartle. 2003. Designing Virtual Worlds. New Riders Games. https://doi.org/10.5555/1196681
  14. Richard A. Bartle. 2010. From MUDs to MMORPGs: The History of Virtual Worlds. Springer Netherlands, Dordrecht, 23–39. https://doi.org/10.1007/978-1-4020-9789-8_2
  15. Alessandro Bava. 2020. Computational Tendencies. https://www.e-flux.com/architecture/intelligence/310405/computational-tendencies/
  16. Multi-chart generative surface modeling. ACM Trans. Graph. 37, 6, Article 215 (dec 2018), 15 pages. https://doi.org/10.1145/3272127.3275052
  17. Impact of Built Environment Design on Emotion Measured via Neurophysiological Correlates and Subjective Indicators: A Systematic Review. Journal of Environmental Psychology 66 (2019), 101344. https://doi.org/10.1016/j.jenvp.2019.101344
  18. Generative and Discriminative Voxel Modeling with Convolutional Neural Networks. https://doi.org/10.48550/arXiv.1608.04236 Publication Title: arXiv e-prints ADS Bibcode: 2016arXiv160804236B.
  19. Learning Gradient Fields for Shape Generation. In Computer Vision – ECCV 2020. Springer International Publishing, Cham, 364–381. https://doi.org/10.1007/978-3-030-58580-8_22
  20. Başak Çakmak. 2022. Extending design cognition with computer vision and generative deep learning. Master’s thesis. Middle East Technical University.
  21. A Comprehensive Survey on Geometric Deep Learning. IEEE Access 8 (2020), 35929–35949. https://doi.org/10.1109/ACCESS.2020.2975067
  22. Mario Carpo. 2017. The Second Digital Turn: Design Beyond Intelligence. The MIT Press. https://doi.org/10.2307/j.ctt1w0db6f
  23. Stanislas Chaillou. 2020. ArchiGAN: Artificial Intelligence x Architecture. Springer Nature Singapore, Singapore, 117–127. https://doi.org/10.1007/978-981-15-6568-7_8
  24. Stanislas Chaillou. 2022. The Advent of Architectural AI: A Historical Perspective. Birkhäuser, Berlin, Boston, 32–61. https://doi.org/10.1515/9783035624045-005
  25. Gregory J. Chaitin. 1975a. RANDOMNESS AND MATHEMATICAL PROOF. Scientific American 232, 5 (1975), 47–53. http://www.jstor.org/stable/24949798
  26. Gregory J. Chaitin. 1975b. A Theory of Program Size Formally Identical to Information Theory. J. ACM 22, 3 (jul 1975), 329–340. https://doi.org/10.1145/321892.321894
  27. pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Nashville, TN, USA, 5795–5805. https://doi.org/10.1109/CVPR46437.2021.00574
  28. Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings. In Computer Vision – ACCV 2018. Springer International Publishing, Perth, Australia, 100–116. https://doi.org/10.1007/978-3-030-20893-6_7
  29. Z. Chen and H. Zhang. 2019. Learning Implicit Fields for Generative Shape Modeling. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 5932–5941. https://doi.org/10.1109/CVPR.2019.00609
  30. VRoamer: Generating On-The-Fly VR Experiences While Walking inside Large, Unknown Real-World Building Environments. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Osaka, Japan, 359–366. https://doi.org/10.1109/VR.2019.8798074
  31. Mollie Claypool. 2019. Discrete automation - architecture - e-flux. https://www.e-flux.com/architecture/becoming-digital/248060/discrete-automation/
  32. Generative Adversarial Networks: An Overview. IEEE Signal Processing Magazine 35, 1 (2018), 53–65. https://doi.org/10.1109/MSP.2017.2765202
  33. Diffusion Models in Vision: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 9 (2023), 10850–10869. https://doi.org/10.1109/TPAMI.2023.3261988
  34. A Survey of Deep Learning and Its Applications: A New Paradigm to Machine Learning. Archives of Computational Methods in Engineering 27, 4 (Sept. 2020), 1071–1092. https://doi.org/10.1007/s11831-019-09344-w
  35. When neighboring disciplines fail to learn from each other: The case of innovation and project management research. Research Policy 47, 5 (2018), 965–979. https://doi.org/10.1016/j.respol.2018.03.002
  36. Matias Del Campo. 2021. Architecture, Language and AI-Language, Attentional Generative Adversarial Networks (AttnGAN) and Architecture Design. In Proceedings of the 26th CAADRIA Conference. Hong Kong, 211–220. https://doi.org/10.52842/conf.caadria.2021.1.211
  37. The Church of AI—An examination of architecture in a posthuman design ecology. In Proceedings of the 24th CAADRIA Conference. Wellington, New Zealand, 15–18. https://doi.org/10.52842/conf.caadria.2019.2.767
  38. Generali Center vienna austria. https://caadria2022.org/projects/generali-center-vienna-austria/
  39. Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence 35, 2 (May 2021), 1201–1209. https://doi.org/10.1609/aaai.v35i2.16207
  40. Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondence. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA, 10281–10291. https://doi.org/10.1109/CVPR46437.2021.01015
  41. Generating 3D Building Volumes for a Given Urban Context using Pix2Pix GAN. In Proceedings of the 40th Conference on Education and Research in Computer Aided Architectural Design in Europe. Ghent, Belgium, 287–295. https://doi.org/10.52842/conf.ecaade.2022.2.287
  42. The Spatial Environment Affects Human Emotion Perception-Using Physiological Signal Modes. In Proceedings of the 27th CAADRIA Conference. Sydney, Australia, 425–434. https://doi.org/10.52842/conf.caadria.2022.2.425
  43. Exploration of Architectural Design in the Co-construction Mode of User and Designer in the Metaverse Environment. In 2022 7th International Conference on Systems, Control and Communications. ACM, Chongqing China, 55–60. https://doi.org/10.1145/3575828.3575838
  44. NICE: Non-linear Independent Components Estimation. https://doi.org/10.48550/arXiv.1410.8516 arXiv:1410.8516 [cs.LG]
  45. Nancy Diniz. 2019. Body Architectures - Real time data visualization and responsive immersive environments. In Sousa, JP, Xavier, JP and Castro Henriques, G (eds.), Architecture in the Age of the 4th Industrial Revolution - Proceedings of the 37th eCAADe and 23rd SIGraDi Conference - Volume 2, University of Porto, Porto, Portugal, 11-13 September 2019, pp. 739-746. CUMINCAD, Porto, Portugal. https://doi.org/10.52842/conf.ecaade.2019.2.739
  46. 3D Virtual worlds and the metaverse: Current status and future possibilities. ACM Comput. Surv. 45, 3, Article 34 (jul 2013), 38 pages. https://doi.org/10.1145/2480741.2480751
  47. A Survey on Deep Learning and Its Applications. Computer Science Review 40 (2021), 100379. https://doi.org/10.1016/j.cosrev.2021.100379
  48. Virtual Aesthetics in Architecture: Designing in Mixed Realities (1 ed.). Routledge, New York. https://doi.org/10.4324/9781003183105
  49. Adrian Forty and Adrian Forty. 2000. Words and buildings: A vocabulary of modern architecture. Vol. 268. Thames & Hudson London.
  50. ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model. arXiv:2207.09446 [cs.CV]
  51. GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images. https://doi.org/10.48550/arXiv.2209.11163 arXiv:2209.11163 [cs.CV]
  52. TM-NET: deep generative networks for textured meshes. ACM Trans. Graph. 40, 6, Article 263 (dec 2021), 15 pages. https://doi.org/10.1145/3478513.3480503
  53. SDM-NET: deep generative network for structured deformable mesh. ACM Trans. Graph. 38, 6, Article 243 (nov 2019), 15 pages. https://doi.org/10.1145/3355089.3356488
  54. Learning Shape Templates With Structured Implicit Functions. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, South Korea, 7153–7163. https://doi.org/10.1109/ICCV.2019.00725
  55. A Comprehensive Survey and Analysis of Generative Models in Machine Learning. Computer Science Review 38 (2020), 100285. https://doi.org/10.1016/j.cosrev.2020.100285
  56. StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis. https://doi.org/10.48550/arXiv.2110.08985 arXiv:2110.08985 [cs.CV]
  57. George Guida. 2023. Multimodal Architecture: Applications of Language in a Machine Learning Aided Design Process. In Proceedings of the 28th CAADRIA Conference. Ahmedabad, India, 561–570. https://doi.org/10.52842/conf.caadria.2023.2.561
  58. Improved training of wasserstein GANs. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS’17). Curran Associates Inc., Red Hook, NY, USA, 5769–5779. https://doi.org/10.5555/3295222.3295327
  59. Deep Learning for 3D Point Clouds: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 12 (2021), 4338–4364. https://doi.org/10.1109/TPAMI.2020.3005434
  60. The method of responsive shape design based on real-time interaction process. In Proceedings of the 26th CAADRIA Conference. Hong Kong, 345–354. https://doi.org/10.52842/conf.caadria.2021.2.345
  61. DualSDF: Semantic Shape Manipulation Using a Two-Level Representation. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA, 7628–7638. https://doi.org/10.1109/CVPR42600.2020.00765
  62. William Grant Hatcher and Wei Yu. 2018. A Survey of Deep Learning: Platforms, Applications and Emerging Research Trends. IEEE Access 6 (2018), 24411–24432. https://doi.org/10.1109/ACCESS.2018.2830661
  63. Escaping Plato’s Cave: 3D Shape From Adversarial Rendering. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 9983–9992. https://doi.org/10.1109/ICCV.2019.01008
  64. Use of Immersive Virtual Environments to Understand Human-Building Interactions and Improve Building Design. In HCI International 2015 - Posters’ Extended Abstracts. Springer International Publishing, Cham, 180–184. https://doi.org/10.1007/978-3-319-21380-4_32
  65. Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS’20). Curran Associates Inc., Red Hook, NY, USA, Article 574, 12 pages. https://doi.org/10.5555/3495724.3496298
  66. The Impact of Moving through the Built Environment on Emotional and Neurophysiological State - A Systematic Literature Review. In Proceedings of the 25th CAADRIA Conference. Bangkok, Thailand, 641–650. https://doi.org/10.52842/conf.caadria.2020.1.641
  67. State-Of-The-Art on Research and Applications of Machine Learning in the Building Life Cycle. Energy and Buildings 212 (2020), 109831. https://doi.org/10.1016/j.enbuild.2020.109831
  68. On GANs, NLP and Architecture: Combining Human and Machine Intelligences for the Generation and Evaluation of Meaningful Designs. Technology|Architecture + Design 5, 2 (2021), 207–224. https://doi.org/10.1080/24751448.2021.1967060 Publisher: Routledge _eprint: https://doi.org/10.1080/24751448.2021.1967060.
  69. (In)Visible Cities: Exploring generative artificial intelligence’s creativity through the analysis of a conscious journey in latent space.
  70. Progressive Point Cloud Deconvolution Generation Network. In Computer Vision – ECCV 2020. Springer International Publishing, Cham, 397–413. https://doi.org/10.1007/978-3-030-58555-6_24
  71. 3D Shape Generation with Grid-based Implicit Functions. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA, 13554–13563. https://doi.org/10.1109/CVPR46437.2021.01335
  72. Building Virtual Cities: Applying Urban Planning Principles to the Design of Virtual Environments. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (Hong Kong) (VRST ’96). Association for Computing Machinery, New York, NY, USA, 83–91. https://doi.org/10.1145/3304181.3304199
  73. A Survey on Generative Adversarial Networks: Variants, Applications, and Training. ACM Comput. Surv. 54, 8, Article 157 (oct 2021), 49 pages. https://doi.org/10.1145/3463475
  74. Semantics-Guided Latent Space Exploration for Shape Generation. Computer Graphics Forum 40, 2 (2021), 115–126. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.142619
  75. Zero-Shot Text-Guided Object Generation with Dream Fields. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, Louisiana, USA, 857–866. https://doi.org/10.1109/CVPR52688.2022.00094
  76. Serlio and Artificial Intelligence: Problematizing the Image-to-Object Workflow. In Proceedings of the 2021 DigitalFUTURES. Singapore, 3–12. https://doi.org/10.1007/978-981-16-5983-6_1
  77. Sun-Young Jang and Sung-Ah Kim. 2024. Automatic generation of virtual architecture using user activities in metaverse. International Journal of Human-Computer Studies 182 (2024), 103163. https://doi.org/10.1016/j.ijhcs.2023.103163
  78. David H. Jonassen and Lucia Rohrer-Murphy. 1999. Activity Theory as a Framework for Designing Constructivist Learning Environments. Educational Technology Research and Development 47, 1 (March 1999), 61–79. https://doi.org/10.1007/BF02299477
  79. Damjan Jovanovic. 2022. Games and Worldmaking. https://journal.b-pro.org/article/p3-games-and-worldmaking/
  80. Augmenting Design. In Proceedings of the 41th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA). Online and Global, 112–121. https://doi.org/10.52842/conf.acadia.2021.112
  81. A Style-Based Generator Architecture for Generative Adversarial Networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 4396–4405. https://doi.org/10.1109/CVPR.2019.00453
  82. Aysegul Akcay Kavakoglu. 2021. Computational Aesthetics of Low Poly: [Re]Configuration of Form. In Blucher Design Proceedings. online, 17–28. https://doi.org/10.5151/sigradi2021-235
  83. Creating Immersive Virtual Environments Based on Open Geospatial Data and Game Engines. KN - Journal of Cartography and Geographic Information 71, 1 (March 2021), 53–65. https://doi.org/10.1007/s42489-020-00069-6
  84. A survey of the recent architectures of deep convolutional neural networks. Artificial intelligence review 53, 8 (dec 2020), 5455–5516. https://doi.org/10.1007/s10462-020-09825-6
  85. PlacemakingAI : Participatory Urban Design with Generative Adversarial Networks. In Proceedings of the 27th CAADRIA Conference. Sydney, Australia, 485–494. https://doi.org/10.52842/conf.caadria.2022.2.485
  86. Frederick Chando Kim and Jeffrey Huang. 2022. Perspectival GAN - Architectural form-making through dimensional transformation. In Proceedings of the 40st Conference on Education and Research in Computer Aided Architectural Design in Europe (eCAADe). Ghent, Belgium, 341–350. https://doi.org/10.52842/conf.ecaade.2022.1.341
  87. CityCraft: 3D Virtual City Creation from a Single Image. The Visual Computer 36, 5 (May 2020), 911–924. https://doi.org/10.1007/s00371-019-01701-x
  88. Diederik P. Kingma and Max Welling. 2019. An Introduction to Variational Autoencoders. Foundations and Trends® in Machine Learning 12, 4 (2019), 307–392. https://doi.org/10.1561/2200000056
  89. Adversarial Generation of Continuous Implicit Shape Representations. arXiv:2002.00349 [cs.CV]
  90. Immanuel Koh. 2022. 3D-Gan-Housing (neural sampling series). https://caadria2022.org/projects/3d-gan-housing-neural-sampling-series/
  91. Machine Learning Operations (MLOps): Overview, Definition, and Architecture. IEEE Access 11 (2023), 31866–31879. https://doi.org/10.1109/ACCESS.2023.3262138
  92. Participatory Design Supported with Design System and Augmented Reality. In Proceedings of the 35th eCAADe Conference. Rome, Italy, 745–754. https://doi.org/10.52842/conf.ecaade.2017.2.745
  93. Tencent AI Lab. 2023. AI enhanced procedural city generation. https://gdcvault.com/play/1028921/Recorded-AI-Enhanced-Procedural-City
  94. Towards Augmented Reality Driven Human-City Interaction: Current Research on Mobile Headsets and Future Challenges. ACM Comput. Surv. 54, 8, Article 165 (oct 2021), 38 pages. https://doi.org/10.1145/3467963
  95. All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda. https://doi.org/10.48550/arXiv.2110.05352 arXiv:2110.05352 [cs.CY]
  96. Lawrence Lessig. 1999. Code and Other Laws of Cyberspace. Basic Books, Inc., USA.
  97. Learning Part Generation and Assembly for Structure-Aware Shape Synthesis. Proceedings of the AAAI Conference on Artificial Intelligence 34, 07 (Apr. 2020), 11362–11369. https://doi.org/10.1609/aaai.v34i07.6798
  98. Chaohe Lin and Tian Tian Lo. 2021. Expanding the Methods of Human-VR Interaction (HVRI) for Architectural Design Process. In Proceedings of the 26th CAADRIA Conference. Hong Kong, 173–182. https://doi.org/10.52842/conf.caadria.2021.2.173
  99. Magic3D: High-Resolution Text-to-3D Content Creation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada, 300–309.
  100. Deformable Shape Completion with Graph Convolutional Autoencoders. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, Utah, USA, 1886–1895. https://doi.org/10.1109/CVPR.2018.00202
  101. Pipes of AI – Machine Learning Assisted 3D Modeling Design. In Proceedings of the 2020 DigitalFUTURES. Springer Singapore, Singapore, 17–26. https://doi.org/10.1007/978-981-33-4400-6_2
  102. An Anonymous Ccomposition: Design Optimization Through Machine Learning Algorithm. In Proceedings of the 39th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA). Austin (Texas), USA, 404–411. https://doi.org/10.52842/conf.acadia.2019.404
  103. Interactive 3D Modeling with a Generative Adversarial Network. 2017 International Conference on 3D Vision (3DV) (2017), 126–134. https://api.semanticscholar.org/CorpusID:1360152
  104. Towards Implicit Text-Guided 3D Shape Generation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, Louisiana, USA, 17875–17885. https://doi.org/10.1109/CVPR52688.2022.01737
  105. William E. Lorensen and Harvey E. Cline. 1987. Marching Cubes: A High Resolution 3D Surface Construction Algorithm. SIGGRAPH Comput. Graph. 21, 4 (aug 1987), 163–169. https://doi.org/10.1145/37402.37422
  106. Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data. https://doi.org/10.48550/arXiv.2002.12674 arXiv:2002.12674 [cs.CV]
  107. Occupancy Networks: Learning 3D Reconstruction in Function Space. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 4455–4465. https://doi.org/10.1109/CVPR.2019.00459
  108. Deep Level Sets: Implicit Surface Representations for 3D Shape Inference. https://doi.org/10.48550/arXiv.1901.06802 arXiv:1901.06802 [cs.CV]
  109. Deep Form Finding Using Variational Autoencoders for deep form finding of structural typologies. In Blucher Design Proceedings. Editora Blucher, Porto, Portugal, 71–80. https://doi.org/10.5151/proceedings-caadesigradi2019_514
  110. Generation of Geometric Interpolations of Building types with Deep Variational Autoencoders. Design Science 6 (2020), e34. https://doi.org/10.1017/dsj.2020.31
  111. NeRF: representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 1 (dec 2021), 99–106. https://doi.org/10.1145/3503250
  112. AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 306–315. https://doi.org/10.1109/CVPR52688.2022.00040
  113. Hybrid Elevations using GAN Networks. In Proceedings of the 39th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA). Austin (Texas), USA, 370–379. https://doi.org/10.52842/conf.acadia.2019.370
  114. A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems. ACM Trans. Interact. Intell. Syst. 11, 3–4, Article 24 (sep 2021), 45 pages. https://doi.org/10.1145/3387166
  115. Lisa-Marie Mueller. 2023. 3D Generative Adversarial Networks to Autonomously Generate Building Geometry. Master’s thesis. TU Delft. https://repository.tudelft.nl/islandora/object/uuid:b4a44f69-e3a4-4a29-85c4-77a7e899b81a?collection=education
  116. Joschka Mütterlein. 2018. The Three Pillars of Virtual Reality? Investigating the Roles of Immersion, Presence, and Interactivity. In Hawaii International Conference on System Sciences. Hawaii. https://doi.org/10.24251/HICSS.2018.174
  117. Taro Narahara. 2022. Kurashiki Viewer: Qualitative Evaluations of Architectural Spaces inside Virtual Reality. In Proceedings of the 27th CAADRIA Conference. Sydney, Australia, 11–18. https://doi.org/10.52842/conf.caadria.2022.1.011
  118. PolyGen: An Autoregressive Generative Model of 3D Meshes. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119). PMLR, online, 7220–7229. https://proceedings.mlr.press/v119/nash20a.html
  119. Alina Nazmeeva. 2019. Constructing the virtual as a social form. Ph. D. Dissertation. Massachusetts Institute of Technology.
  120. Julien Nembrini and Denis Lalanne. 2017. Human-Building Interaction: When the Machine Becomes a Building. In Human-Computer Interaction - INTERACT 2017. Springer International Publishing, Cham, 348–369. https://doi.org/10.1007/978-3-319-67684-5_21
  121. Kim J. L. Nevelsteen. 2018. Virtual World, Defined from a Technological Oerspective and Applied to Video Games, Mixed Reality, and the Metaverse. Computer Animation and Virtual Worlds 29, 1 (2018), e1752. https://doi.org/10.1002/cav.1752
  122. David Newton. 2019. Generative Deep Learning in Architectural Design. Technology—Architecture + Design 3, 2 (2019), 176–189. arXiv:https://doi.org/10.1080/24751448.2019.1640536
  123. KOALA - Developing a Generative House Design System with Agent-Based Modelling of Social Spatial Processes. In Proceedings of the 24th CAADRIA Conference. Wellington, New Zealand, 235–244. https://doi.org/10.52842/conf.caadria.2019.1.235
  124. HoloGAN: Unsupervised Learning of 3D Representations From Natural Images. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 7587–7596. https://doi.org/10.1109/ICCV.2019.00768
  125. Point-E: A System for Generating 3D Point Clouds from Complex Prompts. https://doi.org/10.48550/arXiv.2212.08751 arXiv:2212.08751 [cs.CV]
  126. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models. In Proceedings of the 39th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 162). PMLR, Maryland, USA, 16784–16804. https://proceedings.mlr.press/v162/nichol22a.html
  127. Michael Niemeyer and Andreas Geiger. 2021. GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 11448–11459. https://doi.org/10.1109/CVPR46437.2021.01129
  128. On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models. Proceedings of the AAAI Conference on Artificial Intelligence 34, 04 (Apr. 2020), 5272–5280. https://doi.org/10.1609/aaai.v34i04.5973
  129. Ondrej Vesel‘y. 2022. Building massing generation using GAN trained on Dutch 3D city models. Master’s thesis. TU Delft. https://repository.tudelft.nl/islandora/object/uuid:27085fd4-654a-4748-92d0-61563fe6040c?collection=education
  130. OpenAI. 2022. CHATGPT: Optimizing language models for dialogue. https://openai.com/blog/chatgpt/
  131. StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 13493–13503. https://doi.org/10.1109/CVPR52688.2022.01314
  132. Achraf Oussidi and Azeddine Elhassouny. 2018. Deep generative models: Survey. In 2018 International Conference on Intelligent Systems and Computer Vision (ISCV). Fez, Morocco, 1–8. https://doi.org/10.1109/ISACV.2018.8354080
  133. Güvenç Özel. 2020. Interdisciplinary AI: A Machine Learning System for Streamlining External Aesthetic and Cultural Influences in Architecture. Singapore, 103–116. https://doi.org/10.1007/978-981-15-6568-7_7
  134. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery 88 (2021), 105906. https://doi.org/10.1016/j.ijsu.2021.105906
  135. M. R. Pavan Kumar and Prabhu Jayagopal. 2021. Generative adversarial networks: a survey on applications and challenges. International Journal of Multimedia Information Retrieval 10, 1 (March 2021), 1–24. https://doi.org/10.1007/s13735-020-00196-w
  136. Detecting Virtual Perception Based on Multi-Dimensional Biofeedback - A Method to Pre-Evaluate Architectural Design Objectives. In Proceedings of the 26th CAADRIA Conference. Hong Kong, 183–192. https://doi.org/10.52842/conf.caadria.2021.2.183
  137. A Biofeedback Process: Detecting Architectural Space with the Integration of Emotion Recognition and Eye-tracking Technology. In Proceedings of the 25th CAADRIA Conference. Bangkok, Thailand, 263–272. https://doi.org/10.52842/conf.caadria.2020.2.263
  138. Drew D. Penney and Lizhong Chen. 2019. A Survey of Machine Learning Applied to Computer Architecture Design. https://doi.org/10.48550/arXiv.1909.12373 arXiv:1909.12373 [cs.AR]
  139. Automated 3D reconstruction of LoD2 and LoD1 models for all 10 million buildings of the Netherlands. , 165–170 pages. https://doi.org/10.14358/PERS.21-00032R2
  140. DreamFusion: Text-to-3D using 2D Diffusion. https://doi.org/10.48550/arXiv.2209.14988 arXiv:2209.14988 [cs.CV]
  141. A Survey on Deep Learning: Algorithms, Techniques, and Applications. ACM Comput. Surv. 51, 5, Article 92 (sep 2018), 36 pages. https://doi.org/10.1145/3234150
  142. Spectral-GANs for High-Resolution 3D Point-cloud Generation. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Las Vegas, NV, USA). IEEE Press, Las Vegas, NV, USA, 8169–8176. https://doi.org/10.1109/IROS45743.2020.9341265
  143. Deep Generative Models in Engineering Design: A Review. Journal of Mechanical Design 144, 7 (03 2022), 071704. https://doi.org/10.1115/1.4053859
  144. Yue Ren and Hao Zheng. 2020. The Spire of AI - Voxel-based 3D Neural Style Transfer. In Proceedings of the 25th CAADRIA Conference. Bangkok, Thailand, 619–628. https://doi.org/10.52842/conf.caadria.2020.2.619
  145. Radu Bogdan Rusu and Steve B. Cousins. 2011. 3D is here: Point Cloud Library (PCL). 2011 IEEE International Conference on Robotics and Automation (2011), 1–4.
  146. Waddah Saeed and Christian Omlin. 2023. Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities. Knowledge-Based Systems 263 (2023), 110273. https://doi.org/10.1016/j.knosys.2023.110273
  147. CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, Louisiana, USA, 18582–18592. https://doi.org/10.1109/CVPR52688.2022.01805
  148. Victor Sardenberg and Mirco Becker Theron Burger. 2019. Aesthetic Quantification as Search Criteria in Architectural Design. In Blucher Design Proceedings. Editora Blucher, Porto, Portugal, 17–24. https://doi.org/10.5151/proceedings-ecaadesigradi2019_088
  149. Iqbal H. Sarker. 2021a. Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions. SN Computer Science 2, 6 (Aug. 2021), 420. https://doi.org/10.1007/s42979-021-00815-1
  150. Iqbal H. Sarker. 2021b. Machine Learning: Algorithms, Real-World Applications and Research Directions. SN Computer Science 2, 3 (March 2021), 160. https://doi.org/10.1007/s42979-021-00592-x
  151. Activeworlds: geography and social interaction in virtual reality. Futures 33, 7 (2001), 569–587. https://doi.org/10.1016/S0016-3287(01)00002-7
  152. Towards Abductive Reasoning-Based Computational Design Tools - Using Machine Learning as a way to explore the combined design spaces of multiple parametric models. In Proceedings of the 39th eCAADe Conference. Novi Sad, Serbia, 141–150. https://doi.org/10.52842/conf.ecaade.2021.1.141
  153. BuildingNet: Learning to Label 3D Buildings. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). online, 10377–10387. https://doi.org/10.1109/ICCV48922.2021.01023
  154. The Fourth Virtual Dimension - Stimulating the Human Senses to Create Virtual Atmospheric Qualities. In Proceedings of the 26th CAADRIA Conference. Hong Kong, 213–222. https://doi.org/10.52842/conf.caadria.2021.2.213
  155. Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis. In Advances in Neural Information Processing Systems, Vol. 34. Curran Associates, Inc., online, 6087–6101. https://proceedings.neurips.cc/paper_files/paper/2021/file/30a237d18c50f563cba4531f1db44acf-Paper.pdf
  156. Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Literature Review. Proc. ACM Hum.-Comput. Interact. 7, CSCW2, Article 368 (oct 2023), 35 pages. https://doi.org/10.1145/3610217
  157. Deep Generative Models on 3D Representations: A Survey. https://doi.org/10.48550/arXiv.2210.15663 arXiv:2210.15663 [cs.CV]
  158. 3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 3858–3867. https://doi.org/10.1109/ICCV.2019.00396
  159. Infrastructuring and Participatory Design: Exploring Infrastructural Inversion as Analytic, Empirical and Generative. Comput. Supported Coop. Work 29, 1–2 (apr 2020), 115–151. https://doi.org/10.1007/s10606-019-09365-w
  160. Fresh Eyes: A Framework for the Application of Machine Learning to Generative Architectural Design, and a Report of Activities at Smartgeometry 2018. In Computer-Aided Architectural Design.” Hello, Culture” 18th International Conf., CAAD Futures 2019, Daejeon, S. Korea, June 26–28, 2019. Springer, Daejeon, Souoth Korea, 32–46. https://doi.org/10.1007/978-981-13-8410-3_3
  161. Rethinking Computer-Aided Architectural Design (CAAD) – From Generative Algorithms and Architectural Intelligence to Environmental Design and Ambient Intelligence. In Computer-Aided Architectural Design. Design Imperatives: The Future is Now. Springer Singapore, Singapore, 62–83. https://doi.org/10.1007/978-981-19-1280-1_5
  162. Robert Stuart-Smith and Patrick Danahy. 2022. Visual Character Analysis within Algorithmic Design: Quantifying Aesthetics Relative to Structural and Geometric Design Criteria. In Proceedings of the 27th CAADRIA Conference. Sydney, Australia, 131–140. https://doi.org/10.52842/conf.caadria.2022.1.131
  163. Machine learning for architectural design: Practices and infrastructure. International Journal of Architectural Computing 16, 2 (2018), 123–143. arXiv:https://doi.org/10.1177/1478077118778580
  164. Christian Tonn. 2017. Designing Colour in Virtual Reality - Comparing a Virtual Reality based and a Screen based Colour Design Method. In Proceedings of the 35th eCAADe Conference. Rome, Italy. https://doi.org/10.52842/conf.ecaade.2017.2.721
  165. Maria E. Tosello. 2003. Performing Cyberspace: Dance, Technology and Virtual Architecture. International Journal of Architectural Computing 1, 3 (2003), 393–413. arXiv:https://doi.org/10.1260/147807703322987129
  166. CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, Louisiana, USA, 3825–3834. https://doi.org/10.1109/CVPR52688.2022.00381
  167. Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy. ACM Comput. Surv. 54, 2, Article 37 (feb 2021), 38 pages. https://doi.org/10.1145/3439723
  168. Beauty is in the Eye of the Beholder - Improving the Human-Computer Interface within VRAD by the Active and Two-Way Employment of Our Visual Senses. In Proceedings of the 26th CAADRIA Conference. Hong Kong, 355–364. https://doi.org/10.52842/conf.caadria.2021.2.355
  169. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In Proceedings of the 30th International Conference on Neural Information Processing Systems (Barcelona, Spain) (NIPS’16). Curran Associates Inc., Red Hook, NY, USA, 82–90. https://doi.org/10.5555/3157096.3157106
  170. PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA, 826–835. https://doi.org/10.1109/CVPR42600.2020.00091
  171. SAGNet: structure-aware generative network for 3D-shape modeling. ACM Trans. Graph. 38, 4, Article 91 (jul 2019), 14 pages. https://doi.org/10.1145/3306346.3322956
  172. Weihao Xia and Jing-Hao Xue. 2023. A Survey on Deep Generative 3D-aware Image Synthesis. ACM Comput. Surv. 56, 4, Article 90 (nov 2023), 34 pages. https://doi.org/10.1145/3626193
  173. PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, Seoul, South Korea, 4540–4549. https://doi.org/10.1109/ICCV.2019.00464
  174. 3DCityDB - a 3D geodatabase solution for the management, analysis, and visualization of semantic 3D city models based on CityGML. Open Geospatial Data, Software and Standards 3, 1 (May 2018), 5. https://doi.org/10.1186/s40965-018-0046-7
  175. De Yu. 2020. Reprogramming Urban Block by Machine Creativity - How to use neural networks as generative tools to design space. In Proceedings of the 38st Conference on Education and Research in Computer Aided Architectural Design in Europe (eCAADe). Berlin, Germany, 249–258. https://doi.org/10.52842/conf.ecaade.2020.1.249
  176. Anny Yuniarti and Nanik Suciati. 2019. A Review of Deep Learning Techniques for 3D Reconstruction of 2D Images. In 2019 12th International Conference on Information & Communication Technology and System (ICTS). Surabaya, Indonesia, 327–331. https://doi.org/10.1109/ICTS.2019.8850991
  177. Adversarial autoencoders for compact representations of 3D point clouds. Comput. Vis. Image Underst. 193, C (apr 2020), 8 pages. https://doi.org/10.1016/j.cviu.2020.102921
  178. Design and Development of Interactive Systems for Integration of Comparative Visual Analytics in Design Workflow. In Proceedings of the 26th CAADRIA Conference. Hong Kong, 121–130. https://doi.org/10.52842/conf.caadria.2021.2.121
  179. LION: Latent Point Diffusion Models for 3D Shape Generation. https://doi.org/10.48550/arXiv.2210.06978 arXiv:2210.06978 [cs.CV]
  180. Hang Zhang. 2019. 3D Model Generation on Architectural Plan and Section Training through Machine Learning. Technologies 7, 4 (2019). https://doi.org/10.3390/technologies7040082
  181. Hang Zhang. 2020. Text-to-Form: 3D Prediction by Linguistic Descriptions. In Proceedings of the 40th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA). Online and Global, 238–247. https://doi.org/10.52842/conf.acadia.2020.1.238
  182. Hang Zhang and Ye Huang. 2021. Machine Learning Aided 2D-3D Architectural Form Finding at High Resolution. Springer, Singapore, 159–168. https://doi.org/10.1007/978-981-33-4400-6_15
  183. A Survey of 3D Indoor Scene Synthesis. Journal of Computer Science and Technology 34, 3 (May 2019), 594–608. https://doi.org/10.1007/s11390-019-1929-5
  184. Hao Zheng and Philip F Yuan. 2021. A Generative Architectural and Urban Design Method through Artificial Neural Networks. Building and Environment 205 (2021), 108178. https://doi.org/10.1016/j.buildenv.2021.108178
  185. Deep Implicit Templates for 3D Shape Representation. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA, 1429–1439. https://doi.org/10.1109/CVPR46437.2021.00148
  186. 3D Shape Generation and Completion through Point-Voxel Diffusion. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). online, 5806–5815. https://doi.org/10.1109/ICCV48922.2021.00577
  187. Gizem Özerol and Semra Arslan Selçuk. 2023. Machine Learning in the Discipline of Architecture: A Review on the Research Trends between 2014 and 2020. International Journal of Architectural Computing 21, 1 (2023), 23–41. https://doi.org/10.1177/14780771221100102
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Anqi Wang (26 papers)
  2. Jiahua Dong (48 papers)
  3. Lik-Hang Lee (47 papers)
  4. Jiachuan Shen (1 paper)
  5. Pan Hui (155 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets