Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Visualization of AI Systems in Virtual Reality: A Comprehensive Review (2306.15545v1)

Published 27 Jun 2023 in cs.HC

Abstract: This study provides a comprehensive review of the utilization of Virtual Reality (VR) for visualizing AI systems, drawing on 18 selected studies. The results illuminate a complex interplay of tools, methods, and approaches, notably the prominence of VR engines like Unreal Engine and Unity. However, despite these tools, a universal solution for effective AI visualization remains elusive, reflecting the unique strengths and limitations of each technique. We observed the application of VR for AI visualization across multiple domains, despite challenges such as high data complexity and cognitive load. Moreover, it briefly discusses the emerging ethical considerations pertaining to the broad integration of these technologies. Despite these challenges, the field shows significant potential, emphasizing the need for dedicated research efforts to unlock the full potential of these immersive technologies. This review, therefore, outlines a roadmap for future research, encouraging innovation in visualization techniques, addressing identified challenges, and considering the ethical implications of VR and AI convergence.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Caffe2unity: Immersive visualization and interpretation of deep neural networks. Electronics (Switzerland), 11, 2022. ISSN 20799292. doi: 10.3390/electronics11010083.
  2. An immersive node-link visualization of artificial neural networks for machine learning experts. In 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), pages 33–36, 2020. doi: 10.1109/AIVR50618.2020.00015.
  3. Neural network design using a virtual reality platform. Global Journal of Computer Science and Technology, 2022. ISSN 09754350. doi: 10.34257/gjcstdvol22is1pg45.
  4. Virtual reality-based parallel coordinates plots enhanced with explainable AI and data-science analytics for decision-making processes. Applied Sciences, 12(1):331, Dec 2021. ISSN 2076-3417. doi: 10.3390/app12010331.
  5. Visualization of neural networks in virtual reality using unreal engine. 2018. doi: 10.1145/3281505.3281605.
  6. A survey of surveys on the use of visualization for interpreting machine learning models. Information Visualization, 19(3):207–233, 2020. doi: 10.1177/1473871620904671.
  7. Immersive and collaborative data visualization using virtual reality platforms. In 2014 IEEE International Conference on Big Data (Big Data), pages 609–614, 2014. doi: 10.1109/BigData.2014.7004282.
  8. Anne-Wil Harzing. Publish or perish. Online, 2007. URL https://harzing.com/resources/publish-or-perish.
  9. On the use of virtual reality in visualizing interactive network models for big data. In 2022 8th International Conference on Contemporary Information Technology and Mathematics (ICCITM), pages 20–24, 2022. doi: 10.1109/ICCITM56309.2022.10031631.
  10. Decoding pedestrian and automated vehicle interactions using immersive virtual reality and interpretable deep learning. Transportation Research Part C: Emerging Technologies, 124:102962, 2021. ISSN 0968-090X. doi: 10.1016/j.trc.2020.102962. URL https://www.sciencedirect.com/science/article/pii/S0968090X2030855X.
  11. Visualization in virtual reality: a systematic review. Virtual Reality, pages 1–34, 2023.
  12. VR-LENS: Super learning-based cybersickness detection and explainable AI-guided deployment in virtual reality, 2023.
  13. Submicroscopic and physics simulation of autonomous and intelligent vehicles in virtual reality. In 2010 Second International Conference on Advances in System Simulation, pages 28–33, 2010. doi: 10.1109/SIMUL.2010.19.
  14. Explaining the black-box model: A survey of local interpretation methods for deep neural networks. Neurocomputing, 419:168–182, 2021. ISSN 0925-2312. doi: https://doi.org/10.1016/j.neucom.2020.08.011.
  15. A walk in the black-box: 3D visualization of large neural networks in virtual reality. Neural Computing and Applications, 34(23):21237–21252, 2022. doi: 10.1007/s00521-022-07608-4.
  16. Editors’ note: bibliometrics and the curators of orthodoxy. Mathematical Structures in Computer Science, 19:1–4, February 2009. doi: 10.1017/S0960129508007391.
  17. AIive: Interactive visualization and sonification of neural networks in virtual reality. 2021. doi: 10.1109/AIVR52153.2021.00057.
  18. A virtual simulation environment using deep learning for autonomous vehicles obstacle avoidance. In 2020 IEEE International Conference on Intelligence and Security Informatics (ISI), pages 1–7, 2020. doi: 10.1109/ISI49825.2020.9280513.
  19. Using visualization of convolutional neural networks in virtual reality for machine learning newcomers. 2019. doi: 10.1109/AIVR46125.2019.00031.
  20. Interactive visualization of deep learning models in an immersive environment. In Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology, VRST ’21, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450390927. doi: 10.1145/3489849.3489956.
  21. Survey of the VR environment for deep learning model development. In The 35th Annual Conference of the Japanese Society for Artificial Intelligence (JSAI2021), page 2N3IS2b04, June 2021. doi: 10.11517/pjsai.JSAI2021.0˙2N3IS2b04.
  22. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, 2021. doi: 10.1136/bmj.n71. URL https://www.bmj.com/content/372/bmj.n71.
  23. Neural network visualization in virtual reality: A use case analysis and implementation. In Sakae Yamamoto and Hirohiko Mori, editors, Human Interface and the Management of Information: Visual and Information Design, pages 384–397, Cham, 2022. Springer International Publishing. ISBN 978-3-031-06424-1.
  24. Virtual reality solutions employing artificial intelligence methods: A systematic literature review. ACM Comput. Surv., 55(10), feb 2023. ISSN 0360-0300. doi: 10.1145/3565020. URL https://doi.org/10.1145/3565020.
  25. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296, 2017.
  26. Transparency and trust in artificial intelligence systems. Journal of Decision Systems, 29(4):260–278, 2020.
  27. Visualization and exploration of deep learning networks in 3D and virtual reality. volume 1033, 2019. doi: 10.1007/978-3-030-23528-4˙29.
  28. Toward verified artificial intelligence. Communications of the ACM, 65(7):46–55, 2022.
  29. Artificial neural network in virtual reality : A survey. International Journal of Virtual Reality, 15, 2016. ISSN 1081-1451. doi: 10.20870/ijvr.2016.15.2.2873.
  30. Data and knowledge visualization with virtual reality spaces, neural networks and rough sets: Application to cancer and geophysical prospecting data. Expert Systems with Applications, 39, 2012. ISSN 09574174. doi: 10.1016/j.eswa.2012.05.082.
  31. Democratizing AI in biomedical image classification using virtual reality. Virtual Reality, 26(1):159–171, 2022. doi: 10.1007/s10055-021-00550-1.
  32. Visualizing convolutional neural networks with virtual reality. pages 1–2. ACM, 11 2019. ISBN 9781450370011. doi: 10.1145/3359996.3364817.
  33. Tao Zhang. Research on environmental landscape design based on virtual reality technology and deep learning. Microprocessors and Microsystems, 81, 2021. ISSN 01419331. doi: 10.1016/j.micpro.2020.103796.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Medet Inkarbekov (2 papers)
  2. Rosemary Monahan (20 papers)
  3. Barak A. Pearlmutter (34 papers)
Citations (9)
View on Semantic Scholar