Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 154 tok/s
Gemini 2.5 Pro 40 tok/s Pro
GPT-5 Medium 25 tok/s Pro
GPT-5 High 21 tok/s Pro
GPT-4o 93 tok/s Pro
Kimi K2 170 tok/s Pro
GPT OSS 120B 411 tok/s Pro
Claude Sonnet 4.5 36 tok/s Pro
2000 character limit reached

PanoTree: Autonomous Photo-Spot Explorer in Virtual Reality Scenes (2405.17136v2)

Published 27 May 2024 in cs.CV and cs.GR

Abstract: Social VR platforms enable social, economic, and creative activities by allowing users to create and share their own virtual spaces. In social VR, photography within a VR scene is an important indicator of visitors' activities. Although automatic identification of photo spots within a VR scene can facilitate the process of creating a VR scene and enhance the visitor experience, there are challenges in quantitatively evaluating photos taken in the VR scene and efficiently exploring the large VR scene. We propose PanoTree, an automated photo-spot explorer in VR scenes. To assess the aesthetics of images captured in VR scenes, a deep scoring network is trained on a large dataset of photos collected by a social VR platform to determine whether humans are likely to take similar photos. Furthermore, we propose a Hierarchical Optimistic Optimization (HOO)-based search algorithm to efficiently explore 3D VR spaces with the reward from the scoring network. Our user study shows that the scoring network achieves human-level performance in distinguishing randomly taken images from those taken by humans. In addition, we show applications using the explored photo spots, such as automatic thumbnail generation, support for VR world creation, and visitor flow planning within a VR scene.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. P. Auer. Using confidence bounds for exploitation-exploration trade-offs. The Journal of Machine Learning Research, 3(3):397–422, Mar 2003.
  2. M. Barreda-Ángeles and T. Hartmann. Psychological benefits of using social virtual reality platforms during the covid-19 pandemic: The role of social and spatial presence. Computers in Human Behavior, 127:107047, 2022.
  3. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599, 2010.
  4. X-armed bandits. arXiv preprint arXiv:1012.2599, 2011.
  5. B. Åšliwecki. Virtual reality architectural spaces and the shift of populace in online social vr platforms in 2020. Architecturae et Artibus, 13(4):1–12, 2021.
  6. Cluster Inc. ”Metaverse Platform - Cluster,” https://cluster.mu/en (accessed Jan. 29, 2024).
  7. RandAugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (CVPR), pages 702–703, 2020.
  8. Image aesthetic assessment: An experimental survey. IEEE Signal Processing Magazine, 34(4):80–106, 2017.
  9. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
  10. Introduction to evolutionary computing. Springer, 2015.
  11. A graph-based approach to detecting tourist movement patterns using social media data. Cartography and Geographic Information Science, 46(4):368–382, 2019.
  12. A survey on deep learning techniques for image and video semantic segmentation. Applied Soft Computing, 70:41–65, 2018.
  13. DreamCodeVR: Towards democratizing behavior design in virtual reality with speech-driven programming. In Proceedings of IEEE International Conference of Virtual Reality 2024, pages 579–589, 2024.
  14. Using social media, machine learning and natural language processing to map multiple recreational beneficiaries. Ecosystem Services, 38:100958, 2019.
  15. Increasing the feeling of social presence by incorporating realistic interactions in multi-party VR. In Proceedings of the 31st International Conference on Computer Animation and Social Agents, pages 7–10, May 2018.
  16. Effective aesthetics prediction with multi-level spatially pooled features. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 9375–9383, 2019.
  17. Extracting and understanding urban areas of interest using geotagged photos. Computers, Environment and Urban Systems, 54:240–254, 2015.
  18. On the global convergence of particle swarm optimization methods. Applied Mathematics & Optimization, 88(2):30, 2023.
  19. Virtual reality in art therapy: A pilot qualitative study of the novel medium and implications for practice. Art Therapy, 37(1):16–24, Jan. 2020.
  20. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023.
  21. Big data in tourism research: A literature review. Tourism Management, 68:301–323, 2018.
  22. Magic3d: High-resolution text-to-3d content creation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  23. BARF: Bundle-adjusting neural radiance fields. In IEEE International Conference on Computer Vision (ICCV), 2021.
  24. Language-driven synthesis of 3d scenes from scene databases. In ACM SIGGRAPH Asia 2018 Technical Papers, page 212, 2018.
  25. G. Makransky and L. Lilleholt. A structural equation modeling investigation of the emotional value of immersive virtual reality in education. Springer Educational Technology Research and Development, 66(5):1141–1164, Oct. 2018.
  26. Shaping pro-social interaction in VR: An emerging design framework. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), page 1–12, New York, NY, USA, 2019.
  27. Interactive furniture layout using interior design guidelines. In ACM SIGGRAPH 2011 Papers, SIGGRAPH ’11, New York, NY, USA, 2011. Association for Computing Machinery.
  28. NeRF: Representing scenes as neural radiance fields for view synthesis. In Proceedings of European Conference on Computer Vision, 2020.
  29. Image segmentation using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3523–3542, 2021.
  30. When does label smoothing help? Advances in Neural Information Processing systems, 32, 2019.
  31. J. A. Nelder and R. Mead. A simplex method for function minimization. The Computer Journal, 7(4):308–313, 1965.
  32. Camp: Camera preconditioning for neural radiance fields. ACM Transactions on Graphics, 2023.
  33. Dreamfusion: Text-to-3d using 2d diffusion. In the Eleventh International Conference on Learning Representations (ICLR), 2023.
  34. M. J. Powell. An efficient method for finding the minimum of a function of several variables without calculating derivatives. The Computer Journal, 7(2):155–162, 1964.
  35. Virtual reality solutions employing artificial intelligence methods: A systematic literature review. ACM Computing Survey, 55(10), feb 2023.
  36. Steps towards prompt-based creation of virtual worlds. arXiv preprint arXiv:2211.05875, 2022.
  37. L. G. Roberts. Machine perception of three-dimensional solids. PhD thesis, Massachusetts Institute of Technology, 1963.
  38. Mastering the game of go without human knowledge. Nature, 550(7676):354–359, 2017.
  39. Semantic scene completion from a single depth image. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 190–198, 2017.
  40. Statista. ”Number of Virtual Reality (VR) and Augmented Reality (AR) Users in the United States From 2017 to 2023.,” https://www.statista.com/statistics/1017008/united-states-vr-ar-users/, (accessed Jan. 30, 2024).
  41. 3d-gpt: Procedural 3d modeling with large language models, 2023.
  42. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. arXiv preprint arXiv:2309.16653, 2023.
  43. Mlp-mixer: An all-mlp architecture for vision. Advances in Neural Information Processing systems, 34:24261–24272, 2021.
  44. LLMR: Real-time prompting of interactive worlds using large language models. arXiv preprint arXiv:2309.12276, 2024.
  45. VRChat Inc. ”VRChat,” https://hello.vrchat.com/ (accessed Jan. 29, 2024).
  46. Deep convolutional priors for indoor scene synthesis. ACM Transactions on Graphics, 37(4), jul 2018.
  47. Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20908–20918, 2023.
  48. Y. Yakimovsky and J. A. Feldman. A semantics-based decision theory region analyser. In IJCAI, volume 73, pages 580–588, 1973.
  49. Yellow Dog Man Studios. ”Resonite,”, https://store.steampowered.com/app/2519830/resonite/ (accessed Jan. 29, 2024).
  50. Text2vrscene: Exploring the framework of automated text-driven generation system for vr experience. In Proceedings of IEEE International Conference of Virtual Reality 2024, page preprint, 2024.
  51. Make it home: automatic optimization of furniture arrangement. ACM Transactions on Graphics, 30(4), jul 2011.
  52. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision (CVPR), pages 6023–6032, 2019.
  53. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017.
  54. Random erasing data augmentation. In Proceedings of the conference on artificial intelligence (AAAI), volume 34, pages 13001–13008, 2020.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Questions

We haven't generated a list of open questions mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 1 tweet and received 0 likes.

Upgrade to Pro to view all of the tweets about this paper:

Start a free 7-day Pro trial