Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

eCAR: edge-assisted Collaborative Augmented Reality Framework (2405.06872v1)

Published 11 May 2024 in cs.CV and cs.RO

Abstract: We propose a novel edge-assisted multi-user collaborative augmented reality framework in a large indoor environment. In Collaborative Augmented Reality, data communication that synchronizes virtual objects has large network traffic and high network latency. Due to drift, CAR applications without continuous data communication for coordinate system alignment have virtual object inconsistency. In addition, synchronization messages for online virtual object updates have high latency as the number of collaborative devices increases. To solve this problem, we implement the CAR framework, called eCAR, which utilizes edge computing to continuously match the device's coordinate system with less network traffic. Furthermore, we extend the co-visibility graph of the edge server to maintain virtual object spatial-temporal consistency in neighboring devices by synchronizing a local graph. We evaluate the system quantitatively and qualitatively in the public dataset and a physical indoor environment. eCAR communicates data for coordinate system alignment between the edge server and devices with less network traffic and latency. In addition, collaborative augmented reality synchronization algorithms quickly and accurately host and resolve virtual objects. The proposed system continuously aligns coordinate systems to multiple devices in a large indoor environment and shares augmented reality content. Through our system, users interact with virtual objects and share augmented reality experiences with neighboring users.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. S. Agarwal and K. Mierle. Ceres solver: Tutorial & reference. Google Inc, 2(72):8, 2012.
  2. Ultra-wideband wireless systems. IEEE microwave magazine, 4(2):36–47, 2003.
  3. Edge-slam: Edge-assisted visual simultaneous localization and mapping. ACM Transactions on Embedded Computing Systems, 22(1):1–31, 2022.
  4. Breaking edge shackles: Infrastructure-free collaborative mobile augmented reality. In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pp. 1–15, 2022.
  5. Apple. Creating a multiuser ar experience, 2018.
  6. Edge-slam: edge-assisted visual simultaneous localization and mapping. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services, pp. 325–337, 2020.
  7. Enveloping users and computers in a collaborative 3d augmented reality. In Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR’99), pp. 35–44. IEEE, 1999.
  8. Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam. IEEE Transactions on Robotics, 37(6):1874–1890, 2021.
  9. Device-to-device communications with wi-fi direct: overview and experimentation. IEEE wireless communications, 20(3):96–104, 2013.
  10. Edge assisted mobile semantic visual slam. IEEE Transactions on Mobile Computing, 2022.
  11. Marvel: Enabling mobile augmented reality with low energy and low latency. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, pp. 292–304, 2018.
  12. An empirical study of latency in an emerging class of edge computing applications for wearable cognitive assistance. In Proceedings of the Second ACM/IEEE Symposium on Edge Computing, pp. 1–14, 2017.
  13. Y. Cheng. Edge caching and computing in 5g for mobile augmented reality and haptic internet. Computer Communications, 158:24–31, 2020.
  14. Bundlefusion: Real-time globally consistent 3d reconstruction using on-the-fly surface reintegration. ACM Transactions on Graphics (ToG), 36(4):1, 2017.
  15. M. De Berg. Computational geometry: algorithms and applications. chap. 15, pp. 323–331. Springer Science & Business Media, 2000.
  16. Slam-share: visual simultaneous localization and mapping for real-time multi-user augmented reality. In Proceedings of the 18th International Conference on emerging Networking EXperiments and Technologies, pp. 293–306, 2022.
  17. N. Didar and M. Brocanelli. ear: an edge-assisted and energy-efficient mobile augmented reality framework. IEEE Transactions on Mobile Computing, 2022.
  18. Direct sparse odometry. IEEE transactions on pattern analysis and machine intelligence, 40(3):611–625, 2017.
  19. Lsd-slam: Large-scale direct monocular slam. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part II 13, pp. 834–849. Springer, 2014.
  20. M. Erol-Kantarci and S. Sukhmani. Caching and computing at the edge for mobile augmented reality and virtual reality (ar/vr) in 5g. In Ad Hoc Networks: 9th International Conference, AdHocNets 2017, Niagara Falls, ON, Canada, September 28–29, 2017, Proceedings, pp. 169–177. Springer, 2018.
  21. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981.
  22. Svo: Fast semi-direct monocular visual odometry. In 2014 IEEE international conference on robotics and automation (ICRA), pp. 15–22. IEEE, 2014.
  23. Google. Working with anchors, 2018.
  24. Google. Cloud anchors allow different users to share ar experiences, 2021.
  25. M. Grinberg. Flask web development: developing web applications with python. ” O’Reilly Media, Inc.”, 2018.
  26. J. K. Haas. A history of the unity game engine. 2014.
  27. Comic: A collaborative mobile immersive computing infrastructure for conducting multi-user xr research. IEEE Network, 2022.
  28. Face to face collaborative ar on mobile phones. In Fourth ieee and acm international symposium on mixed and augmented reality (ismar’05), pp. 80–89. IEEE, 2005.
  29. Curl: a gentle slope language for the web. World wide web journal, 2(2):121–134, 1997.
  30. Edge robotics: Edge-computing-accelerated multirobot simultaneous localization and mapping. IEEE Internet of Things Journal, 9(15):14087–14102, 2022.
  31. Synchronizar: Instant synchronization for spontaneous and spatial collaborations in augmented reality. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pp. 19–30, 2018.
  32. Overlay: Practical mobile augmented reality. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services, pp. 331–344, 2015.
  33. Low bandwidth offload for mobile ar. In Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies, pp. 237–251, 2016.
  34. Virtual object manipulation on a table-top ar environment. In Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000), pp. 111–119. Ieee, 2000.
  35. G. Klein and D. Murray. Parallel tracking and mapping for small ar workspaces. In 2007 6th IEEE and ACM international symposium on mixed and augmented reality, pp. 225–234. IEEE, 2007.
  36. g2o: A general framework for graph optimization. In 2011 IEEE International Conference on Robotics and Automation, pp. 3607–3613. IEEE, 2011.
  37. Edge assisted real-time object detection for mobile augmented reality. In The 25th annual international conference on mobile computing and networking, pp. 1–16, 2019.
  38. Q. Liu and T. Han. Dare: Dynamic adaptive mobile augmented reality with edge computing. In 2018 IEEE 26th International Conference on Network Protocols (ICNP), pp. 1–11. IEEE, 2018.
  39. Meta. Spatial anchors overview, 2021.
  40. Microsoft. Azure spatial anchors documentation, 2019.
  41. Microsoft. Anchor relationships and way-finding in azure spatial anchors, 2022.
  42. Orb-slam: a versatile and accurate monocular slam system. IEEE transactions on robotics, 31(5):1147–1163, 2015.
  43. R. Mur-Artal and J. D. Tardós. Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras. IEEE transactions on robotics, 33(5):1255–1262, 2017.
  44. Kinectfusion: Real-time dense surface mapping and tracking. In 2011 10th IEEE international symposium on mixed and augmented reality, pp. 127–136. Ieee, 2011.
  45. Dtam: Dense tracking and mapping in real-time. In 2011 international conference on computer vision, pp. 2320–2327. IEEE, 2011.
  46. T. Ohshima. Ar hockey: Acase study of collaborative augmented reality. In Proc. Int. Conf. PatternRecognition, vol. 2, pp. 1226–1229, 1998.
  47. Effects of network characteristics on human performance in a collaborative virtual environment. In Proceedings IEEE Virtual Reality (Cat. No. 99CB36316), pp. 104–111. IEEE, 1999.
  48. Avr: Augmented vehicular reality. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, pp. 81–95, 2018.
  49. Sharear: Communication-efficient multi-user mobile augmented reality. In Proceedings of the 18th ACM Workshop on Hot Topics in Networks, pp. 109–116, 2019.
  50. Multi-user augmented reality with communication efficient and spatially consistent virtual objects. In Proceedings of the 16th International Conference on emerging Networking EXperiments and Technologies, pp. 386–398, 2020.
  51. Magicmeeting: A collaborative tangible augmented reality system. Virtual Reality, 6(3):151–166, 2002.
  52. Edge ar x5: An edge-assisted multi-user collaborative framework for mobile web augmented reality in 5g and beyond. IEEE Transactions on Cloud Computing, 10(4):2521–2537, 2020.
  53. C2tam: A cloud framework for cooperative tracking and mapping. Robotics and Autonomous Systems, 62(4):401–413, 2014.
  54. Secure {{\{{Multi-User}}\}} content sharing for augmented reality applications. In 28th USENIX Security Symposium (USENIX Security 19), pp. 141–158, 2019.
  55. Bluetooth in wireless communication. IEEE Communications Magazine, 40(6):90–96, 2002.
  56. P. Schmuck and M. Chli. Multi-uav collaborative monocular slam. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3863–3870. IEEE, 2017.
  57. P. Schmuck and M. Chli. Ccm-slam: Robust and efficient centralized collaborative monocular simultaneous localization and mapping for robotic teams. Journal of Field Robotics, 36(4):763–781, 2019.
  58. A novel joint mobile cache and power management scheme for energy-efficient mobile augmented reality service in mobile edge computing. IEEE Wireless Communications Letters, 10(5):1061–1065, 2021.
  59. A benchmark for the evaluation of rgb-d slam systems. In Proc. of the International Conference on Intelligent Robot Systems (IROS), Oct. 2012.
  60. “studierstube”: An environment for collaboration in augmented reality. Virtual Reality, 3(1):37–48, 1998.
  61. Bundle adjustment—a modern synthesis. In Vision Algorithms: Theory and Practice: International Workshop on Vision Algorithms Corfu, Greece, September 21–22, 1999 Proceedings, pp. 298–372. Springer, 2000.
  62. Depth from motion for smartphone ar. ACM Transactions on Graphics (ToG), 37(6):1–19, 2018.
  63. Edge assisted mobile semantic visual slam. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications, pp. 1828–1837. IEEE, 2020.
  64. Latency-aware hybrid edge cloud framework for mobile augmented reality applications. In 2020 17th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), pp. 1–9. IEEE, 2020.
  65. Jaguar: Low latency mobile augmented reality with flexible tracking. In Proceedings of the 26th ACM international conference on Multimedia, pp. 355–363, 2018.
  66. Sear: Scaling experiences in multi-user augmented reality. IEEE Transactions on Visualization and Computer Graphics, 28(5):1982–1992, 2022.
  67. Cars: Collaborative augmented reality for socialization. In Proceedings of the 19th International Workshop on Mobile Computing Systems & Applications, pp. 25–30, 2018.
  68. F. Zheng. Spatio-temporal registration in augmented reality. PhD thesis, The University of North Carolina at Chapel Hill, 2015.
  69. Semantic understanding of scenes through the ade20k dataset. International Journal on Computer Vision, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Jinwoo Jeon (7 papers)
  2. Woontack Woo (10 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets