Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A First Look at Immersive Telepresence on Apple Vision Pro (2405.10422v2)

Published 16 May 2024 in cs.NI

Abstract: Due to the widespread adoption of "work-from-home" policies, videoconferencing applications (e.g., Zoom) have become indispensable for remote communication. However, they often lack immersiveness, leading to the so-called "Zoom fatigue" and degrading communication efficiency. The recent debut of Apple Vision Pro, a mobile headset that supports "spatial persona", aims to offer an immersive telepresence experience. In this paper, we conduct a first-of-its-kind in-depth and empirical study to analyze the performance of immersive telepresence with Apple FaceTime, Cisco Webex, Microsoft Teams, and Zoom on Vision Pro. We find that only FaceTime provides a truly immersive experience with spatial personas, whereas others still operate 2D personas. Our measurement results reveal that (1) FaceTime delivers semantic data to optimize bandwidth consumption, which is even lower than that of 2D persona for other applications, and (2) it employs visibility-aware optimizations to reduce rendering overhead. However, the scalability of FaceTime remains limited, with a simple server-allocation strategy that potentially leads to high network delay for users.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (76)
  1. 2012. Sketchfab. https://sketchfab.com/feed. [accessed on 05/15/2024].
  2. 2019. Microsoft HoloLens 2. https://www.microsoft.com/en-us/hololens. [accessed on 05/15/2024].
  3. 2022. Magic Leap 2. https://www.magicleap.com/magic-leap-2. [accessed on 05/15/2024].
  4. 2023. Expand your world with Meta Quest 3. https://www.meta.com/quest/quest-3/. [accessed on 05/15/2024].
  5. 2024. Apple Vision Pro. https://www.apple.com/apple-vision-pro/. [accessed on 05/15/2024].
  6. 2024. Linux TC Man Page. https://linux.die.net/man/8/tc.
  7. 2024. ZED 2i. https://www.stereolabs.com/zed-2i/l. [accessed on 05/15/2024].
  8. 2024. Zoom Meetings. https://zoom.us/.
  9. Network Traffic in the Metaverse: The Case of Social VR. In Proceedings of IEEE International Conference on Distributed Computing Systems Workshops.
  10. Apple. 2024a. Analyzing the performance of your visionOS app. https://developer.apple.com/documentation/visionOS/analyzing-the-performance-of-your-visionOS-app.
  11. Apple. 2024b. Apple Vision Pro Privacy Overview. https://www.apple.com/privacy/docs/Apple_Vision_Pro_Privacy_Overview.pdf. [accessed on 05/15/2024].
  12. Apple. 2024c. Buy Apple Vision Pro. https://www.apple.com/shop/buy-vision/apple-vision-pro.
  13. Apple. 2024d. Create and manage Freeform boards on Apple Vision Pro. https://support.apple.com/guide/apple-vision-pro/create-and-manage-freeform-boards-tan5281cfbb6/visionos.
  14. Apple. 2024e. FaceTime. https://support.apple.com/guide/apple-vision-pro/make-or-receive-a-facetime-call-tan440238696/visionos.
  15. Apple. 2024f. Use SharePlay in FaceTime calls on Apple Vision Pro. https://support.apple.com/guide/apple-vision-pro/use-shareplay-in-facetime-calls-tan15b2c7bf9/visionos.
  16. Apple. 2024g. Use spatial Persona (beta) on Apple Vision Pro. https://support.apple.com/guide/apple-vision-pro/use-spatial-persona-tana1ea03f18/visionos.
  17. Apple. 2024h. Xcode. https://developer.apple.com/xcode/. [accessed on 05/15/2024].
  18. Mario Baldi and Yoram Ofek. 2000. End-to-end Delay Analysis of Videoconferencing Over Packet-switched Networks. IEEE/ACM Transactions On Networking 8, 4 (2000), 479–492.
  19. Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. In Proceedings of IEEE/CVF CVPR.
  20. Can You See Me Now? A Measurement Study of Zoom, Webex, and Meet. In Proceedings of ACM IMC. https://dl.acm.org/doi/abs/10.1145/3487552.3487847
  21. MARVEL: Enabling Mobile Augmented Reality with Low Energy and Low Latency. In Proceedings of ACM SenSys. https://doi.org/10.1145/3274783.3274834
  22. Enriching Telepresence with Semantic-driven Holographic Communication. In Proceedings of ACM HotNets.
  23. Reality Check of Metaverse: A First Look at Commercial Social Virtual Reality Platforms. In Proceedings of IEEE Workshop for Building the Foundations of the Metaverse (Metabuild), co-located with IEEE Conference on Virtual Reality and 3D User Interfaces (VR).
  24. Are We Ready for Metaverse? A Measurement Study of Social Virtual Reality Platforms. In Proceedings of ACM IMC.
  25. Cisco. 2024a. Check the Audio and Video Statistics of Your Cisco Webex Meeting. https://help.webex.com/en-us/article/nmghd9e/Check-the-Audio-and-Video-Statistics-of-Your-Cisco-Webex-Meeting.
  26. Cisco. 2024b. Webex. https://www.webex.com/.
  27. Survey on 6G Frontiers: Trends, Applications, Requirements, Technologies and Future Research. IEEE Open Journal of the Communications Society 2 (2021), 836–886.
  28. ARTEMIS: A Collaborative Mixed-Reality System for Immersive Surgical Telementoring. In Proceedings of ACM Conference on Human Factors in Computing Systems (CHI). https://doi.org/10.1145/3411764.3445576
  29. Google. 2024. Draco 3D Data Compression. https://google.github.io/draco/. [accessed on 05/15/2024].
  30. MetaStream: Live Volumetric Content Capture, Creation, Delivery, and Rendering in Real Time. In Proceedings of ACM MobiCom.
  31. Foveated 3D graphics. ACM Transactions on Graphics (TOG) 31, 6 (2012), 1–10.
  32. Modern Lossless Compression Techniques: Review, Comparison and Analysis. In Proceedings of IEEE International Conference on Electrical, Computer and Communication Technologies.
  33. Bringing the Web up to Speed with WebAssembly. In Proceedings of ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). https://doi.org/10.1145/3062341.3062363
  34. ViVo: Visibility-Aware Mobile Volumetric Video Streaming. In Proceedings of ACM MobiCom.
  35. A Measurement-Derived Functional Model for the Interaction Between Congestion Control and QoE in Video Conferencing. In Proceedings of International Conference on Passive and Active Network Measurement (PAM).
  36. Rubiks: Practical 360° Streaming for Smartphones. In Proceedings of ACM International Conference on Mobile Systems, Applications, and Services (MobiSys).
  37. HoloBots: Augmenting Holographic Telepresence with Mobile Robots for Tangible Remote Collaboration in Mixed Reality. In Proceedings of ACM Symposium on User Interface Software and Technology (UIST). https://doi.org/10.1145/3586183.3606727
  38. ipinfo.io. 2024. https://ipinfo.io/. [accessed on 05/15/2024].
  39. Performance Evaluation of WebRTC-based Video Conferencing. ACM SIGMETRICS Performance Evaluation Review 45, 3 (2018), 56–68.
  40. MeshReduce: Scalable and Bandwidth Efficient 3D Scene Capture. In Proceedings of IEEE Conference Virtual Reality and 3D User Interfaces (VR).
  41. Davis E King. 2009. Dlib-ml: A Machine Learning Toolkit. The Journal of Machine Learning Research 10 (2009), 1755–1758.
  42. Project Starline: a High-fidelity Telepresence System. ACM Transactions on Graphics (TOG) 40, 6 (2021), 1–16.
  43. FarfetchFusion: Towards Fully Mobile Live 3D Telepresence Platform. In Proceedings of ACM MobiCom.
  44. Demystifying Web-based Mobile Extended Reality Accelerated by WebAssembly. In Proceedings of ACM IMC. https://doi.org/10.1145/3618257.3624833
  45. Vues: Practical Volumetric Video Streaming through Multiview Transcoding. In Proceedings of ACM MobiCom.
  46. RenderFusion: Balancing Local and Remote Rendering for Interactive 3D Scenes. In Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR).
  47. MetaVRadar: Measuring Metaverse Virtual Reality Network Activity. In Proceedings of ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS).
  48. Pixel Codec Avatars. In Proceedings of IEEE/CVF CVPR.
  49. Measuring the Performance and Network Utilization of Popular Video Conferencing Applications. In Proceedings of ACM IMC. https://dl.acm.org/doi/abs/10.1145/3487552.3487842
  50. MaxMind. 2024. https://www.maxmind.com/en/home. [accessed on 05/15/2024].
  51. Host Anycasting Service. RFC 1546. https://www.rfc-editor.org/info/rfc1546 [accessed on 05/15/2024].
  52. Enabling Passive Measurement of Zoom Performance in Production Networks. In Proceedings of ACM IMC.
  53. Microsoft. 2024a. Monitor Call and Meeting Quality in Microsoft Teams. https://support.microsoft.com/en-us/office/monitor-call-and-meeting-quality-in-microsoft-teams-7bb1747c-d91a-4fbb-84f6-ad3f48e73511.
  54. Microsoft. 2024b. Teams. https://www.microsoft.com/en-us/microsoft-teams/group-chat-software.
  55. “Just Not Together”: The Experience of Videoconferencing for People with Aphasia during the Covid-19 Pandemic. In Proceedings of ACM Conference on Human Factors in Computing Systems (CHI).
  56. A Comparative Study of RTC Applications. In Proceedings of IEEE International Symposium on Multimedia (ISM).
  57. Holoportation: Virtual 3D Teleportation in Real-time. In Proceedings of ACM UIST. 741–754.
  58. Technologies for 3D Mesh Compression: A Survey. Journal of Visual Communication and Image Representation 16, 6 (2005), 688–733.
  59. Flare: Practical Viewport-Adaptive 360-Degree Video Streaming for Mobile Devices. In Proceedings of ACM MobiCom.
  60. Henning Schulzrinne and Stephen Casner. 2003. RTP Profile for Audio and Video Conferences with Minimal Control. RFC 3551. https://rfc-editor.org/rfc/rfc3551.txt [accessed on 05/15/2024].
  61. RTP: A Transport Protocol for Real-Time Applications. RFC 3550. https://rfc-editor.org/rfc/rfc3550.txt [accessed on 05/15/2024].
  62. Estimating WebRTC Video QoE Metrics Without Using Application Headers. In Proceedings of ACM IMC. https://doi.org/10.1145/3618257.3624828
  63. Switchboard. 2024. 40+ meeting statistics you need to know in 2024. https://www.switchboard.app/learn/article/meeting-statistics-2024. [accessed on 05/15/2024].
  64. A Speculative Study on 6G. IEEE Wireless Communications 27, 4 (2020), 118–125. https://doi.org/10.1109/MWC.001.1900488
  65. Loki: Facilitating Remote Instruction of Physical Tasks Using Bi-Directional Mixed-Reality Telepresence. In Proceedings of ACM Symposium on User Interface Software and Technology (UIST). https://doi.org/10.1145/3332165.3347872
  66. Michael C. Toren. 2024. tcptraceroute(1) - Linux man page. https://linux.die.net/man/1/tcptraceroute. [accessed on 05/15/2024].
  67. QUIC: A UDP-Based Multiplexed and Secure Transport. RFC 9000. https://datatracker.ietf.org/doc/html/rfc9000 [accessed on 05/15/2024].
  68. Christian J Van den Branden Lambrecht and Olivier Verscheure. 1996. Perceptual Quality Measure Using a Spatiotemporal Model of the Human Visual System. In Proceedings of Digital Video Compression: Algorithms and Technologies. https://doi.org/10.1117/12.235440
  69. Performance Characterization of Videoconferencing in the Wild. In Proceedings of ACM IMC. https://doi.org/10.1145/3517745.3561442
  70. Wikipedia. 2022. Zoom fatigue. https://en.wikipedia.org/wiki/Zoom_fatigue. [accessed on 05/15/2024].
  71. Wireshark. 1998. https://www.wireshark.org/. [accessed on 05/15/2024].
  72. WonderNetwork. 2024. Global Ping Statistics. https://wondernetwork.com/pings.
  73. DeepVista: 16K Panoramic Cinema on Your Mobile Device. In Proceedings of ACM Web Conference (WWW).
  74. QUIC is not Quick Enough over Fast Internet. In Proceedings of ACM Web Conference (WWW).
  75. A Measurement Study of Oculus 360 Degree Video Streaming. In Proceedings of ACM on Multimedia Systems Conference. https://doi.org/10.1145/3083187.3083190
  76. Zoom. 2024. Accessing Meeting and Phone Statistics. https://support.zoom.com/hc/en/article?id=zm_kb&sysparm_article=KB0070504.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now