Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Swarm manipulation: An efficient and accurate technique for multi-object manipulation in virtual reality (2410.18924v1)

Published 24 Oct 2024 in cs.HC and cs.RO

Abstract: The theory of swarm control shows promise for controlling multiple objects, however, scalability is hindered by cost constraints, such as hardware and infrastructure. Virtual Reality (VR) can overcome these limitations, but research on swarm interaction in VR is limited. This paper introduces a novel Swarm Manipulation interaction technique and compares it with two baseline techniques: Virtual Hand and Controller (ray-casting). We evaluated these techniques in a user study ($N$ = 12) in three tasks (selection, rotation, and resizing) across five conditions. Our results indicate that Swarm Manipulation yielded superior performance, with significantly faster speeds in most conditions across the three tasks. It notably reduced resizing size deviations but introduced a trade-off between speed and accuracy in the rotation task. Additionally, we conducted a follow-up user study ($N$ = 6) using Swarm Manipulation in two complex VR scenarios and obtained insights through semi-structured interviews, shedding light on optimized swarm control mechanisms and perceptual changes induced by this interaction paradigm. These results demonstrate the potential of the Swarm Manipulation technique to enhance the usability and user experience in VR compared to conventional manipulation techniques. In future studies, we aim to understand and improve swarm interaction via internal swarm particle cooperation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Swarm robotics: Past, present, and future. Proceedings of the IEEE 2021;109(7):1152–1165. doi:10.1109/JPROC.2021.3072740.
  2. Self-organization in biological systems. Princeton university press; 2001.
  3. Inspiration for optimization from social insect behaviour. Nature 2000;406(6791):39–42. URL: https://www.nature.com/articles/35017500. doi:10.1038/35017500; number: 6791 Publisher: Nature Publishing Group.
  4. The design challenges of drone swarm control. In: Harris, D, Li, WC, editors. Engineering Psychology and Cognitive Ergonomics. Lecture Notes in Computer Science; Cham: Springer International Publishing. ISBN 978-3-030-77932-0; 2021, p. 408–426. doi:10.1007/978-3-030-77932-0_32.
  5. Robotic swarm control from spatio-temporal specifications. In: IEEE 55th Conference on Decision and Control (CDC). 2016, p. 5708–5713. doi:10.1109/CDC.2016.7799146.
  6. AeroRigUI: Actuated TUIs for spatial interaction using rigging swarm robots on ceilings in everyday space. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. CHI ’23; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-9421-5; 2023, p. 1–18. URL: https://dl.acm.org/doi/10.1145/3544548.3581437. doi:10.1145/3544548.3581437.
  7. Human interaction with robot swarms: A survey. IEEE Transactions on Human-Machine Systems 2016;46(1):9–26. doi:10.1109/THMS.2015.2480801.
  8. (Dis)Appearables: A concept and method for actuated tangible UIs to appear and disappear based on stages. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. CHI ’22; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-9157-3; 2022, p. 1–13. URL: https://dl.acm.org/doi/10.1145/3491102.3501906. doi:10.1145/3491102.3501906.
  9. HERMITS: Dynamically reconfiguring the interactivity of self-propelled TUIs with mechanical shell add-ons. In: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. UIST ’20; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-7514-6; 2020, p. 882–896. URL: https://dl.acm.org/doi/10.1145/3379337.3415831. doi:10.1145/3379337.3415831.
  10. Reactile: Programming swarm user interfaces through direct physical manipulation. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. CHI ’18; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-5620-6; 2018, p. 1–13. URL: https://dl.acm.org/doi/10.1145/3173574.3173773. doi:10.1145/3173574.3173773.
  11. Toward robust and intelligent drone swarm: challenges and future directions. IEEE Network 2020;34(4):278–283. doi:10.1109/MNET.001.1900521.
  12. Human-swarm interactions based on managing attractors. In: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction. HRI ’14; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-2658-2; 2014, p. 90–97. URL: https://doi.org/10.1145/2559636.2559661. doi:10.1145/2559636.2559661.
  13. Holmquist, LE. Bits are cheap, atoms are expensive: critiquing the turn towards tangibility in HCI. In: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. CHI EA ’23; New York, NY, USA: Association for Computing Machinery. ISBN 9781450394222; 2023,URL: https://doi.org/10.1145/3544549.3582744. doi:10.1145/3544549.3582744.
  14. Being part of the swarm: Experiencing human-swarm interaction with VR and tangible robots. In: Proceedings of the 2020 ACM Symposium on Spatial User Interaction. SUI ’20; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-7943-4; 2020, p. 1–2. URL: https://dl.acm.org/doi/10.1145/3385959.3422695. doi:10.1145/3385959.3422695.
  15. How to evaluate object selection and manipulation in VR? Guidelines from 20 years of studies. In: Proceedings of the 2021 CHI conference on human factors in computing systems. 2021, p. 1–20.
  16. A survey on 3D virtual object manipulation: From the desktop to immersive virtual environments. Computer Graphics Forum 2019;38(1):21–45. URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.13390. doi:10.1111/cgf.13390; _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13390.
  17. A survey of design issues in spatial input. In: Proceedings of the 7th annual ACM symposium on User interface software and technology. UIST ’94; New York, NY, USA: Association for Computing Machinery. ISBN 978-0-89791-657-8; 1994, p. 213–222. URL: https://dl.acm.org/doi/10.1145/192426.192501. doi:10.1145/192426.192501.
  18. An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In: Proceedings of the 1997 symposium on Interactive 3D graphics. I3D ’97; New York, NY, USA: Association for Computing Machinery. ISBN 978-0-89791-884-8; 1997, p. 35–ff. URL: https://dl.acm.org/doi/10.1145/253284.253301. doi:10.1145/253284.253301.
  19. Ninja hands: Using many hands to improve target selection in VR. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. CHI ’21; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-8096-6; 2021, p. 1–14. URL: https://doi.org/10.1145/3411764.3445759. doi:10.1145/3411764.3445759.
  20. Swarm manipulation in virtual reality. In: Proceedings of the 2023 ACM Symposium on Spatial User Interaction. SUI ’23; New York, NY, USA: Association for Computing Machinery. ISBN 9798400702815; 2023,URL: https://doi.org/10.1145/3607822.3614519. doi:10.1145/3607822.3614519.
  21. Influence of degrees of freedom’s manipulation on performances during orientation tasks in virtual reality environments. In: Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology. VRST ’09; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-60558-869-8; 2009, p. 51–58. URL: https://dl.acm.org/doi/10.1145/1643928.1643942. doi:10.1145/1643928.1643942.
  22. The benefits of DOF separation in mid-air 3D object manipulation. In: Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology. VRST ’16; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-4491-3; 2016, p. 261–268. URL: https://dl.acm.org/doi/10.1145/2993369.2993396. doi:10.1145/2993369.2993396.
  23. Bare-handed 3D drawing in augmented reality. In: Proceedings of the 2018 Designing Interactive Systems Conference. 2018, p. 241–252.
  24. Gaze-supported 3D object manipulation in virtual reality. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. CHI ’21; New York, NY, USA: Association for Computing Machinery. ISBN 9781450380966; 2021,URL: https://doi.org/10.1145/3411764.3445343. doi:10.1145/3411764.3445343.
  25. Blending on-body and mid-air interaction in virtual reality. In: 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 2022, p. 637–646. doi:10.1109/ISMAR55827.2022.00081.
  26. Exploring visual techniques for boundary awareness During interaction in augmented reality head-mounted displays. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 2020, p. 204–211. doi:10.1109/VR46266.2020.00039; iSSN: 2642-5254.
  27. The go-go interaction technique: Non-linear mapping for direct manipulation in vr. In: Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology. UIST ’96; New York, NY, USA: Association for Computing Machinery. ISBN 0897917987; 1996, p. 79–80. URL: https://doi.org/10.1145/237091.237102. doi:10.1145/237091.237102.
  28. Moving objects in space: Exploiting proprioception in virtual-environment interaction. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques. 1997, p. 19–26.
  29. Hybridpointing: Fluid switching between absolute and relative pointing with a direct input device. In: Proceedings of the 19th annual ACM symposium on User interface software and technology. 2006, p. 211–220.
  30. Voodoo dolls: Seamless interaction at multiple scales in virtual environments. In: Proceedings of the 1999 symposium on Interactive 3D graphics. I3D ’99; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-58113-082-9; 1999, p. 141–145. URL: https://dl.acm.org/doi/10.1145/300523.300540. doi:10.1145/300523.300540.
  31. Virtual reality on a WIM: Interactive worlds in miniature. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI ’95; USA: ACM Press/Addison-Wesley Publishing Co. ISBN 978-0-201-84705-5; 1995, p. 265–272. URL: https://dl.acm.org/doi/10.1145/223904.223938. doi:10.1145/223904.223938.
  32. The design and evaluation of selection techniques for 3D volumetric displays. In: Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology. UIST ’06; New York, NY, USA: Association for Computing Machinery. ISBN 1595933131; 2006, p. 3–12. URL: https://doi.org/10.1145/1166253.1166257. doi:10.1145/1166253.1166257.
  33. Influence of personality traits and dody awareness on the sense of embodiment in virtual reality. In: 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 2019, p. 123–134. doi:10.1109/ISMAR.2019.00-12; iSSN: 1554-7868.
  34. First person experience of body transfer in virtual reality. PLOS ONE 2010;5(5). doi:10.1371/journal.pone.0010564; mAG ID: 2163290657.
  35. Two invariants of human-swarm interaction. Journal of Human-Robot Interaction 2016;5(1):1–31. URL: https://doi.org/10.5898/JHRI.5.1.Brown. doi:10.5898/JHRI.5.1.Brown.
  36. Human-swarm interaction: An experimental study of two types of interaction with foraging swarms. Journal of Human-Robot Interaction 2013;2(2):103–129. URL: https://doi.org/10.5898/JHRI.2.2.Kolling. doi:10.5898/JHRI.2.2.Kolling.
  37. Human perception of swarm robot motion. In: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. CHI EA ’17; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-4656-6; 2017, p. 2520–2527. URL: https://doi.org/10.1145/3027063.3053220. doi:10.1145/3027063.3053220.
  38. Zooids: Building blocks for swarm user interfaces. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology. UIST ’16; New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-4189-9; 2016, p. 97–109. URL: https://dl.acm.org/doi/10.1145/2984511.2984547. doi:10.1145/2984511.2984547.
  39. “Wow! I have six fingers!”: Would you accept structural changes of your hand in VR? Frontiers in Robotics and AI 2016;3:27.
  40. Ninja cursors: Using multiple cursors to assist target acquisition on large screens. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI ’08; New York, NY, USA: Association for Computing Machinery. ISBN 9781605580111; 2008, p. 949–958. URL: https://doi.org/10.1145/1357054.1357201. doi:10.1145/1357054.1357201.
  41. Disambiguating ninja cursors with eye gaze. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2009, p. 1411–1414.
  42. Are 4 hands better than 2? Bimanual interaction for quadmanual user interfaces. In: Proceedings of the 2nd ACM symposium on Spatial user interaction. 2014, p. 123–126.
  43. vrCAPTCHA: Exploring CAPTCHA designs in virtual reality. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. CHI EA ’21; New York, NY, USA: Association for Computing Machinery. ISBN 9781450380959; 2021,URL: https://doi.org/10.1145/3411763.3451985. doi:10.1145/3411763.3451985.
  44. Efficient special character entry on a virtual keyboard by hand gesture-based mode switching. In: IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE; 2022, p. 864–871.
  45. Testbed evaluation of virtual environment interaction techniques. In: Proceedings of the ACM symposium on Virtual reality software and technology. 1999, p. 26–33.
  46. Hart, SG. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2006;50(9):904–908. URL: https://doi.org/10.1177/154193120605000909. doi:10.1177/154193120605000909; publisher: SAGE Publications Inc.
  47. SUS-a quick and dirty usability scale. Usability evaluation in industry 1996;189(194):4–7.
  48. Construction of a benchmark for the user experience questionnaire (UEQ). International Journal of Interactive Multimedia and Artificial Intelligence 2017;.
  49. Integrating real-world distractions into virtual reality. In: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. UIST ’22; New York, NY, USA: Association for Computing Machinery. ISBN 9781450393201; 2022,URL: https://doi.org/10.1145/3526113.3545682. doi:10.1145/3526113.3545682.
  50. Council, D. The ‘double diamond’ design process model. Design Council 2005;2:1.
  51. Beyond the force: Using quadcopters to appropriate objects and the environment for haptics in virtual reality. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. CHI ’19; New York, NY, USA: Association for Computing Machinery. ISBN 9781450359702; 2019, p. 1–13. URL: https://doi.org/10.1145/3290605.3300589. doi:10.1145/3290605.3300589.
  52. Beyond being real: A sensorimotor control perspective on interactions in virtual reality. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. CHI ’22; New York, NY, USA: Association for Computing Machinery. ISBN 9781450391573; 2022,URL: https://doi.org/10.1145/3491102.3517706. doi:10.1145/3491102.3517706.
  53. Sensorimotor simulation of redirected reaching using stochastic optimal feedback control. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. CHI ’23; New York, NY, USA: Association for Computing Machinery. ISBN 9781450394215; 2023,URL: https://doi.org/10.1145/3544548.3580767. doi:10.1145/3544548.3580767.
  54. What is interaction? In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2017, p. 5040–5052.
  55. Computational Interaction. Oxford University Press; 2018.
  56. Limited control over the body as intriguing play design resource. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. CHI ’21; New York, NY, USA: Association for Computing Machinery. ISBN 9781450380966; 2021,URL: https://doi.org/10.1145/3411764.3445744. doi:10.1145/3411764.3445744.
  57. Towards understanding the design of intertwined human-computer integrations. ACM Trans Comput-Hum Interact 2023;URL: https://doi.org/10.1145/3590766. doi:10.1145/3590766; just Accepted.
  58. Maes, P. Artificial life meets entertainment: Lifelike autonomous agents. Commun ACM 1995a;38(11):108–114. URL: https://doi.org/10.1145/219717.219808. doi:10.1145/219717.219808.
  59. Maes, P. Agents that reduce work and information overload. In: Readings in human–computer interaction. Elsevier; 1995b, p. 811–821.
  60. Direct manipulation vs. interface agents. interactions 1997;4(6):42–61.
  61. Ishii, H. Tangible bits: Beyond pixels. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction. TEI ’08; New York, NY, USA: Association for Computing Machinery. ISBN 9781605580043; 2008, p. xv–xxv. URL: https://doi.org/10.1145/1347390.1347392. doi:10.1145/1347390.1347392.
  62. Radical atoms: Beyond tangible bits, toward transformable materials. Interactions 2012;19(1):38–51. URL: https://doi.org/10.1145/2065327.2065337. doi:10.1145/2065327.2065337.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now