Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Design Space of Visual Feedforward And Corrective Feedback in XR-Based Motion Guidance Systems (2402.09182v2)

Published 14 Feb 2024 in cs.HC

Abstract: Extended reality (XR) technologies are highly suited in assisting individuals in learning motor skills and movements -- referred to as motion guidance. In motion guidance, the "feedforward" provides instructional cues of the motions that are to be performed, whereas the "feedback" provides cues which help correct mistakes and minimize errors. Designing synergistic feedforward and feedback is vital to providing an effective learning experience, but this interplay between the two has not yet been adequately explored. Based on a survey of the literature, we propose design space for both motion feedforward and corrective feedback in XR, and describe the interaction effects between them. We identify common design approaches of XR-based motion guidance found in our literature corpus, and discuss them through the lens of our design dimensions. We then discuss additional contextual factors and considerations that influence this design, together with future research opportunities for motion guidance in XR.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. YouMove: enhancing movement training with an augmented reality mirror. In ACM Symp. User Interface Software and Technology (UIST). 311–320. https://doi.org/10.1145/2501988.2502045
  2. BalletVR: a Virtual Reality System for Ballet Arm Positions Training. In Symp. Virtual and Augmented Reality (SVR). IEEE, 10–16. https://doi.org/10.1109/SVR.2019.00018
  3. Olivier Bau and Wendy E Mackay. 2008. OctoPocus: a dynamic guide for learning gesture-based command sets. In ACM Symp. User Interface Software and Technology (UIST). 37–46. https://doi.org/10.1145/1449715.1449724
  4. Serious games for upper limb rehabilitation following stroke. In 2009 Conference in Games and Virtual Worlds for Serious Applications. IEEE, 103–110. https://doi.org/10.1109/VS-GAMES.2009.17
  5. A Taxonomy for Augmented and Mixed Reality Applications to Support Physical Exercises in Medical Rehabilitation—A Literature Review. In Healthcare, Vol. 10. MDPI, 646. https://doi.org/10.3390/healthcare10040646
  6. An exploratory study of augmented reality presence for tutoring machine tasks. In ACM Conf. Human Factors in Computing Systems (CHI). 1–13. https://doi.org/10.1145/3313831.3376688
  7. Full-Body Motion Recognition in Immersive-Virtual-Reality-Based Exergame. IEEE Trans. Games 14, 2 (2021), 243–252. https://doi.org/10.1109/TG.2021.3064749
  8. Kabalan Chaccour and Georges Badr. 2016. Computer vision guidance system for indoor navigation of visually impaired people. In IEEE Int. Conf. intelligent systems (IS). IEEE, 449–454. https://doi.org/10.1109/IS.2016.7737460
  9. VCoach: Enabling Personalized Boxing Training in Virtual Reality. In IEEE Conf. Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 546–547. https://doi.org/10.1109/VRW55335.2022.00124
  10. MirrARbilitation: A clinically-related gesture recognition interactive tool for an AR rehabilitation system. Computer methods and programs in biomedicine 135 (2016), 105–114. https://doi.org/10.1016/j.cmpb.2016.07.014
  11. Designing guiding systems for gesture-based interaction. In Proc. ACM SIGCHI Symp. Engineering Interactive Computing SystemsJune (EICS). 44–53. https://doi.org/10.1145/2774225.2774847
  12. Designing 3D gesture guidance: visual feedback and feedforward design options. In ACM Int. Working Conf. Advanced Visual Interfaces (AVI). 152–159. https://doi.org/10.1145/2909132.2909260
  13. Visual Cue Based Corrective Feedback for Motor Skill Training in Mixed Reality: A Survey. IEEE Trans. Visualization and Computer Graphics (TVCG) (2022). https://doi.org/10.1109/TVCG.2022.3227999
  14. EGuide: Investigating different visual appearances and guidance techniques for egocentric guidance visualizations. In Proc. Int. Conf. Tangible, Embedded, and Embodied Interaction (TEI). 311–322. https://doi.org/10.1145/3374920.3374945
  15. Understanding Perspectives for Single-and Multi-Limb Movement Guidance in Virtual 3D Environments. In ACM Symp. Virtual Reality Software and Technology (VRST). 1–10. https://doi.org/10.1145/3562939.3565635
  16. EVA: EVAluating at-home rehabilitation exercises using augmented reality and low-cost sensors. Virtual Reality 24 (2020), 567–581. https://doi.org/10.1007/s10055-019-00419-4
  17. Octopocus in VR: using a dynamic guide for 3D mid-air gestures in virtual reality. IEEE Trans. Visualization and Computer Graphics (TVCG) 27, 12 (2021), 4425–4438. https://doi.org/10.1109/TVCG.2021.3101854
  18. ShadowGuides: visualizations for in-situ learning of multi-touch and whole-hand gestures. In Proc. the ACM Int. Conf. interactive tabletops and surfaces (ITS). 165–172. https://doi.org/10.1145/1731903.1731935
  19. Design of an AR-supported System for Skill Training. In 2018 Tohoku-Section Joint Convention. 213–213.
  20. Auxiliary Means to Improve Motion Guidance Memorability in Extended Reality. In IEEE Conf. Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 689–690. https://doi.org/10.1109/VRW58643.2023.00187
  21. Modality Influence on the Motor Learning of Ballroom Dance with a Mixed-Reality Human-Machine Interface. In IEEE/SICE Int. Symp. System Integration (SII). IEEE, 177–182. https://doi.org/10.1109/SII52469.2022.9708803
  22. Ar-arm: Augmented visualization for guiding arm movement in the first-person perspective. In Proc. Augmented Human International Conference (AH). 1–4. https://doi.org/10.1145/2875194.2875237
  23. My Tai-Chi coaches: an augmented-learning tool for practicing Tai-Chi Chuan. In Proc. Augmented Human International Conference (AH). 1–4. https://doi.org/10.1145/3041164.3041194
  24. Christopher Healey and James Enns. 2011. Attention and visual memory in visualization and computer graphics. IEEE Trans. Visualization and Computer Graphics (TVCG) 18, 7 (2011), 1170–1188. https://doi.org/10.1109/TVCG.2011.127
  25. Christopher G Healey and James T Enns. 1999. Large datasets at a glance: Combining textures and colors in scientific visualization. IEEE Trans. Visualization and Computer Graphics (TVCG) 5, 2 (1999), 145–167.
  26. Onebody: remote posture guidance system using first person view in virtual environment. In Proc. Nordic Conf. Human-Computer Interaction (NordiCHI). 1–10. https://doi.org/10.1145/2971485.2971521
  27. Superimposed skilled performance in a virtual mirror improves motor performance and cognitive representation of a full body motor action. Frontiers in Robotics and AI 6 (2019), 43. https://doi.org/10.3389/frobt.2019.00043
  28. A real-time projection system for golf training using virtual shadow. In IEEE Conf. Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 1527–1528. https://doi.org/10.1109/VR.2019.8798196
  29. AR based Self-sports Learning System using Decayed Dynamic TimeWarping Algorithm.. In ICAT-EGVE 2018 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments. 171–174. https://doi.org/10.2312/egve.20181330
  30. Vibrotactile Cues for Motion Guidance. In Intelligent Autonomous Systems 12: Volume 2 Proceedings of the 12th International Conference IAS-12, held June 26-29, 2012, Jeju Island, Korea. Springer, 651–659. https://doi.org/10.1007/978-3-642-33932-5_61
  31. Vr-assisted physical rehabilitation: Adapting to the needs of therapists and patients. In Virtual Realities: International Dagstuhl Seminar, Dagstuhl Castle, Germany, June 9-14, 2013, Revised Selected Papers. Springer, 147–168. https://doi.org/10.1007/978-3-319-17043-5_9
  32. Effects of Collaborative Training Using Virtual Co-embodiment on Motor Skill Learning. IEEE Trans. Visualization and Computer Graphics (TVCG) 29, 5 (2023), 2304–2314. https://doi.org/10.1109/TVCG.2023.3247112
  33. Climbvis: Investigating in-situ visualizations for understanding climbing movements by demonstration. In Proc. ACM Int. Conf. Interactive Surfaces and Spaces (ISS). 270–279. https://doi.org/10.1145/3132272.3134119
  34. An approach to ballet dance training through ms kinect and visualization in a cave virtual reality environment. ACM Tran. Intelligent Systems and Technology (TIST) 6, 2 (2015), 1–37. https://doi.org/10.1145/2735951
  35. Superimposing 3D virtual self+ expert modeling for motor learning: Application to the throw in American football. Frontiers in ICT 6 (2019), 16. https://doi.org/10.3389/fict.2019.00016
  36. Data Visceralization: Enabling deeper understanding of data using virtual reality. IEEE Trans. Visualization and Computer Graphics (TVCG) 27, 2 (2020), 1095–1105. https://doi.org/10.1109/TVCG.2020.3030435
  37. Design Patterns for Situated Visualization in Augmented Reality. IEEE Trans. Visualization and Computer Graphics (TVCG) (2023). https://doi.org/10.1109/TVCG.2023.3327398
  38. Aoqi Li and Zhenzhong Chen. 2018. Personalized visual saliency: Individuality affects image perception. IEEE Access 6 (2018), 16099–16109. https://doi.org/10.1109/ACCESS.2018.2800294
  39. Correction of avatar hand movements supports learning of a motor skill. In IEEE Conf. Virtual Reality and 3D User Interfaces (VR). IEEE, 1–8. https://doi.org/10.1109/VR50410.2021.00069
  40. Towards an understanding of situated ar visualization for basketball free-throw training. In ACM Conf. Human Factors in Computing Systems (CHI). 1–13. https://doi.org/10.1145/3411764.3445649
  41. STTAR: surgical tool tracking using off-the-shelf augmented reality head-mounted displays. IEEE Trans. Visualization and Computer Graphics (TVCG) (2023). https://doi.org/10.1109/TVCG.2023.3238309
  42. Evaluation of Pre-Flight and On Orbit Training Methods Utilizing Virtual Reality. In Proceedings of the AIAA Scitech 2020 Forum. https://doi.org/10.2514/6.2020-0168
  43. Upbeat: augmented reality-guided dancing for prosthetic rehabilitation of upper limb amputees. Journal of healthcare engineering 2019 (2019). https://doi.org/10.1155/2019/2163705
  44. Welcoming a holographic virtual coach for balance training at home: two focus groups with older adults. In IEEE Conf. Virtual Reality and 3D User Interfaces (VR). IEEE, 1465–1470. https://doi.org/10.1109/VR.2019.8797813
  45. Tamara Munzner. 2014. Visualization analysis and design. CRC press.
  46. Using Feedforward to Reveal Interaction Possibilities in Virtual Reality. ACM Trans. Computer-Human Interaction (2023). https://doi.org/10.1145/3603623
  47. Self-training system for tennis shots with motion feature assessment and visualization. In IEEE Int. Conf. Cyberworlds (CW). IEEE, 82–89. https://doi.org/10.1109/CW.2018.00025
  48. Assistance system for rehabilitation and valuation of motor skills. In Augmented Reality, Virtual Reality, and Computer Graphics: 4th International Conference, AVR 2017, Ugento, Italy, June 12-15, 2017, Proceedings, Part II 4. Springer, 166–174. https://doi.org/10.1007/978-3-319-60928-7_14
  49. Multimodal motion guidance: techniques for adaptive and dynamic feedback. In Proc. ACM Int. Conf. Multimodal interaction (ICMI). 133–140. https://doi.org/10.1145/2388676.2388706
  50. Yoones A Sekhavat and Mohammad S Namani. 2018. Projection-based AR: Effective visual feedback in gait rehabilitation. IEEE Tran. Human-Machine Systems 48, 6 (2018), 626–636. https://doi.org/10.1109/THMS.2018.2860579
  51. LightGuide: projected visualizations for hand movement guidance. In ACM Conf. Human Factors in Computing Systems (CHI). 179–188. https://doi.org/10.1145/2207676.2207702
  52. SleeveAR: Augmented reality for rehabilitation using realtime feedback. In ACM Int. Conf. Intelligent User Interfaces (IUI). 175–185. https://doi.org/10.1145/2856767.2856773
  53. Physio@ Home: Exploring visual guidance and feedback techniques for physiotherapy exercises. In ACM Conf. Human Factors in Computing Systems (CHI). 4123–4132. https://doi.org/10.1145/2702123.2702401
  54. Bodylights: Open-ended augmented feedback to support training towards a correct exercise execution. In ACM Conf. Human Factors in Computing Systems (CHI). 1–14. https://doi.org/10.1145/3313831.3376268
  55. The impact of latency on perceptual judgments and motor performance in closed-loop interaction in virtual reality. In ACM Symp. Virtual Reality Software and Technology (VRST). 27–35. https://doi.org/10.1145/2993369.2993381
  56. Survey of movement reproduction in immersive virtual rehabilitation. IEEE Trans. Visualization and Computer Graphics (TVCG) (2022). https://doi.org/10.1109/TVCG.2022.3142198
  57. Claes Wohlin. 2014. Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proc. Int. conf. evaluation and assessment in software engineering (EASE). 1–10. https://doi.org/10.1145/2601248.2601268
  58. Ungyeon Yang and Gerard Jounghyun Kim. 2002. Implementation and evaluation of “just follow me”: An immersive, VR-based, motion-training system. Presence 11, 3 (2002), 304–323. https://doi.org/10.1162/105474602317473240
  59. Perspective matters: Design implications for motion guidance in mixed reality. In IEEE Symp. Mixed and Augmented Reality (ISMAR). IEEE, 577–587. https://doi.org/10.1109/ISMAR50242.2020.00085
  60. Watch-Your-Skiing: Visualizations for VR Skiing using Real-time Body Tracking. In IEEE Symp. Mixed and Augmented Reality (ISMAR). IEEE, 387–388. https://doi.org/10.1109/ISMAR-Adjunct54149.2021.00088
  61. Movement guidance using a mixed reality mirror. In ACM SIGCHI Conf. Designing Interactive Systems (DIS). 821–834. https://doi.org/10.1145/3532106.3533466
  62. MuscleRehab: Improving Unsupervised Physical Rehabilitation by Monitoring and Visualizing Muscle Engagement. In ACM Symp. User Interface Software and Technology (UIST). 1–14. https://doi.org/10.1145/3526113.3545705
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets