Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Independence in the Home: A Wearable Interface for a Person with Quadriplegia to Teleoperate a Mobile Manipulator (2312.15071v2)

Published 22 Dec 2023 in cs.RO and cs.HC

Abstract: Teleoperation of mobile manipulators within a home environment can significantly enhance the independence of individuals with severe motor impairments, allowing them to regain the ability to perform self-care and household tasks. There is a critical need for novel teleoperation interfaces to offer effective alternatives for individuals with impairments who may encounter challenges in using existing interfaces due to physical limitations. In this work, we iterate on one such interface, HAT (Head-Worn Assistive Teleoperation), an inertial-based wearable integrated into any head-worn garment. We evaluate HAT through a 7-day in-home study with Henry Evans, a non-speaking individual with quadriplegia who has participated extensively in assistive robotics studies. We additionally evaluate HAT with a proposed shared control method for mobile manipulators termed Driver Assistance and demonstrate how the interface generalizes to other physical devices and contexts. Our results show that HAT is a strong teleoperation interface across key metrics including efficiency, errors, learning curve, and workload. Code and videos are located on our project website.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Evan Ackerman. 2023. This robot could be the key to helping people with disabilities. https://spectrum.ieee.org/stretch-assistive-robot
  2. Brenna D. Argall. 2018. Autonomy in Rehabilitation Robotics: An Intersection. Annual Review of Control, Robotics, and Autonomous Systems 1, 1 (2018), 441–463. https://doi.org/10.1146/annurev-control-061417-041727
  3. Prevalence and causes of paralysis—United States, 2013. American journal of public health 106, 10 (2016), 1855–1857.
  4. The camera mouse: visual tracking of body features to provide computer access for people with severe disabilities. IEEE Transactions on neural systems and Rehabilitation Engineering 10, 1 (2002), 1–10.
  5. Is More Autonomy Always Better?: Exploring Preferences of Users with Mobility Impairments in Robot-assisted Feeding. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction. ACM, Cambridge United Kingdom, 181–190. https://doi.org/10.1145/3319502.3374818
  6. Improved Electric Wheelchair Controlled by Head Motion. In Research in Intelligent and Computing in Engineering. Springer, 121–129.
  7. Surgical restoration of hand function in Tetraplegia. Spinal Cord Series and Cases 7, 1 (2021). https://doi.org/10.1038/s41394-021-00387-5
  8. An exploration of accessible remote tele-operation for assistive mobile manipulators in the home. In 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN). IEEE, 1202–1209.
  9. Robots for humanity: using assistive robotics to empower people with disabilities. IEEE Robotics & Automation Magazine 20, 1 (2013), 30–39.
  10. Mobile manipulation through an assistive home robot. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 5313–5320.
  11. High-performance neuroprosthetic control by an individual with tetraplegia. The Lancet 381, 9866 (2013), 557–564. https://doi.org/10.1016/s0140-6736(12)61816-9
  12. MPJM Dijkers. 2005. Quality of life of individuals with spinal cord injury: a review of conceptualization, measurement, and research findings. Journal of rehabilitation research and development 42, 3 (2005), 87.
  13. Anca D Dragan and Siddhartha S Srinivasa. 2013. A policy-blending formalism for shared control. The International Journal of Robotics Research 32, 7 (June 2013), 790–805. https://doi.org/10.1177/0278364913490324
  14. Ahmetcan Erdogan and Brenna D. Argall. 2017. The effect of robotic wheelchair control paradigm and interface on user performance, effort and preference: An experimental assessment. Robotics and Autonomous Systems 94 (2017), 282–297. https://doi.org/10.1016/j.robot.2017.04.013
  15. 3D human gesture capturing and recognition by the IMMU-based data glove. Neurocomputing 277 (2018), 198–207.
  16. Adequacy of power wheelchair control interfaces for persons with severe disabilities: A clinical survey. Journal of rehabilitation research and development 37, 3 (2000), 353–360.
  17. Health implications of physical activity in individuals with spinal cord injury: a literature review. Journal of health and human services administration 30 4 (2008), 468–502.
  18. The potential of virtual reality-based training to enhance the functional autonomy of Alzheimer’s disease patients in cooking activities: A single case study. Neuropsychological rehabilitation 28, 5 (2018), 709–733.
  19. Wheelchair prototype controlled by position, speed and orientation using head movement. HardwareX 11 (2022), e00306.
  20. Human-in-the-Loop Optimization of Shared Autonomy in Assistive Robotics. IEEE Robotics and Automation Letters 2, 1 (2017), 247–254. https://doi.org/10.1109/LRA.2016.2593928
  21. Phillip M. Grice and Charles C. Kemp. 2019. In-home and remote use of robotic body surrogates by people with profound motor deficits. PLOS ONE 14, 3 (2019). https://doi.org/10.1371/journal.pone.0212904
  22. Control System Design and Methods for Collaborative Robots: Review. Applied Sciences 13, 1 (2023). https://doi.org/10.3390/app13010675
  23. Assistive mobile manipulation for self-care tasks around the head. In 2014 IEEE Symposium on computational intelligence in robotic rehabilitation and assistive technologies (CIR2AT). IEEE, 16–25.
  24. Robert JK Jacob. 1995. Eye tracking in advanced interface design. Virtual environments and advanced interface design 258 (1995), 288.
  25. Siddarth Jain and Brenna Argall. 2019. Probabilistic Human Intent Recognition for Shared Autonomy in Assistive Robotics. J. Hum.-Robot Interact. 9, 1, Article 2 (dec 2019), 23 pages. https://doi.org/10.1145/3359614
  26. Shared autonomy via hindsight optimization for teleoperation and teaming. The International Journal of Robotics Research 37, 7 (June 2018), 717–742. https://doi.org/10.1177/0278364918776060
  27. A system for bedside assistance that integrates a robotic bed and a mobile manipulator. Plos one 14, 10 (2019), e0221854.
  28. An eye tracking computer user interface. In Proceedings of 1993 IEEE Research Properties in Virtual Reality Symposium. IEEE, 120–121.
  29. The design of Stretch: A compact, lightweight mobile manipulator for indoor human environments. In 2022 International Conference on Robotics and Automation (ICRA). IEEE, 3150–3157.
  30. Dusty: an assistive mobile manipulator that retrieves dropped objects for people with motor impairments. Disability and Rehabilitation: Assistive Technology 7, 2 (2012), 168–179.
  31. A mobile robot hand-arm teleoperation system by vision and imu. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 10900–10906.
  32. A Review of Intent Detection, Arbitration, and Communication Aspects of Shared Control for Physical Human–Robot Interaction. Applied Mechanics Reviews 70, 1 (02 2018), 010804. https://doi.org/10.1115/1.4039145 arXiv:https://asmedigitalcollection.asme.org/appliedmechanicsreviews/article-pdf/70/1/010804/5964415/amr_070_01_010804.pdf
  33. Wearable Technology in stroke rehabilitation: Towards improved diagnosis and treatment of upper-limb motor impairment. Journal of NeuroEngineering and Rehabilitation 16, 1 (2019). https://doi.org/10.1186/s12984-019-0612-y
  34. Development of a control system for electric wheelchairs based on head movements. In 2017 Intelligent Systems Conference (IntelliSys). IEEE, 996–1001.
  35. Impact of motor, cognitive, and perceptual disorders on ability to perform activities of daily living after stroke. Stroke 32, 11 (2001), 2602–2608.
  36. Robust shared autonomy for mobile manipulation with continuous scene monitoring. In 2017 13th IEEE Conference on Automation Science and Engineering (CASE). https://doi.org/10.1109/COASE.2017.8256092
  37. A high-performance neuroprosthesis for speech decoding and avatar control. Nature (2023), 1–10.
  38. Simple open-vocabulary object detection with vision transformers. arXiv 2022. arXiv preprint arXiv:2205.06230 ([n. d.]).
  39. Machine learning for anomaly detection: A systematic review. Ieee Access 9 (2021), 78658–78700.
  40. Clinical application of an EEG-based brain–computer interface: a case study in a patient with severe motor impairment. Clinical neurophysiology 114, 3 (2003), 399–409.
  41. Vy Nguyen. [n. d.]. Increasing Independence with Stretch: A Mobile Robot Enabling Functional Performance in Daily Activities. ([n. d.]).
  42. World Health Organization and International Spinal Cord Society. 2013. International perspectives on spinal cord injury. World Health Organization.
  43. Clinical trial of the soft extra muscle glove to assess orthotic and long-term functional gain following chronic incomplete tetraplegia: Preliminary functional results. Biosystems & Biorobotics (2018), 385–389. https://doi.org/10.1007/978-3-030-01845-0_77
  44. HAT: Head-Worn Assistive Teleoperation of Mobile Manipulators. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 12542–12548.
  45. Multimodal execution monitoring for anomaly detection during robot manipulation. In 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 407–414.
  46. Mariella Pazzaglia and Marco Molinari. 2016. The embodiment of assistive devices—from wheelchair to exoskeleton. Physics of life reviews 16 (2016), 163–175.
  47. FaceMouse: A human-computer interface for tetraplegic people. In European Conference on Computer Vision. Springer, 99–108.
  48. Shared control–based bimanual robot manipulation. Science Robotics 4, 30 (2019), eaaw0955. https://doi.org/10.1126/scirobotics.aaw0955 arXiv:https://www.science.org/doi/pdf/10.1126/scirobotics.aaw0955
  49. Evaluating Customization of Remote Tele-operation Interfaces for Assistive Robots. arXiv preprint arXiv:2304.02771 (2023).
  50. A systematic review of depression and anxiety measures used with individuals with spinal cord injury. Spinal Cord 47, 12 (2009), 841–851. https://doi.org/10.1038/sc.2009.93
  51. Autonomy in Physical Human-Robot Interaction: A Brief Survey. IEEE Robotics and Automation Letters 6, 4 (Oct. 2021), 7989–7996. https://doi.org/10.1109/LRA.2021.3100603
  52. Dmitry A. Sinyukov and Taskin Padir. 2018. A Novel Shared Position Control Method for Robot Navigation Via Low Throughput Human-Machine Interfaces. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, Madrid, 3913–3920. https://doi.org/10.1109/IROS.2018.8593921
  53. Biomechanical modeling of brachialis-to-wrist extensor muscle transfer function for daily activities in Tetraplegia. JBJS Open Access 7, 3 (2022). https://doi.org/10.2106/jbjs.oa.22.00018
  54. A review of emerging access technologies for individuals with severe motor impairments. Assistive technology 20, 4 (2008), 204–221.
  55. A Robotic Shared Control Teleoperation Method Based on Learning from Demonstrations. International Journal of Advanced Robotic Systems 16, 4 (July 2019), 172988141985742. https://doi.org/10.1177/1729881419857428
  56. Shared Control of a Medical Robot with Haptic Guidance. International Journal of Computer Assisted Radiology and Surgery 12, 1 (Jan. 2017), 137–147. https://doi.org/10.1007/s11548-016-1425-0
  57. High-density Electromyography for Effective Gesture-based Control of Physically Assistive Mobile Manipulators. arXiv preprint arXiv:2312.07745 (2023).
  58. Robotic training and clinical assessment of upper extremity movements after spinal cord injury: a single case report. Journal of rehabilitation medicine 44, 2 (2012), 186.
  59. Human-Robot Shared Control for Surgical Robot Based on Context-Aware Sim-to-Real Adaptation. In 2022 International Conference on Robotics and Automation (ICRA) (Philadelphia, PA, USA). IEEE Press, 7694–7700. https://doi.org/10.1109/ICRA46639.2022.9812379
  60. Human–Robot Shared Control for Humanoid Manipulator Trajectory Planning. Industrial Robot: the international journal of robotics research and application 47, 3 (Feb. 2020), 395–407. https://doi.org/10.1108/IR-10-2019-0217
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube