Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

To the Noise and Back: Diffusion for Shared Autonomy (2302.12244v3)

Published 23 Feb 2023 in cs.RO and cs.LG

Abstract: Shared autonomy is an operational concept in which a user and an autonomous agent collaboratively control a robotic system. It provides a number of advantages over the extremes of full-teleoperation and full-autonomy in many settings. Traditional approaches to shared autonomy rely on knowledge of the environment dynamics, a discrete space of user goals that is known a priori, or knowledge of the user's policy -- assumptions that are unrealistic in many domains. Recent works relax some of these assumptions by formulating shared autonomy with model-free deep reinforcement learning (RL). In particular, they no longer need knowledge of the goal space (e.g., that the goals are discrete or constrained) or environment dynamics. However, they need knowledge of a task-specific reward function to train the policy. Unfortunately, such reward specification can be a difficult and brittle process. On top of that, the formulations inherently rely on human-in-the-loop training, and that necessitates them to prepare a policy that mimics users' behavior. In this paper, we present a new approach to shared autonomy that employs a modulation of the forward and reverse diffusion process of diffusion models. Our approach does not assume known environment dynamics or the space of user goals, and in contrast to previous work, it does not require any reward feedback, nor does it require access to the user's policy during training. Instead, our framework learns a distribution over a space of desired behaviors. It then employs a diffusion model to translate the user's actions to a sample from this distribution. Crucially, we show that it is possible to carry out this process in a manner that preserves the user's control authority. We evaluate our framework on a series of challenging continuous control tasks, and analyze its ability to effectively correct user actions while maintaining their autonomy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Adaptive virtual fixtures for machine-assisted teleoperation tasks. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2005.
  2. A topology of shared control systems—finding common ground in diversity. IEEE Transactions on Human-Machine Systems, 48(5), 2018.
  3. Human integration into robot control utilising potential fields. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 1997.
  4. Brenna D. Argall. Modular and adaptive wheelchair automation. In Proceedings of the International Symposium on Experimental Robotics (ISER), 2016.
  5. Towards automated sample collection and return in extreme underwater environments. arXiv preprint arXiv:2112.15127, 2021.
  6. Highly parallelized data-driven MPC for minimal intervention shared control. In Proceedings of Robotics: Science and Systems (RSS), 2019.
  7. OpenAI Gym, 2016.
  8. ASHA: Assistive teleoperation via human-in-the-loop reinforcement learning. arXiv preprint arXiv:2202.02465, 2022.
  9. PyBullet, a Python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016–2021.
  10. Characterizing efficiency of human robot interaction: A case study of shared-control teleoperation. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2002.
  11. Cooperative human and machine perception in teleoperated assembly. In Proceedings of the International Symposium on Experimental Robotics (ISER), 2000.
  12. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
  13. Formalizing assistive teleoperation. In Proceedings of Robotics: Science and Systems (RSS), 2012.
  14. A policy-blending formalism for shared control. International Journal of Robotics Research, 32(7), 2013.
  15. AvE: Assistance via empowerment. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
  16. Incorporating second-order functional knowledge for better option pricing. In Advances in Neural Information Processing Systems (NeurIPS), 2000.
  17. Implicit behavioral cloning. In Proceedings of the Conference on Robot Learning (CoRL), 2021.
  18. D4RL: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219, 2020.
  19. Cristian Garcia. Denoising diffusion probabilistic models, 2021. URL https://github.com/acids-ircam/diffusion_models. Online.
  20. A decision-theoretic approach for the collaborative control of a smart wheelchair. International Journal of Robotics Research, 10:131–145, 2017.
  21. Ray C. Goertz. Manipulators used for handling radioactive materials. Human Factors in Technology, 1963.
  22. Decision-making authority, team efficiency and human worker satisfaction in mixed human—robot teams. In Proceedings of Robotics: Science and Systems (RSS), 2014.
  23. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. CoRR, abs/1801.01290, 2018.
  24. Kris Hauser. Recognition, prediction, and planning for assisted teleoperation of freeform tasks. Autonomous Robots, 35(4), 2013.
  25. Denoising diffusion probabilistic models. CoRR, abs/2006.11239, 2020.
  26. Effects of anticipatory action on human-robot teamwork efficiency, fluency, and perception of team. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2007.
  27. Planning with diffusion for flexible behavior synthesis. In Proceedings of the International Conference on Machine Learning (ICML), 2022.
  28. Shared autonomy via hindsight optimization. In Proceedings of Robotics: Science and Systems (RSS), 2015.
  29. Continuous shared control for stabilizing reaching and grasping with brain-machine interfaces. IEEE Transactions on Biomedical Engineering, 53(6), 2006.
  30. Teleoperation of a robot manipulator using a vision-based human-robot interface. IEEE Transactions on Industrial Electronics, 52(5), 2005.
  31. Accelerating diffusion models via early stop of the diffusion process. arXiv preprint arXiv:2205.12524, 2022.
  32. SDEdit: Image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073, 2021.
  33. Learning strategies in table tennis using inverse reinforcement learning. Biological Cybernetics, 108(5), 2014.
  34. Autonomy infused teleoperation with application to brain computer interface controlled manipulation. Autonomous Robots, 41(6), August 2017.
  35. Learning to arbitrate human and robot control using disagreement between sub-policies. arXiv preprint arXiv:2108.10634, 2021.
  36. Imitating human behaviour with diffusion models. In Proceedings of the International Conference on Learning Representations (ICLR), 2023.
  37. Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2015.
  38. DreamFusion: Text-to-3D using 2D diffusion. arXiv preprint arXiv:2209.14988, 2022.
  39. Shared autonomy via deep reinforcement learning. In Proceedings of Robotics: Science and Systems (RSS), 2018.
  40. First contact: Unsupervised human-machine co-adaptation via mutual information maximization. arXiv preprint arXiv:2205.12381, 2022.
  41. Louis B. Rosenberg. Virtual fixtures: Perceptual tools for telerobotic manipulation. In Proceedings of the Virtual Reality Annual International Symposium (VRAIS), 1993.
  42. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.11487, 2022.
  43. Residual policy learning for shared autonomy. In Proceedings of Robotics: Science and Systems (RSS), 2020.
  44. An autonomous robotic assistant for drinking. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2015.
  45. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the International Conference on Machine Learning (ICML), 2015.
  46. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
  47. On optimizing interventions in shared autonomy. arXiv preprint arXiv:2112.09169, 2021.
  48. MuJoCo: A physics engine for model-based control. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012.
  49. Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23(7):1661–1674, 2011.
  50. Score Jacobian chaining: Lifting pretrained 2D diffusion models for 3D generation. arXiv preprint arXiv:2212.00774, 2022a.
  51. Diffusion policies as an expressive policy class for offline reinforcement learning. arXiv preprint arXiv:2208.06193, 2022b.
  52. Zero shot image restoration using denoising diffusion null-space model. arXiv preprint arXiv:2212.00490, 2022.
  53. Telemanipulation assistance based on motion intention recognition. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2005.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Takuma Yoneda (14 papers)
  2. Luzhe Sun (4 papers)
  3. and Ge Yang (1 paper)
  4. Bradly Stadie (6 papers)
  5. Matthew Walter (8 papers)
Citations (22)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now