2000 character limit reached
Robot Behavior Personalization from Sparse User Feedback (2410.19219v1)
Published 25 Oct 2024 in cs.RO and cs.AI
Abstract: As service robots become more general-purpose, they will need to adapt to their users' preferences over a large set of all possible tasks that they can perform. This includes preferences regarding which actions the users prefer to delegate to robots as opposed to doing themselves. Existing personalization approaches require task-specific data for each user. To handle diversity across all household tasks and users, and nuances in user preferences across tasks, we propose to learn a task adaptation function independently, which can be used in tandem with any universal robot policy to customize robot behavior. We create Task Adaptation using Abstract Concepts (TAACo) framework. TAACo can learn to predict the user's preferred manner of assistance with any given task, by mediating reasoning through a representation composed of abstract concepts built based on user feedback. TAACo can generalize to an open set of household tasks from small amount of user feedback and explain its inferences through intuitive concepts. We evaluate our model on a dataset we collected of 5 people's preferences, and show that TAACo outperforms GPT-4 by 16% and a rule-based system by 54%, on prediction accuracy, with 40 samples of user feedback.
- Y. Jiang, A. Gupta, Z. Zhang, G. Wang, Y. Dou, Y. Chen, L. Fei-Fei, A. Anandkumar, Y. Zhu, and L. Fan, “Vima: General robot manipulation with multimodal prompts,” in ICML, 2023.
- M. Shridhar, L. Manuelli, and D. Fox, “Perceiver-actor: A multi-task transformer for robotic manipulation,” pp. 785–799, PMLR, 2023.
- C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, and P. Florence, “Interactive language: Talking to robots in real time,” IEEE RAL, 2023.
- −--, “Learning to learn faster from human feedback with language model predictive control,” 2024.
- M. Patel and S. Chernova, “Proactive robot assistance via spatio-temporal object modeling,” vol. 205, pp. 881–891, PMLR, 2023.
- M. Patel, A. G. Prakash, and S. Chernova, “Predicting routine object usage for proactive robot assistance,” vol. 229, PMLR, 2023.
- E. V. Mascaro, D. Sliwowski, and D. Lee, “HOI4ABOT: Human-object interaction anticipation for human intention reading collaborative roBOTs,” in 7th Annual, 2023.
- I. Kapelyukh and E. Johns, “My house, my rules: Learning tidying preferences with graph neural networks,” pp. 740–749, PMLR, 2022.
- V. Jain, Y. Lin, E. Undersander, Y. Bisk, and A. Rai, “Transformers are adaptable task planners,” pp. 1011–1037, PMLR, 2023.
- A. Peng, A. Netanyahu, M. K. Ho, T. Shu, A. Bobu, J. Shah, and P. Agrawal, “Diagnosis, feedback, adaptation: A human-in-the-loop framework for test-time policy adaptation,” in ICML, 2023.
- L. Yuan, X. Gao, Z. Zheng, M. Edmonds, Y. N. Wu, F. Rossano, H. Lu, Y. Zhu, and S.-C. Zhu, “In situ bidirectional human-robot value alignment,” Science robotics, vol. 7, 2022.
- F. Ranz, V. Hummel, and W. Sihn, “Capability-based task allocation in human-robot collaboration,” Procedia Manufacturing, vol. 9, 2017.
- A. A. Malik and A. Bilberg, “Complexity-based task allocation in human-robot collaborative assembly,” Industrial Robot: the international journal of robotics research and application, vol. 46, 2019.
- N. Gjeldum, A. Aljinovic, M. Crnjac Zizic, and M. Mladineo, “Collaborative robot task allocation on an assembly line using the decision support system,” International Journal of Computer Integrated Manufacturing, vol. 35, 2022.
- M. D. Zhao, R. Simmons, and H. Admoni, “Learning human contribution preferences in collaborative human-robot tasks,” in CORL, 2023.
- C.-A. Smarr, T. L. Mitzner, J. M. Beer, A. Prakash, T. L. Chen, C. C. Kemp, and W. A. Rogers, “Domestic robots for older adults: attitudes, preferences, and potential,” IJSC, vol. 6, 2014.
- J. Saunders, D. S. Syrdal, K. L. Koay, N. Burke, and K. Dautenhahn, ““teach me–show me”—end-user personalization of a smart home and companion robot,” HMS, vol. 46, 2015.
- N. Leonardi, M. Manca, F. Paternò, and C. Santoro, “Trigger-action programming for personalising humanoid robot behaviour,” in CHI, 2019.
- A. Kubota, E. I. Peterson, V. Rajendren, H. Kress-Gazit, and L. D. Riek, “Jessie: Synthesizing social robot behaviors for personalized neurorehabilitation and beyond,” in HRI, 2020.
- S. Zhuang and D. Hadfield-Menell, “Consequences of misaligned ai,” NeurIPS, vol. 33, 2020.
- J. Skalse, N. Howe, D. Krasheninnikov, and D. Krueger, “Defining and characterizing reward gaming,” in NeurIPS, vol. 35, 2022.
- Y. Cui, S. Karamcheti, R. Palleti, N. Shivakumar, P. Liang, and D. Sadigh, “No, to the right: Online language corrections for robotic manipulation via shared autonomy,” in HRI, 2023.
- P. Sharma, B. Sundaralingam, V. Blukis, C. Paxton, T. Hermans, A. Torralba, J. Andreas, and D. Fox, “Correcting robot plans with natural language feedback,” arXiv, 2022.
- Y. Cui, Q. Zhang, B. Knox, A. Allievi, P. Stone, and S. Niekum, “The empathic framework for task learning from implicit human feedback,” vol. 155, PMLR, 2021.
- K. Ramachandruni, M. Zuo, and S. Chernova, “Consor: A context-aware semantic object rearrangement framework for partially arranged scenes,” in IROS, IEEE, 2023.
- D. Lindner, R. Shah, P. Abbeel, and A. Dragan, “Learning what to do by simulating the past,” arXiv, 2021.
- T. Munzer, M. Toussaint, and M. Lopes, “Preference learning on the execution of collaborative human-robot tasks,” in ICRA, IEEE, 2017.
- M. Zhao, R. Simmons, and H. Admoni, “Coordination with humans via strategy matching,” in IROS, IEEE, 2022.
- H. Nemlekar, N. Dhanaraj, A. Guan, S. K. Gupta, and S. Nikolaidis, “Transfer learning of human preferences for proactive robot assistance in assembly tasks,” in ACM/IEEE HRI, 2023.
- D. Hadfield-Menell, S. J. Russell, P. Abbeel, and A. Dragan, “Cooperative inverse reinforcement learning,” NeurIPS, vol. 29, 2016.
- Y. Du, S. Tiomkin, E. Kiciman, D. Polani, P. Abbeel, and A. Dragan, “Ave: Assistance via empowerment,” Advances in Neural Information Processing Systems, vol. 33, pp. 4560–4571, 2020.
- B. Quartey, E. Rosen, S. Tellex, and G. Konidaris, “Verifiably following complex robot instructions with foundation models,” arXiv, 2024.
- C. H. Song, J. Wu, C. Washington, B. M. Sadler, W.-L. Chao, and Y. Su, “Llm-planner: Few-shot grounded planning for embodied agents with large language models,” in IEEE/CVF CVPR, pp. 2998–3009, 2023.
- B. Chen, F. Xia, B. Ichter, K. Rao, K. Gopalakrishnan, M. S. Ryoo, A. Stone, and D. Kappler, “Open-vocabulary queryable scene representations for real world planning,” in ICRA, IEEE, 2023.
- C. Huang, O. Mees, A. Zeng, and W. Burgard, “Visual language maps for robot navigation,” in ICRA, IEEE, 2023.
- D. Sobrín-Hidalgo, M. A. González-Santamarta, Á. M. Guerrero-Higueras, F. J. Rodríguez-Lera, and V. Matellán-Olivera, “Explaining autonomy: Enhancing human-robot interaction through explanation generation with large language models,” arXiv, 2024.
- J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser, “Tidybot: Personalized robot assistance with large language models,” arXiv, 2023.
- B. Zhang and H. Soh, “Large language models as zero-shot human models for human-robot interaction,” in IROS, IEEE, 2023.
- A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, et al., “Rt-1: Robotics transformer for real-world control at scale,” arXiv, 2022.
- K. E. C. Booth, T. T. Tran, G. Nejat, and J. C. Beck, “Mixed-integer and constraint programming techniques for mobile robot task planning,” IEEE RAL, vol. 1, 2016.
- N. T. Dantam, Z. K. Kingston, S. Chaudhuri, and L. E. Kavraki, “Incremental task and motion planning: A constraint-based approach.,” in RSS, vol. 12, 2016.
- D. Das, S. Chernova, and B. Kim, “State2explanation: Concept-based explanations to benefit agent learning and user understanding,” 2023.
- R. Zabounidis, J. Campbell, S. Stepputtis, D. Hughes, and K. P. Sycara, “Concept learning for interpretable multi-agent reinforcement learning,” pp. 1828–1837, PMLR, 2023.
- J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al., “Gpt-4 technical report,” arXiv, 2023.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.