2000 character limit reached
LLM-SAP: Large Language Models Situational Awareness Based Planning (2312.16127v5)
Published 26 Dec 2023 in cs.AI
Abstract: This study explores integrating LLMs with situational awareness-based planning (SAP) to enhance the decision-making capabilities of AI agents in dynamic and uncertain environments. We employ a multi-agent reasoning framework to develop a methodology that anticipates and actively mitigates potential risks through iterative feedback and evaluation processes. Our approach diverges from traditional automata theory by incorporating the complexity of human-centric interactions into the planning process, thereby expanding the planning scope of LLMs beyond structured and predictable scenarios. The results demonstrate significant improvements in the model's ability to provide comparative safe actions within hazard interactions, offering a perspective on proactive and reactive planning strategies. This research highlights the potential of LLMs to perform human-like action planning, thereby paving the way for more sophisticated, reliable, and safe AI systems in unpredictable real-world applications.
- “Core challenges in embodied vision-language planning,” Journal of Artificial Intelligence Research, vol. 74, pp. 459–515, 2022.
- Mica R. Endsley, “Toward a theory of situation awareness in dynamic systems,” Human Factors, vol. 37, no. 1, pp. 32–64, 1995.
- Wikipedia contributors, “Situation awareness — Wikipedia, the free encyclopedia,” 2023, [Online; accessed 25-December-2023].
- Dipendra Yadav, “Evaluating dangerous capabilities of large language models: An examination of situational awareness,” DC@KI2023: Proceedings of Doctoral Consortium at KI 2023, 2023.
- “Concrete problems in ai safety,” 2016.
- Stuart Russell, “Human compatible. ai and the problem of control, london: Allen lane,” 2019.
- “Do as i can, not as i say: Grounding language in robotic affordances,” arXiv preprint arXiv:2204.01691, 2022.
- “Rt-2: Vision-language-action models transfer web knowledge to robotic control,” in Conference on Robot Learning. PMLR, 2023, pp. 2165–2183.
- “Open x-embodiment: Robotic learning datasets and rt-x models,” arXiv preprint arXiv:2310.08864, 2023.
- “Palm-e: An embodied multimodal language model,” arXiv preprint arXiv:2303.03378, 2023.
- “Progprompt: Generating situated robot task plans using large language models,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 11523–11530.
- “Housekeep: Tidying virtual households using commonsense reasoning,” in European Conference on Computer Vision. Springer, 2022, pp. 355–373.
- “Integrating action knowledge and llms for task planning and situation handling in open worlds,” Autonomous Robots, 2023.
- “Llm-planner: Few-shot grounded planning for embodied agents with large language models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023.
- “Llm-brain: Ai-driven fast generation of robot behaviour tree based on large language model,” arXiv preprint arXiv:2305.19352, 2023.
- “Inner monologue: Embodied reasoning through planning with language models,” in arXiv preprint arXiv:2207.05608, 2022.
- “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” in International Conference on Machine Learning. PMLR, 2022, pp. 9118–9147.
- “Transformers are adaptable task planners,” in Conference on Robot Learning. PMLR, 2023, pp. 1011–1037.
- “Chatgpt for robotics: Design principles and model abilities,” Tech. Rep. MSR-TR-2023-8, Microsoft, February 2023.
- “Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents,” arXiv preprint arXiv:2302.01560, 2023.
- “Improving factuality and reasoning in language models through multiagent debate,” 2023.
- “Encouraging divergent thinking in large language models through multi-agent debate,” 2023.
- “Camel: Communicative agents for ”mind” exploration of large language model society,” 2023.
- Wikipedia contributors, “Finite-state machine — Wikipedia, the free encyclopedia,” 2023, [Online; accessed 17-December-2023].
- “Gpt-4 technical report,” 2023.
- “Coverage criteria for state-based testing: A systematic review,” International Journal of Information Technology Project Management, vol. 10, pp. 1–20, 01 2019.
- Wikipedia contributors, “Cyclomatic complexity — Wikipedia, the free encyclopedia,” 2023, [Online; accessed 19-December-2023].
- Modeling software with finite state machines: a practical approach, CRC Press, 2006.
- “Housekeep: Tidying virtual households using commonsense reasoning,” 2022.
- “Robogen: Towards unleashing infinite data for automated robot learning via generative simulation,” arXiv preprint arXiv:2311.01455, 2023.
- “Reliability of human evaluation for text summarization: Lessons learned and challenges ahead,” in Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval), Anya Belz, Shubham Agarwal, Yvette Graham, Ehud Reiter, and Anastasia Shimorina, Eds., Online, Apr. 2021, pp. 86–96, Association for Computational Linguistics.
- “Evaluation metrics in the era of gpt-4: Reliably evaluating large language models on sequence to sequence tasks,” 2023.
- Anthropic, “Claude (version 2),” 2023, [Computer software].
- “Llama 2: Open foundation and fine-tuned chat models,” 2023.
- “Visual instruction tuning,” in NeurIPS, 2023.
- “Judging llm-as-a-judge with mt-bench and chatbot arena,” 2023.
- “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” 2023.
- “Code llama: Open foundation models for code,” 2023.
- “Large language models are zero-shot reasoners,” 2023.
- “Large language models understand and can be enhanced by emotional stimuli,” 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
