Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents (2201.07207v2)

Abstract: Can world knowledge learned by LLMs be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. "make breakfast"), to a chosen set of actionable steps (e.g. "open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into mid-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from LLMs. Website at https://huangwl18.github.io/language-planner

Overview of "LLMs as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents"

The paper "LLMs as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents" by Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch explores the potential of leveraging LLMs such as GPT-3 and Codex to generate actionable plans for high-level tasks in interactive environments without additional training.

Core Contributions and Findings

This research presents several notable contributions to the field:

1. Zero-Shot Task Decomposition:
   • It is demonstrated that sufficiently large pre-trained LLMs can decompose high-level tasks (e.g., "make breakfast") into mid-level action steps (e.g., "open fridge") purely through appropriate prompting, without the need for further fine-tuning or training on specific scenarios.
2. Executable Plan Enhancement:
   • The paper identifies that action plans generated by LLMs in their raw form often fail to be executable due to linguistic ambiguities and formulation issues. As a solution, the authors propose several techniques to transform these plans into sequences that can be realistically executed in an embodied environment like VirtualHome. These transformations include semantic similarity-based action translation and autoregressive step-conditioned generation.
3. Human Evaluation Metrics:
   • Human-evaluated correctness and computational measures of executability are used to assess the quality and feasibility of the generated plans. The evaluation framework crucially involves human judgments to determine whether action sequences complete the intended tasks appropriately.

Techniques and Methodologies

The methodologies proposed in the paper include:

1. Semantic Action Translation:
   • By integrating a Translation LM (e.g., Sentence-RoBERTa), the approach converts generated action steps into the closest admissible actions through sentence embeddings and cosine similarity measures.
2. Autoregressive Trajectory Correction:
   • Instead of generating entire action sequences in one go, the method generates action steps iteratively, correcting each step to ensure they remain executable by constantly referring to a pre-defined set of admissible actions.
3. Dynamic Example Selection:
   • Enhancing the in-context learning capabilities of LLMs, the process dynamically chooses the most similar example task from a demonstration set to condition the model suitably, improving the generation of contextually appropriate action plans.

Results and Performance

Key results of the paper show:

• The application of the proposed semantics-based translation improves executability from a baseline of 18% to around 79%, albeit with a trade-off in correctness levels, falling slightly compared to the generative baseline evaluated solely on natural language outputs.
• The experimental setups and analysis demonstrate that while LLMs contain substantial actionable knowledge and can generate plans of high semantic quality, making these plans executable requires substantial intervention through semantic translation and corrective techniques.
• Larger models (e.g., GPT-3 and Codex) show differentiation in their ability to generate realistic and contextually correct plans but often at the cost of higher erroneous or non-executable steps compared to their smaller counterparts.

Implications and Future Directions

This research opens several pathways for future work and implications in AI:

1. Practical Integration in Robotics:
   • By enhancing the ability to generate executable plans, this work bridges a critical gap in the deployment of AI systems in home automation and interactive robotic agents, making them more autonomous and contextually aware.
2. Advancements in Human-AI Interaction:
   • Techniques to dynamically prompt LLMs and conditionally generate action plans could enhance virtual assistants and AI companions, providing more robust and context-sensitive interactions.
3. Theoretical Implications:
   • The paper further validates the hypothesis that LLMs learn significant world knowledge during pre-training. However, the application of this knowledge to actionable contexts requires sophisticated translation and grounding techniques.
4. Model Fine-Tuning and Adaptation:
   • Future work might focus on fine-tuning strategies that further reduce the error rates in action plan generation, marrying the strengths of large-scale pre-training with fine-tuned contextual adjustments.
5. Enhanced Evaluation Frameworks:
   • Developing more nuanced metrics and evaluation frameworks that better capture the semantic and pragmatic correctness of generated plans could provide deeper insights into improving LLM performance in interactive settings.

Conclusion

The paper makes meaningful strides in using LLMs for generating actionable plans in embodied environments, highlighting both the latent potential and the challenges inherent in this novel application of AI. By proposing semantic enhancement techniques and demonstrating their efficacy, the research provides a robust foundation for further advancements in interactive, goal-driven AI systems.