What's the Plan? Evaluating and Developing Planning-Aware Techniques for Language Models (2402.11489v2)

Abstract: Planning is a fundamental task in artificial intelligence that involves finding a sequence of actions that achieve a specified goal in a given environment. LLMs are increasingly used for applications that require planning capabilities, such as web or embodied agents. In line with recent studies, we demonstrate through experimentation that LLMs lack necessary skills required for planning. Based on these observations, we advocate for the potential of a hybrid approach that combines LLMs with classical planning methodology. Then, we introduce SimPlan, a novel hybrid-method, and evaluate its performance in a new challenging setup. Our extensive experiments across various planning domains demonstrate that SimPlan significantly outperforms existing LLM-based planners.

Evaluating and Enhancing Planning-Aware Techniques in LLMs

The paper under review explores the intersection of classical planning methodologies and LLMs, a topic gaining traction due to LLMs' expanding applications in tasks requiring planning and reasoning. Despite the capabilities of LLMs in understanding and generating natural language, their competence in structured problem-solving, such as planning, remains limited. This paper presents a hybrid approach, combining the strengths of LLMs and traditional planning tools, embodied in a new method named SimPlan, to enhance the planning performance of LLMs.

Experimental Insights

The paper initially conducts a detailed analysis of LLMs' performance in planning tasks using a set of well-known domains like Blocksworld, Ferry, Grippers, Depots, and Minigrid. It highlights the deficiencies of LLMs in understanding the effects of actions, predicting applicable actions, and prioritizing goals. These skills are crucial for effective planning, yet experiments show that LLMs struggle significantly. For instance, the success rates in describing the current state and predicting applicable actions are consistently low across different LLM architectures and domains, as shown in Tables 1 and 2. A notable observation is the decreasing accuracy of state estimation with an increasing number of actions, which further underscores the planning limitations of LLMs.

The SimPlan Approach

In response to these limitations, the authors develop SimPlan, a hybrid planning method that integrates action-ranking models with a greedy best-first search (GBFS) algorithm. Unlike standard LLM-based planners reliant on linear generation methods like beam search, SimPlan leverages the exploratory nature of graph-based algorithms. This approach permits deeper exploration of the planning space and effectively manages state transitions, significantly enhancing the accuracy of state representation—a key factor identified as lacking in LLMs.

SimPlan employs a bi-encoder model architecture inspired by ColBERT's late interaction mechanism, optimizing action selection by retrieving them based on semantic similarity with the current state and goals. This model is trained using cross-entropy loss, ensuring diverse and challenging examples through batch sampling and hard negatives, ultimately refining the model’s ability to discern subtle differences between potential actions.

Results and Analysis

The experimental results reveal that SimPlan outperforms existing LLM-based planners and some traditional methods, particularly in complex problem configurations (Table 3). The paper provides a comparative analysis of LLM-based planners, naive baselines, and the proposed hybrid model, illustrating the substantial improvements presented by SimPlan across multiple domains.

A detailed ablation paper further validates the contributions of different components of SimPlan. It highlights the importance of hard negatives, data augmentation strategies, and direct state updates through action execution rather than relying on LLM inferences. These elements are vital in achieving generalizations from simple to complex configurations, which traditional LLM approaches struggle with.

Implications and Future Directions

The research presented in this paper is significant as it articulates the limitations of LLMs in structured tasks and provides a robust alternative through a hybrid methodology. The success of SimPlan indicates a promising path forward for integrating LLMs with classical AI techniques, enhancing their applicability in real-world scenarios involving complex planning tasks.

These findings suggest several future research avenues. One direction could involve extending SimPlan's framework to more diverse tasks and environments beyond those strictly defined by PDDL, thereby leveraging the approach in non-traditional contexts like web navigation and control of autonomous agents. Moreover, exploring techniques that dynamically blend neural and symbolic reasoning at various stages of planning could further optimize efficiency and performance. The paper sets a foundation for these explorations, providing a tangible step towards more capable and flexible AI systems.