CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans (2406.15823v2)

Abstract: Understanding the abilities of LLMs to reason about natural language plans, such as instructional text and recipes, is critical to reliably using them in decision-making systems. A fundamental aspect of plans is the temporal order in which their steps needs to be executed, which reflects the underlying causal dependencies between them. We introduce CaT-Bench, a benchmark of Step Order Prediction questions, which test whether a step must necessarily occur before or after another in cooking recipe plans. We use this to evaluate how well frontier LLMs understand causal and temporal dependencies. We find that SOTA LLMs are underwhelming (best zero-shot is only 0.59 in F1), and are biased towards predicting dependence more often, perhaps relying on temporal order of steps as a heuristic. While prompting for explanations and using few-shot examples improve performance, the best F1 result is only 0.73. Further, human evaluation of explanations along with answer correctness show that, on average, humans do not agree with model reasoning. Surprisingly, we also find that explaining after answering leads to better performance than normal chain-of-thought prompting, and LLM answers are not consistent across questions about the same step pairs. Overall, results show that LLMs' ability to detect dependence between steps has significant room for improvement.

• The paper introduces the SOP benchmark to evaluate LLMs' ability to determine the correct step order and dependencies in recipe plans.
• It employs binary prediction and explanation tasks, reporting top F1 scores of 0.59 (zero-shot) and 0.73 (with explanations).
• Findings highlight significant limitations in current models, calling for further research to strengthen plan-based reasoning in real-world applications.

Benchmarking LLM Understanding of Causal and Temporal Dependencies in Plans

This paper introduces Step Order Prediction (SOP), a novel benchmark designed to evaluate the capability of LLMs in understanding causal and temporal dependencies within natural language plans, using cooking recipes as a testbed. The benchmark is constructed based on the Recipe Flow Graph Corpus and contains 4280 questions that challenge models to determine the order of steps in a procedure and their interdependencies.

Key Insights

The core focus of the research is on understanding how well cutting-edge LLMs can reason about causal and temporal relationships in instructions. The researchers aim to move beyond traditional settings where LLM performance is typically evaluated on more straightforward tasks such as text generation or sequence prediction. Instead, they delve into more complex reasoning tasks to see if state-of-the-art models can comprehend when a specific step should precede another step to maintain the logical flow of the recipe.

Methodology

The SOP benchmark involves two main tasks:

1. Step Order Prediction (SOP): This involves binary (yes/no) questions testing whether one step in a recipe must occur before or after another step.
2. Step Order Explanation (SOE): Here, models are required to provide explanations for their predictions, helping to elucidate the underlying reasoning for their answers.

The dataset includes both dependent (Dep) and non-dependent (NonDep) pairs of steps, ensuring a balanced representation of scenarios where steps either have a temporal dependency or can occur independently of each other.

Numerical Results

The empirical evaluation shows that existing models exhibit significant limitations in understanding causal and temporal dependencies:

• The best-performing model in the zero-shot setting achieved an F1 score of 0.59, which is only marginally better than random chance, indicating a lack of deep understanding.
• When generating explanations alongside their predictions, model performance improved slightly, with the best F1 score reaching 0.73. However, these results reveal substantial room for improvement, underscoring the complexity of the task.

Additionally, human evaluations of the explanations indicate a mean rating of around 3 on a five-point Likert scale, reflecting that human judges often do not agree with the model's reasoning.

Robustness and Consistency

The paper also explores the robustness of the models using two metrics:

1. Temporal Consistency (TC): Measures the consistency of model predictions across questions asking about before and after relations between the same step pairs.
2. Order Contrastive Consistency (OCC): Evaluates if the model predictions remain consistent when the order of non-dependent step pairs is swapped in the plan.

The results reveal significant inconsistencies, demonstrating that models frequently change their outputs based on the phrasing of the question or the presentation order of steps.

Analysis of Prompting Techniques

The researchers compared different prompting strategies:

• Zero-shot: Providing only the question.
• Answer-then-explain (A+E): Asking for an explanation after predicting the answer.
• Chain-of-Thought (E+A): Generating explanations as intermediate steps before arriving at the final answer.

Surprisingly, the answer-then-explain approach yielded better results than the chain-of-thought prompting, suggesting that current models may benefit from post-hoc rationalization rather than step-by-step reasoning. Few-shot prompting also showed improvements, particularly when relevant examples were dynamically retrieved based on contextual similarity.

Implications and Future Directions

The findings from this work have both practical and theoretical implications:

• Practical: For applications requiring reliable plan-based reasoning, such as cooking assistants, healthcare diagnostics, or workflow automation, the current limitations highlight the potential risks and call for more robust models before deploying in real-world scenarios.
• Theoretical: These results emphasize the need for further research to enhance the causal and temporal reasoning abilities of LLMs. Potential avenues include improved modeling of sequential dependencies, better intermediate reasoning techniques, and more effective use of exemplars for few-shot learning.

Conclusion

This paper contributes significant insights into the current state of LLMs in reasoning about complex dependencies in procedural tasks. Despite improvements with explanation and few-shot prompting, models still fall short in reliably predicting and explaining step dependencies, which underscores the complexity of true plan understanding and the necessity for continued advancements in this area of NLP research.