Large Language Models can Learn Rules (2310.07064v3)

Abstract: When prompted with a few examples and intermediate steps, LLMs have demonstrated impressive performance in various reasoning tasks. However, prompting methods that rely on implicit knowledge in an LLM often generate incorrect answers when the implicit knowledge is wrong or inconsistent with the task. To tackle this problem, we present Hypotheses-to-Theories (HtT), a framework that learns a rule library for reasoning with LLMs. HtT contains two stages, an induction stage and a deduction stage. In the induction stage, an LLM is first asked to generate and verify rules over a set of training examples. Rules that appear and lead to correct answers sufficiently often are collected to form a rule library. In the deduction stage, the LLM is then prompted to employ the learned rule library to perform reasoning to answer test questions. Experiments on relational reasoning, numerical reasoning and concept learning problems show that HtT improves existing prompting methods, with an absolute gain of 10-30% in accuracy. The learned rules are also transferable to different models and to different forms of the same problem.

LLMs Can Learn Rules: An Analytical Perspective

The paper "LLMs can Learn Rules" presents an innovative approach to enhancing the reasoning capabilities of LLMs by employing a novel framework called Hypotheses-to-Theories (HtT). The core motivation behind HtT is to address the prevalent issue of hallucinations in LLMs, which occur when the model generates outputs that seem plausible but are incorrect. This often arises from the mismatch between the implicit knowledge embedded in the model during pretraining and the explicit knowledge required for specific reasoning tasks.

Framework Overview

HtT is structured into two main stages: an induction stage and a deduction stage. During the induction stage, the model is prompted to generate rules from a set of training examples and subsequently verify these rules. The rules that frequently lead to correct outcomes are collected into a rule library. The deduction stage involves using this learned rule library to guide the LLM in solving reasoning problems.

Empirical Results

The empirical evaluation demonstrates substantial improvements in accuracy, with HtT providing an absolute gain of 11-27% in various reasoning tasks compared to baseline prompting methods. These include numerical reasoning challenges exemplified by arithmetic in non-decimal systems, and relational reasoning tasks as demonstrated by the CLUTRR dataset. Notably, the learned rules showed transferability across models and different problem formulations.

Technical Contributions

1. Rule Generation and Verification: The induction stage uniquely uses the capability of LLMs to hypothesize and empirically verify rules, thereby reducing reliance solely on the implicit knowledge of the models.
2. Induction from Deduction: This strategy simplifies prompt engineering by merging rule generation and verification under a single deductive reasoning prompt, leveraging existing techniques like chain-of-thought.
3. XML Tagging for Rule Retrieval: To enhance the model’s in-context retrieval abilities, the paper introduces an XML tagging mechanism that organizes rules hierarchically. This facilitates effective retrieval even with a large rule set.

Implications and Future Directions

The development of HtT has significant implications for the field of AI, particularly in improving the robustness and accuracy of LLMs in reasoning tasks. By providing a method to learn and apply explicit knowledge, LLMs become less dependent on probabilistic guessing and more capable of structured reasoning. This could enhance their applicability in areas requiring high accuracy, such as legal reasoning, financial forecasting, and scientific discovery.

Future research could explore the scalability of this approach, especially for models with longer context lengths and larger rule libraries. Additionally, integrating fine-tuning for retrieval capabilities could further enhance performance. There remains an open challenge to refine the learning of complex rules that span multiple reasoning steps, potentially through more advanced machine learning techniques or integrating external symbolic reasoning systems.

Conclusion

The paper offers a promising direction for enhancing the reasoning abilities of LLMs by reducing hallucinations through explicit rule learning and application. The Hypotheses-to-Theories framework presents a structured method that leverages both the strengths of LLMs and the necessity for explicit reasoning, marking a notable advancement in computational reasoning methodologies.