Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus (2411.12498v2)

Abstract: LLMs are capable of solving a wide range of tasks, yet they have struggled with reasoning. To address this, we propose $\textbf{Additional Logic Training (ALT)}$, which aims to enhance LLMs' reasoning capabilities by program-generated logical reasoning samples. We first establish principles for designing high-quality samples by integrating symbolic logic theory and previous empirical insights. Then, based on these principles, we construct a synthetic corpus named $\textbf{Formal Logic Deduction Diverse}$ ($\textbf{FLD}$${\times 2}$), comprising numerous samples of multi-step deduction with unknown facts, diverse reasoning rules, diverse linguistic expressions, and challenging distractors. Finally, we empirically show that ALT on FLD${\times2}$ substantially enhances the reasoning capabilities of state-of-the-art LLMs, including LLaMA-3.1-70B. Improvements include gains of up to 30 points on logical reasoning benchmarks, up to 10 points on math and coding benchmarks, and 5 points on the benchmark suite BBH.

• The paper introduces ALT that uses a synthetic logic corpus to boost LLM reasoning, achieving up to 30-point improvements on logical benchmarks.
• It details the creation of the Principled Logic Dataset with axiomatic samples and varied linguistic expressions for robust logic training.
• Experimental results demonstrate cross-domain benefits, enhancing math, coding, and NLI performance while preventing knowledge forgetting.

The paper "Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus" addresses a critical aspect of artificial intelligence development: improving the reasoning capabilities of LLMs, which traditionally exhibit limitations in reasoning tasks despite their proficiency in various other domains. The authors propose a novel approach termed Additional Logic Training (ALT) to address this limitation. Here's a detailed breakdown of the paper's contributions and findings:

Introduction and Motivation

• The paper begins by highlighting the importance of reasoning in AI, distinguishing between knowledge (facts about the world) and reasoning (the ability to derive new knowledge by combining facts according to specific rules).
• Despite the capacity of LLMs to perform diverse tasks, they struggle with novel, unseen problems requiring reasoning rather than relying on memorized knowledge.

Approach: Additional Logic Training (ALT)

• ALT is introduced as a method to enhance LLMs' reasoning abilities by training them on logic samples generated programmatically. This method involves using a synthetic corpus grounded in symbolic logic theory.
• The paper develops principles for creating high-quality synthetic logic samples by integrating empirical insights and symbolic logic.

Synthetic Corpus: Principled Logic Dataset (PLD)

• The authors construct a synthetic corpus named the Principled Logic Dataset (PLD), designed to cover multiple aspects of logical reasoning, including multi-step deduction, diverse linguistic expressions, and challenging distractors. This corpus aims to address the limitations of human-written texts that LLMs are initially trained on.
• The PLD is characterized by:
  • Samples incorporating unknown facts to emphasize reasoning beyond memorized knowledge.
  • Inclusion of illogical reasoning examples to help LLMs discern logic from fallacy.
  • Use of axiomatic and theorem-based samples to cover diverse reasoning rules efficiently, drawing from Gödel's completeness theorem that any reasoning rule can be reduced to these axioms.
  • Varied linguistic expressions to prevent bias towards specific phrases, thus enhancing generalization across different language constructs.

Experimental Results

• Empirical Validation: The paper conducts extensive experiments to validate the effectiveness of ALT on LLMs, employing various benchmarks for logical reasoning, math, coding, and natural language inference (NLI).
• Significant Improvements: ALT resulted in substantial improvements in reasoning tasks, with performance gains of up to 30 points on logical reasoning benchmarks. It also demonstrated up to 10-point improvements in math and coding tasks and notable gains in NLI tasks, indicating generalization beyond the original training corpus.
• Prevention of Knowledge Forgetting: The use of the Recall Adam optimizer is critical in preventing the forgetful displacement of previously acquired knowledge during additional training with synthetic samples.

Analysis and Insights

• The paper provides insights into why ALT enhances capabilities across a variety of tasks:
  • Logical Reasoning: LLMs show improvements in handling logical operations and identifying fallacies, attributed to exposure to synthetic logic samples tailored for diverse deduction rules.
  • Transference to Math and Coding: Enhanced logical reasoning transfers into mathematical problem-solving and coding, indicating the interconnectedness of reasoning skills across domains.
  • Abductive Reasoning and NLI: Despite not being directly trained on abductive reasoning tasks, the trained LLMs improved, suggesting that fundamental logical principles acquired through ALT aid in broader reasoning scenarios.

Conclusion and Contributions

• This work contributes a methodological advancement in training LLMs for reasoning, leveraging synthetic data to supplement traditional pre-training approaches.
• It establishes a framework for creating synthetic logic samples that can be integral to developing versatile AI systems equipped with robust reasoning capabilities.
• The authors release the corpus, code, and trained models, facilitating further research and exploration in enhancing AI reasoning through synthetic corpora.

Overall, the paper presents a comprehensive strategy for augmenting LLMs' problem-solving abilities, advancing the goal of developing truly versatile AI systems that integrate both knowledge and reasoning capabilities. This work holds promise for enhancing the interpretability and reliability of AI applications across diverse fields.