Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts? (2411.16679v2)

Abstract: We evaluate how well LLMs latently recall and compose facts to answer multi-hop queries like "In the year Scarlett Johansson was born, the Summer Olympics were hosted in the country of". One major challenge in such evaluation is that LLMs may have developed shortcuts by encountering the head entity "Scarlett Johansson" and the answer entity "United States" in the same training sequences or merely guess the answer based on frequency-based priors. To prevent shortcuts, we exclude test queries where the head and answer entities might have co-appeared during training. Through careful selection of relations and facts and systematic removal of cases where models might guess answers or exploit partial matches, we construct an evaluation dataset SOCRATES (ShOrtCut-fRee lATent rEaSoning). We observe that LLMs demonstrate promising latent multi-hop reasoning abilities without exploiting shortcuts, but only for certain types of queries. For queries requiring latent recall of countries as the intermediate answer, the best models achieve 80% latent composability, but this drops to just 5% for the recall of years. Comparisons with Chain-of-Thought highlight a significant gap between the ability of models to reason latently versus explicitly. Analysis reveals that latent representations of the intermediate answer are constructed more often in queries with higher latent composability, and shows the emergence of latent multi-hop reasoning during pretraining.

• The paper introduces a structured evaluation for latent multi-hop reasoning by curating the Socrates dataset to avoid shortcut exploitation.
• The paper reveals that LLMs achieve over 80% accuracy with country-type queries but drop to about 5% with year-type entities.
• The paper contrasts latent reasoning with chain-of-thought methods, highlighting the need for architectural improvements to enhance genuine reasoning capabilities.

Evaluation of Latent Multi-Hop Reasoning in LLMs

The paper "Do LLMs Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?" addresses a crucial aspect of understanding the reasoning capabilities of LLMs by examining their ability to engage in latent multi-hop reasoning. The authors question whether these models can effectively recall and synergize single-hop facts without taking shortcuts that might compromise the reliability of their reasoning pathways. This paper is situated in a broader context that explores the potentials and limitations of factual knowledge representation in AI.

The authors propose a structured approach to assess the latent multi-hop reasoning capabilities of LLMs by introducing the Socrates dataset, which is carefully curated to avoid known shortcut practices. The focus is on measuring how well models can integrate individual facts to answer complex queries without defaulting to the shortcuts created by co-occurrences in training data. The dataset is designed to exclude test cases where head or answer entities co-appear in model pretraining sequences, thus filtering out scenarios where models might bypass the reasoning process.

A range of LLMs were tested using the Socrates dataset, revealing varied performance across different types of queries. The results indicate that models demonstrate strong latent composability with certain queries, particularly those involving country-type intermediate entities, where accuracy exceeds 80%. However, this performance drops significantly in scenarios involving year-type intermediate entities, with accuracy falling to approximately 5%. This discrepancy underscores the contextual variability inherent in multi-hop reasoning tasks and highlights the potential influence of bridge entity types on models’ latent reasoning abilities.

The paper also contrasts latent reasoning with Chain-of-Thought (CoT) reasoning abilities in models. While CoT composability varies less with different task types and scales more efficiently with model size, latent composability shows substantial room for improvement. This discrepancy suggests that while models can articulate thought processes explicitly with CoT reasoning, achieving latent knowledge synergy remains challenging.

An additional layer of analysis was performed using Patchscopes, which showed the rate at which bridge and answer entity representations were constructed during query processing. The findings from these analyses suggest that second-hop reasoning challenges LLMs more than the first hop does, indicating potential differences in cognitive strain when linking successive facts.

One of the notable contributions of this paper is the methodological framework for creating a shortcut-free evaluation environment, a crucial consideration given the propensity of LLMs to leverage learned shortcuts in scenarios that don't require deep reasoning. Future research directions might involve exploring architectural enhancements or alternative training paradigms that better facilitate latent multi-hop reasoning and improving understanding of why certain bridge entity types lead to higher composability.

In conclusion, while the paper elucidates certain nuances of multi-hop reasoning in LLMs, it also highlights the complexities involved in truly evaluating these capabilities without biases introduced by training corpus shortcuts. The rigorous methodological approach and insights into bridge entity influence present opportunities for refining AI models to enhance their compositional reasoning skills, ultimately contributing to more reliable AI systems. The results have significant implications, both theoretical and practical, for developing LLMs that are less reliant on superficial dataset biases and more geared toward genuine reasoning capabilities.