In-situ graph reasoning and knowledge expansion using Graph-PReFLexOR (2501.08120v1)

Abstract: The pursuit of automated scientific discovery has fueled progress from symbolic logic to modern AI, forging new frontiers in reasoning and pattern recognition. Transformers function as potential systems, where every possible relationship remains latent potentiality until tasks impose constraints, akin to measurement. Yet, refining their sampling requires more than probabilistic selection: solutions must conform to specific structures or rules, ensuring consistency and the invocation of general principles. We present Graph-PReFLexOR (Graph-based Preference-based Recursive LLMing for Exploratory Optimization of Reasoning), a framework that combines graph reasoning with symbolic abstraction to dynamically expand domain knowledge. Inspired by reinforcement learning, Graph-PReFLexOR defines reasoning as a structured mapping, where tasks yield knowledge graphs, abstract patterns, and ultimately, final answers. Inspired by category theory, it encodes concepts as nodes and their relationships as edges, supporting hierarchical inference and adaptive learning through isomorphic representations. Demonstrations include hypothesis generation, materials design, and creative reasoning, such as discovering relationships between mythological concepts like 'thin places' with materials science. We propose a 'knowledge garden growth' strategy that integrates insights across domains, promoting interdisciplinary connections. Results with a 3-billion-parameter Graph-PReFLexOR model show superior reasoning depth and adaptability, underscoring the potential for transparent, multidisciplinary AI-driven discovery. It lays the groundwork for general autonomous reasoning solutions.

• The paper introduces Graph-PReFLexOR, a framework that fuses graph reasoning with symbolic abstraction for dynamic, recursive knowledge expansion.
• It constructs structured knowledge graphs and generates abstract patterns inspired by category theory, enabling iterative hierarchical inference.
• Demonstrated with a 3-billion-parameter model, the approach enhances interdisciplinary reasoning for applications like hypothesis generation and creative problem solving.

The paper introduces Graph-PReFLexOR (Graph-based Preference-based Recursive LLMing for Exploratory Optimization of Reasoning), a novel framework designed to enhance reasoning and knowledge expansion in AI models. It combines graph reasoning with symbolic abstraction to dynamically expand domain knowledge. The authors formalize reasoning as a structured mapping $$\mathcal{M}: \mathcal{T} \rightarrow (\mathcal{G}, \mathcal{P}, \mathcal{A})$$, where tasks $\mathcal{T}$ yield knowledge graphs $\mathcal{G}$, abstract patterns $\mathcal{P}$, and final answers $\mathcal{A}$.

Key aspects of the framework include:

• Knowledge Graph Construction: The model constructs knowledge graphs $\mathcal{G} = (V, E)$, where nodes $V$ represent concepts and edges $E$ represent relationships between them.
• Abstract Pattern Generation: Inspired by category theory, the model encodes concepts as nodes and their relationships as edges, supporting hierarchical inference and adaptive learning through isomorphic representations.
• Recursive Reasoning: The framework employs iterative refinement of the knowledge graph and symbolic abstractions.

The paper draws inspiration from category theory and reinforcement learning, encoding concepts as nodes and relationships as edges to support hierarchical inference. The approach facilitates discovery across domains, integrating insights to promote interdisciplinary connections. The authors demonstrate the efficacy of Graph-PReFLexOR in hypothesis generation, materials design, and creative reasoning, including discovering relationships between mythological concepts and materials science.

The authors implemented a 3-billion-parameter Graph-PReFLexOR model to demonstrate its capabilities, showing enhanced reasoning depth and adaptability. The work lays a foundation for autonomous reasoning solutions.

The authors also state that Transformers implicitly function as graph isomorphism neural networks [Buehler2025GraphAwareGPT], which provides a powerful approach for explicitly integrating graph-based reasoning into these architectures.

The paper builds upon earlier work on PReFLexOR [buehler2024preflexorpreferencebasedrecursivelanguage], expanding the concept of "thinking before answering" to incorporate in situ graph-based reasoning. The method mimics the reflective, iterative reasoning processes found in scientific inquiry. By enabling models to autonomously construct and manipulate symbolic graph representations, the model mimics reflective and iterative reasoning processes. Knowledge is represented as a superposition of potentialities that collapse into specific outputs when conditioned on tasks, balancing structured coherence with divergent exploration.

The structured reasoning is organized into headings such as Core Concepts and Relationships',Reasoning Steps', and `Abstract Patterns', formalizing reasoning as a structured mapping $$\mathcal{M}: \mathcal{T} \rightarrow (\mathcal{G}, \mathcal{P}, \mathcal{A})$$.

The authors train the model using a multi-stage training process, featuring ORPO and then DPO-EXO, on top of the meta-llama/Llama-3.2-3B-Instruct model.

The authors introduced a novel approach that unifies the linguistic fluency of LLMs with the relational reasoning capabilities of Graph Neural Networks (GNNs). Through symbolic representations via special tokens, the model engages in a "thinking phase," reasoning over the graph to refine its understanding before generating an answer.

The authors use Graph Isomorphism Networks (GINs) to model isomorphisms computationally, leveraging their ability to operate on graph-structured data.

The authors introduce a novel approach that unifies the linguistic fluency of LLMs with the relational reasoning capabilities that have been quite successful in architectures such as Graph Neural Networks (GNNs).

This work pushes the boundaries of what AI can achieve in scientific domains [rumelhart1986learning, newell1972human, mccarthy1960programs], creating models that have a more explicit process of relational deliberation, both symbolically and structurally, before answering.

The plan of the paper includes a review of the PReFLexOR architecture [buehler2024preflexorpreferencebasedrecursivelanguage], the training process of developing Graph-PReFLexOR, and case studies of how the method can be applied.

In one example, the model relates music and materials, formalizing their relationship through an abstract pattern derived from their fundamental interactions, expressed as the triple system $(\alpha, \beta, \gamma)$.

• $\alpha$ represents music as an audio signal.
• $\beta$ represents the material as a physical substance.
• $\gamma$ represents the material's mechanical properties.

The core relationship follows the pattern $\alpha \rightarrow \beta \rightarrow \gamma$ with the transformation rule $\alpha \propto \beta$ and essential condition $\gamma \rightarrow \alpha$, proposing a closed feedback loop.

The authors also demonstrate in-situ graph generation and recursive reasoning through multi-agent modeling. The model can expand its capabilities beyond the materials-science focused training data and successfully integrate different domains into the structured reasoning paradigm. For example, recursive reasoning is performed using a two-agent setup, with thinking steps critiqued, improved, and fed back to the model, resulting in an integrated response.

The authors also develop an idea they call "knowledge garden growth" via in-situ graph generation and knowledge expansion. In this experiment, the graph reasoning model generates graphs describing phenomena in a relational manner, then applies it to explore ever-expanding graph structures by repeatedly prompting the model. This recursively grows a graph, starting from an initial concept or response to a task, to examine how the model can blend creative and analytical reasoning.