Visual Graph Scaffolds for Structural Reasoning in Large Language Models

Abstract: Graphs have been used to enhance LLMs for structured reasoning, mostly as external knowledge sources are provided to models at test time. In this paper, we take a different view: the value of graphs for LLMs lie not only in supplying information, but also in organizing reasoning. Inspired by how humans use graph-structured mind maps to organize branching and converging thoughts, we ask whether graphs can serve as an internal form of reasoning assistance. We study this question on multi-hop question answering tasks, where teacher-provided reasoning traces are rewritten as graph mind maps and used to guide a student model. Our experiments reveal a clear modality gap. When graph structures are flattened into text, their benefits become limited once direct answer hints are removed. Under this abstract guidance setting, both reasoning efficiency and answer quality degrade substantially. In contrast, visual graph guidance remains effective without direct answer clues, and its advantage persists after supervised fine-tuning and KL-based distillation. The above findings support the claim that graphs should be studied not only as external knowledge structures for LLMs, but also as visual scaffolds for organizing reasoning.

• The paper demonstrates that visual graph guidance preserves structural reasoning under abstract constraints, achieving 70.80% accuracy versus 51.97% for text guidance and 46.40% for graph-to-text controls.
• The paper shows that the advantage persists after internalization through Self-SFT and KL distillation, with image-guided models outperforming text-guided models while remaining below direct teacher chain-of-thought training.
• The paper finds that topology—not images alone—drives the benefit, as linearizing graphs or limiting node budgets causes roughly 14–16-point accuracy drops, while visual scaffolds reduce abstract reasoning outputs from about 703 to 226 tokens.

Motivation and central claim

This paper, by Lei, Xiao, and Wei (2606.02673), argues that graphs should be studied not only as external knowledge sources for LLMs but as interfaces for organizing reasoning itself. Existing graph–LLM work—GraphRAG-style retrieval over entity–relation graphs, subgraph retrieval as in G-Retriever, and knowledge-graph traversal in ToG-2—treats the graph as a store of facts to be retrieved or grounded. The authors instead ask whether a graph can transfer the structure of a successful reasoning process from a stronger teacher to a weaker student model. Their position is that a rendered graph image acts as a topology-preserving interface: branching, convergence, and local dependencies remain spatially inspectable rather than being serialized into linear text.

The study is conducted in a teacher–student setting on multi-hop QA. A teacher (DeepSeek-V3.2) solves questions that a student (Qwen3-VL-8B-Instruct) initially fails; verified teacher trajectories are rewritten into guidance artifacts consumed by the student, and successful guided behavior is later internalized via Self-SFT or KL distillation.

Experimental design

The pipeline distinguishes two axes of variation:

• Modality: image guidance (teacher reasoning rendered as a Graphviz mind map) versus text guidance expressing comparable support in plain text, plus a graph-to-text control that converts the generated DOT code into text, preserving node content while removing visual layout.
• Content style: direct guidance may include key facts, intermediate conclusions, and answer-local hints (though never the final answer); abstract guidance forbids all answer-specific clues and may contain only general strategies and logical operations.

The abstract setting is the paper's critical test: it forces the student to use the scaffold to reason rather than as a shortcut to the answer. Experiments cover HotpotQA, 2WikiMultiHopQA, and MuSiQue, with 14,490 teacher-correct supervision cases, a held-out test set of 3,000 questions, and a separate 3,000-example ablation subset. The verifier is Qwen3-8B-Instruct.

Main findings

The modality gap emerges only under abstract guidance. In guided re-evaluation with a frozen student, direct image and direct text guidance are essentially tied (71.21% vs. 71.22%). Under the abstract constraint, however, image guidance retains near-full effectiveness at 70.80%, while text guidance falls to 51.97% and the graph-to-text control drops further to 46.40%. This is the paper's strongest quantitative result: removing answer-local hints costs visual guidance almost nothing but costs textual guidance roughly twenty accuracy points. The implication is that when guidance must teach how to reason rather than reveal what to recover, preserved topology carries information that flattened text does not.

The advantage survives internalization. After Self-SFT on the student's own correct guided responses, image-guided training reaches 64.00% (direct) and 63.27% (abstract) on held-out QA versus 61.53% and 58.23% for text. KL distillation shows the same ordering, with the largest gap again in the abstract condition (64.47% vs. 58.37%). All internalized variants improve over the CoT baseline of 59.37%, though none matches direct SFT on the teacher's full chain-of-thought (67.17%).

Images yield shorter outputs. In abstract re-evaluation, image-guided students average 226 output tokens versus 703 for text and 697 for graph-to-text; after training the gap persists (377 vs. 780/810 tokens under Self-SFT). The authors interpret this as compression: the student recovers the reasoning pattern from a compact structural interface rather than unpacking long sequential instructions.

Topology, not visual presentation per se, carries the signal. Ablating structure in abstract image guidance causes large drops: forcing the graph into a linear chain reduces re-evaluation accuracy from 72.23% to 57.82% ($\Delta = -14.41$), and restricting node budgets to five or ten nodes yields drops of roughly sixteen points. Since these ablations preserve the visual modality, the degradation must be attributed to lost branching and convergence rather than to images as such.

Limitations and open questions

The paper states two limitations plainly. First, transfer beyond the QA family is weak: on an out-of-domain benchmark of six reasoning datasets (CLUTRR, LogiQA, AR-LSAT, AIW-Easy/Hard, Bamboogle), image-guided models score only 39.15% against a 47.33% CoT baseline, even though the ordering image > text (28.42%) holds. The pipeline is therefore task-specific, and whether broadly reusable structured-reasoning capability can be established remains open. Second, image guidance never outperforms direct training on teacher CoT within the QA domain; the contribution is a better interface under restricted supervision, not a replacement for standard CoT distillation. A further methodological caveat is that evaluable denominators vary across conditions because image guidance requires valid, renderable graph code, so re-evaluation accuracies are computed over condition-specific subsets. An additional open question is why the graph-to-text control underperforms plain text guidance in the abstract setting despite preserving node content—suggesting that faithful textual serialization of topology is itself lossy, but the mechanism is not analyzed in depth.

Conclusion

The paper demonstrates a clear modality gap: visual graph scaffolds match textual guidance when answer-local hints are available, but retain their effectiveness under abstract constraints where textual guidance degrades sharply, and this advantage persists through both SFT and KL distillation while producing substantially shorter reasoning traces. Ablations indicate the benefit derives from preserved topology rather than visual presentation alone. The results support reframing part of the graph–LLM integration agenda away from retrieval and grounding toward graphs as compact abstractions for organizing reasoning, with vision as the modality that avoids flattening structure into text.