WFM: Wiki Foundation Model for Complex Agentic Reasoning

Abstract: Real-world agents fundamentally require persistent non-parametric knowledge for dynamic reasoning, i.e., long-term memory and retrieval-augmented generation. While graphs have shown reliable advantages in providing structured evidence, the sparse graph representations naturally restrict machine readability and semantic density required for complex agentic workflows. Driven by this limitation, the entire industry is witnessing a paradigm shift from traditional sparse graphs to LLM Wiki, an agent-native knowledge representation that couples dense document contexts with markdown files containing multi-layered topological linkages. However, parameterizing such rich semantics is challenging to encode dense textual contexts using traditional sparse graph embeddings. Moreover, learning LLM Wiki with existing graph encoders could overwhelm distributed system overheads that hinder deployment in large-scale commercial scenarios. To this end, we propose a novel paradigm Wiki Foundation Model, i.e., WFM, tailored for scalable, agent-native representation and retrieval. Specifically, (i) we formalize a Wiki Graph schema that seamlessly bridges fine-grained structures with dense contexts, maintaining explicit topologies alongside continuous semantics; (ii) A query-conditioned attentive aggregation is tailored for rich wiki message passing and explicit attention variance regularization; (iii) We engineer an infrastructural NCCL boundary exchange protocol that hoists static partition indices and leverages fixed-shape GPU-to-GPU collectives, bypassing CPU serialization and memory copy overheads. Extensive evaluations across five long-term agent memory and multi-hop reasoning benchmarks demonstrate the remarkable performance of WFM, while achieving a 10.5 times training acceleration on distributed clusters.

• The paper proposes a hybrid Wiki Graph that combines entity-relation structure and dense passage semantics for enhanced agentic reasoning.
• WFM demonstrates significant improvements in retrieval recall and answer accuracy on multi-hop QA benchmarks, such as HotpotQA and 2WikiMultihopQA, and on long-horizon memory benchmarks.
• The system introduces an NCCL-native distributed communication protocol, reducing training latency by 10.5-fold and enabling scalable distributed training.
• The paper is to provide a holistic solution for efficiently integrating complex textual information into agentic reasoning, addressing limitations of both sparse and decoupled retrieval methods.
• To isolate the impact of individual components of WFM, further studies are needed.

WFM addresses a specific limitation of graph-based retrieval for agentic reasoning: sparse relational representations preserve entity connectivity but discard much of the textual context required for multi-hop inference and long-term memory retrieval. The paper proposes a unified representation-learning and systems architecture in which entity–relation structure and dense passage semantics are encoded in a shared hybrid graph, propagated by a query-conditioned graph encoder, and trained with objectives intended to prevent attention collapse. It additionally introduces an NCCL-native distributed communication protocol for reducing the cost of cross-partition message passing. The resulting system is evaluated on three multi-hop QA benchmarks and two long-horizon memory benchmarks, with reported improvements in retrieval recall, answer accuracy, and distributed training throughput (2609.18182).

Problem formulation and motivation

The paper characterizes LLM Wiki as a knowledge representation that combines structured Markdown documents, dense passages, entities, and multi-layer linkages. This representation is positioned between conventional RAG, which generally treats passages as independent retrieval units, and GraphRAG, which often compresses documents into sparse entity–relation triples. The central claim is that triple-based graphs provide useful topology but insufficient semantic density for complex agentic workflows. Compression into triples can remove discourse continuity, local qualifications, temporal information, and other textual details that are difficult to encode in a fixed relational schema.

WFM formalizes this hybrid representation as a Wiki Graph containing entity nodes, passage nodes, typed entity–entity edges, and entity–passage cross-layer links. The design preserves explicit topology while treating passages as first-class nodes rather than merely as metadata associated with entities. This distinction is important: passage nodes can participate directly in message passing, allowing textual evidence to influence entity states and allowing entity structure to guide passage retrieval.

The paper identifies two technical failure modes in applying existing GFMs to this setting. First, dense text-augmented neighborhoods can cause Softmax attention logits to converge toward similar values. The resulting near-uniform coefficients make message passing resemble repeated neighborhood averaging, weakening path selectivity and potentially producing what the paper calls gradient locks. Second, distributed graph training requires repeated synchronization of boundary-node states. Conventional CPU-mediated serialization and host-to-device copies can dominate computation, particularly when the graph is partitioned across GPUs.

Wiki Graph representation

The Wiki Graph has two node classes:

• Entity nodes represent fine-grained concepts and are initialized with trainable structural embeddings.
• Passage nodes represent dense textual contexts and are initialized with embeddings from a pretrained LLM.

Entity–entity edges retain the original typed relations. Entity–passage edges connect entities to passages in which they are described or mentioned. Passage embeddings are projected into the structural propagation dimension, producing a common space in which entity and passage states can be jointly aggregated. This projection does not eliminate the distinction between the two modalities; rather, it enables them to compete within a common query-conditioned neighborhood.

The resulting graph is optimized through three complementary objectives. A TransE-style margin loss preserves relational topology, an InfoNCE loss aligns linked entities and passages, and an attention-variance loss discourages degenerate attention distributions. The warm-start curriculum first trains the projection and lookup components using entity–passage alignment, then enables full joint optimization. This ordering is intended to establish a compatible structural–semantic space before graph propagation mixes the representations.

The architectural overview is summarized below.

Figure 1: WFM combines entity–relation topology and passage nodes in a shared Wiki Graph, optimizes structural, alignment, and attention-variance objectives, and performs GPU-resident boundary exchange across graph partitions.

The formulation is technically coherent, but its effectiveness depends on the quality of the Wiki construction process. The paper does not isolate errors introduced by entity linking, relation extraction, passage segmentation, or entity–passage association. Consequently, the reported gains establish the value of the proposed representation under the paper’s graph-construction pipeline, but do not independently demonstrate robustness to noisy or dynamically changing Wiki structures.

Query-conditioned attentive propagation

WFM uses relation-aware attention over both entity and passage neighbors. For a target node and a typed neighbor, the attention logit depends on the transformed source and target states, the relation embedding, and a learned scoring vector. Softmax normalization is performed over all typed incident messages, so structural and textual neighbors compete within the same local aggregation.

The message update combines an additive branch and a multiplicative branch. The additive branch preserves information from either the current node state or its neighborhood message, whereas the Hadamard-product branch emphasizes feature dimensions shared by the two. This Bi-Interaction update is applied uniformly to entity and passage nodes after projection into a common dimensionality.

Query conditioning occurs before propagation by identifying seed entities and passages and inducing an active local computation graph. The query therefore determines which portion of the Wiki participates in computation, while the attention mechanism determines the relative contribution of neighbors within that subgraph. This separation makes the model computationally more tractable than unrestricted propagation over the entire graph, although the paper does not report the induced-subgraph sizes or their effect on latency and recall.

The attention-variance regularizer is central to the optimization design. It imposes a hinge penalty when the variance of a node’s attention logits falls below a threshold. Unlike an entropy-minimization objective, it does not directly specify which neighbor should dominate; it only prevents all logits from becoming insufficiently distinguishable. This is a relatively weak prior and therefore leaves task losses responsible for determining the semantic ranking of neighbors. The paper’s ablations indicate that removing this term reduces both retrieval and memory performance, supporting the claim that attention selectivity is important in dense Wiki neighborhoods.

Iterative retrieval and self-reflection

WFM embeds retrieval inside a bounded self-reflection loop. At each round, the current query is matched against propagated passage representations, the top-$k$ passages are added to an accumulated evidence set, and an LLM generates an answer, a follow-up query, and a completion flag. If the evidence is judged sufficient, the loop terminates early; otherwise, the follow-up query targets missing information.

This mechanism distinguishes WFM from a single-pass retriever. The accumulated context allows later rounds to expand coverage without discarding earlier evidence, which is particularly relevant for compositional questions and long interaction histories. The maximum reflection budget is set to four rounds, while the final-answer flag provides adaptive stopping. In the reported analysis, the average number of executed rounds at this budget is 2.58, indicating that the maximum budget is not equivalent to average inference cost.

The budget sweep shows that average memory accuracy increases from 51.51 with one round to 55.33 with four rounds, but reaches only 55.57 at six rounds. This result supports the default budget of four: additional rounds provide diminishing returns under the evaluated tasks. It does not, however, establish that the same budget is optimal under different LLMs, retrieval depths, graph sizes, or interaction distributions.

Distributed training co-design

The systems contribution addresses communication rather than changing the mathematical propagation rule. WFM precomputes graph partition layouts and boundary-node ownership offline. During training, each GPU gathers the required local states into fixed-shape, GPU-resident buffers and uses NCCL collectives for direct GPU-to-GPU exchange. Received states are scattered into ghost-node slots, after which the standard attention and Bi-Interaction update is evaluated.

The reported per-step latency decreases from 2.40 seconds to 0.23 seconds, corresponding to a 10.5-fold end-to-end training acceleration. The paper describes this result as bit-exact, meaning that the distributed protocol is intended to preserve the computed node states rather than introduce an approximation to message passing. The implication is that a substantial systems bottleneck can be removed without modifying the model’s retrieval or representation semantics.

This result is contingent on static partition topology and fixed communication layouts. The protocol is well suited to the paper’s setting, where graph structure is treated as static during training, but the paper does not measure repartitioning, incremental Wiki updates, skewed boundary distributions, or heterogeneous interconnects. These conditions are material for the paper’s broader framing around persistent and dynamic agent memory.

Experimental evaluation

The evaluation covers HotpotQA, 2WikiMultihopQA, and MuSiQue for multi-hop retrieval and QA, as well as PersonaMem-1M and RHELM for long-horizon memory. Retrieval is evaluated using Recall@$k$, while answer generation is evaluated using LLM-judged accuracy. Multi-hop QA reports both Open mode, which permits parametric knowledge, and Reject mode, which requires reliance on retrieved evidence.

Multi-hop retrieval and question answering

WFM achieves the highest reported Recall@20 on HotpotQA and 2Wiki, with scores of 93.20 and 90.15, respectively. On MuSiQue, WFM reaches 75.24 at Recall@20, 0.66 points below Youtu-GraphRAG, although it leads at smaller retrieval depths. Relative to GFM-RAG, WFM improves Recall@20 by 11.82 points on HotpotQA, 17.52 points on 2Wiki, and 26.67 points on MuSiQue.

Benchmark	WFM Recall@20	Best comparison stated in paper	WFM advantage or gap
HotpotQA	93.20	Youtu-GraphRAG: 89.70	+3.50
2Wiki	90.15	Youtu-GraphRAG: 88.50	+1.65
MuSiQue	75.24	Youtu-GraphRAG: 75.90	-0.66

The end-to-end QA results are similarly strong. WFM obtains Open/Reject accuracies of 89.6/84.3 on HotpotQA, 90.2/82.4 on 2Wiki, and 69.8/52.6 on MuSiQue. Against Youtu-GraphRAG, the gains are 2.8/4.1, 3.2/4.8, and 4.1/5.1 points, respectively. The larger Reject-mode gains are important because they indicate improved evidence coverage rather than merely improved use of parametric knowledge. The strongest relative improvement occurs on MuSiQue in Reject mode, where longer reasoning chains make incomplete first-pass retrieval more consequential.

Figure 2: WFM’s cross-benchmark effectiveness on Recall@20 for multi-hop QA and overall accuracy for long-horizon memory QA.

The results support the paper’s claim that dense passage nodes and iterative retrieval complement explicit graph topology. They do not, however, establish that all of the gain comes from the Wiki representation itself: WFM simultaneously introduces hybrid graph construction, attentive propagation, variance regularization, warm-start training, and self-reflection. The ablation study partially separates these factors, but the principal comparison remains against complete baseline systems with different architectural and optimization choices.

Long-horizon memory QA

On PersonaMem-1M, WFM reaches 58.49 overall accuracy; on RHELM, it reaches 52.17. These scores exceed the strongest non-WFM baseline by 8.39 and 5.37 points, respectively. Removing self-reflection reduces overall accuracy to 54.12 on PersonaMem and 48.90 on RHELM, but the resulting system still exceeds the strongest baseline by 4.02 and 2.10 points.

WFM also obtains the strongest reported Recall@20 on both memory benchmarks: 52.63 on PersonaMem and 60.03 on RHELM. Compared with A-mem, the strongest memory-specific baseline in the paper, these are improvements of 14.43 and 6.13 points. On PersonaMem, WFM is slightly below A-mem at Recall@5, but its advantage expands to 7.19 points at Recall@10 and 14.43 points at Recall@20. This pattern is consistent with broader evidence coverage as the retrieval budget increases rather than uniformly superior top-ranked retrieval.

Figure 3: Component ablations show the effect of Wiki representation, attentive aggregation, optimization stabilization, self-reflection, and adaptive stopping on effectiveness and normalized cost.

The memory results are particularly relevant to the paper’s motivation because they test contexts extending to one million tokens and include temporal, aggregation, mixed-context, misleading, and hallucination-oriented questions. Nevertheless, PersonaMem uses an LLM judge for recall because turn-level evidence labels are unavailable, whereas RHELM uses exact annotated turn-level recall. Comparisons across these datasets should therefore be interpreted separately, as the paper itself notes.

Ablations and parameter sensitivity

The ablations support the paper’s claim that the components are complementary. Replacing the Wiki Graph with an ordinary entity graph reduces average multi-hop Recall@20 by 7.48 points and memory accuracy by 7.68 points. This is the clearest evidence that passage nodes and cross-layer links contribute beyond sparse entity–relation structure.

Replacing WFM’s attentive aggregator with a capped DistMult variant reduces multi-hop recall by 11.89 points and memory accuracy by 12.47 points while increasing wall-clock cost by $2.65\times$. The uncapped variant runs out of memory on the 1M setting. This comparison is informative but not fully symmetric: the DistMult implementation requires neighborhood capping, so part of the difference may reflect constrained receptive fields or implementation-specific scaling rather than only the aggregation function.

Removing variance regularization and removing warm-start training both degrade performance, indicating that attention stabilization and initialization are not redundant. Three attentive layers provide the best reported balance, with average Recall@20 of 86.20 and memory accuracy of 55.33. Deeper stacks progressively reduce both metrics, consistent with oversmoothing or repeated neighborhood mixing in dense graphs.

The parameter analysis is reproduced conceptually below.

Figure 4: Sensitivity to attentive depth, self-reflection budget, executed rounds, and the variance-regularization parameters $\epsilon$ and $\lambda_2$.

Adaptive stopping also has a measurable systems effect. Removing the final-answer flag and forcing all four rounds raises normalized cost to $1.58\times$ while recovering most of the effectiveness. Thus, the stopping mechanism is not presented as a major source of retrieval quality by itself; its principal value is reducing unnecessary inference rounds while retaining most of the benefit of iterative retrieval.

Limitations and open questions

Several limitations follow directly from the reported methodology and experimental design. First, the implementation section states that newly completed cells and diagnostic sweeps are “planning values” requiring validation with measured runs before external use. This qualification applies to parts of the presented analysis and weakens the evidentiary status of those results until independently reproduced.

Second, the paper evaluates WFM using fixed corpora and largely static graph structures. Although the motivation emphasizes dynamic long-term memory, the experiments do not quantify update latency, consistency under graph modification, incremental embedding maintenance, or communication overhead during repartitioning. The NCCL protocol’s strongest claims therefore apply to static partition layouts.

Third, the paper does not provide a complete accounting of indexing, Wiki construction, passage encoding, LLM generation, judging, storage, or inference costs. The 10.5-fold acceleration concerns distributed training step latency, not end-to-end system cost. Similarly, retrieval improvements are measured under a common embedding model and retrieval depth, but sensitivity to different encoders and LLM backbones remains open.

Fourth, the attention-variance mechanism prevents low logit variance but does not guarantee semantically correct selectivity. A neighborhood can have dispersed logits while assigning high weight to irrelevant nodes. The reported ablations establish empirical utility, not that the variance lower bound is sufficient to prevent all forms of representation collapse.

Finally, the comparisons do not fully disentangle WFM’s representation, propagation, curriculum, self-reflection, and systems contributions. The paper leaves open whether a simpler dense passage graph with an equivalent iterative controller would achieve similar accuracy, and whether the observed gains persist when all baselines receive comparable reflection budgets, graph-construction resources, and distributed implementations.

Conclusion

WFM presents a unified approach to agentic retrieval that combines dense textual passages with explicit graph topology in a trainable Wiki Graph. Its principal methodological contributions are hybrid entity–passage representation, query-conditioned relation-aware propagation, attention-variance regularization with warm-start optimization, and NCCL-native boundary exchange. Across the reported benchmarks, WFM achieves strong multi-hop retrieval and QA results, improves long-horizon memory accuracy, and reduces distributed training step latency from 2.40 seconds to 0.23 seconds.

The paper’s strongest empirical evidence concerns the value of passage nodes, iterative retrieval, and GPU-resident communication. Its principal unresolved questions concern dynamic graph updates, full end-to-end cost, robustness to Wiki-construction noise, and validation of the planning-value analyses. Within the evaluated static settings, however, WFM provides a technically integrated alternative to sparse GraphRAG and decoupled memory retrieval pipelines (2609.18182).