DySK-Attn: Dynamic Sparse Knowledge Attention

• DySK-Attn is a framework that updates LLMs in real time by integrating an external, mutable knowledge graph via a dynamic sparse attention mechanism.
• It uses a coarse-to-fine grained retrieval approach to select only the most relevant facts, reducing computational cost and minimizing noise.
• Empirical evaluations demonstrate that DySK-Attn outperforms traditional methods on time-sensitive QA tasks, offering near-instant knowledge updates.

DySK-Attn is a framework for efficient, real-time knowledge updating in LLMs via dynamic sparse knowledge attention. It addresses the limitation that LLMs have static knowledge that becomes outdated, while retraining is computationally prohibitive and existing knowledge editing techniques can be slow and may introduce unforeseen side effects. The framework synergizes an LLM with a dynamic Knowledge Graph that can be updated instantaneously, and uses a sparse knowledge attention mechanism to perform a coarse-to-fine grained search over the graph, identifying and focusing on a small, highly relevant subset of facts rather than applying dense attention over the entire knowledge base (Khan et al., 10 Aug 2025).

1. Problem setting and design objective

DySK-Attn is motivated by factual staleness in LLMs. In the formulation given for the framework, the core requirement is to efficiently update LLM knowledge in real time, without costly retraining or unsafe editing, while avoiding the inefficiency and noise inherent in dense information retrieval or reasoning over very large knowledge bases. The design goal is therefore not only factual updating, but also a scalable mechanism for fusing symbolic, graph-based knowledge into neural representations (Khan et al., 10 Aug 2025).

The framework is explicitly positioned against three established strategies. The first is full retraining, described as prohibitively expensive. The second is model editing, which changes network weights directly but is characterized as slow, potentially introducing undesired side-effects, and scaling poorly for frequent or large-scale updates. The third is dense retrieval or dense reasoning over a large external store, which incurs high computational cost and can amplify irrelevant information. DySK-Attn addresses these issues by combining a dynamic external source with sparsity in the attention mechanism (Khan et al., 10 Aug 2025).

A plausible implication is that the framework treats knowledge freshness as an inference-time systems problem rather than exclusively a parametric memorization problem. In that sense, the dynamic Knowledge Graph is the mutable substrate, while the LLM remains the language-processing backbone.

2. System architecture

The architecture consists of three main components: a backbone LLM, a dynamic Knowledge Graph, and a sparse knowledge attention mechanism. The backbone LLM handles natural language understanding and generation; the dynamic Knowledge Graph is a structured, updatable database of facts represented as triplets $(h, r, t)$; and the sparse knowledge attention module selects and injects only the most relevant small subset of knowledge from the graph into the LLM (Khan et al., 10 Aug 2025).

The end-to-end pipeline is hierarchical. Given a query $Q$, the framework first performs coarse retrieval to narrow the Knowledge Graph to a candidate subgraph, then applies sparse knowledge attention to select the top-$k$ most relevant facts from that subgraph, and finally fuses the resulting knowledge vector into the LLM’s intermediate layers for guided generation (Khan et al., 10 Aug 2025). This “coarse-to-fine” organization is central to the framework’s efficiency claims: the model avoids attending over the entire graph while still allowing query-specific access to updated facts.

The dynamic Knowledge Graph is described as a graph of triplets with entities and relations embedded in a low-dimensional space using something like RotatE embeddings. The scoring function is given as

$$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$

where $\mathbf{h}$, $\mathbf{r}$, and $\mathbf{t}$ are complex embeddings and $\circ$ is the Hadamard product (Khan et al., 10 Aug 2025). The graph exposes an API allowing instant addition and removal of triplets and embeddings, with new embeddings retrained or adjusted periodically for consistency (Khan et al., 10 Aug 2025). This makes the external knowledge store operationally dynamic even when the LLM parameters are not being rewritten.

3. Retrieval and sparse knowledge attention

The retrieval stage begins with coarse-grained filtering. The query $Q$ is encoded to $\mathbf{q}_{dense} \in \mathbb{R}^{d_h}$, each entity description $Q$0 is embedded as $Q$1, and cosine similarity is used to identify the top-$Q$2 relevant entities:

$Q$3

The selected entities and their immediate neighbors form the candidate subgraph $Q$4 (Khan et al., 10 Aug 2025). This stage reduces the search space before any fact-level attention is computed.

Sparse knowledge attention then operates over the candidate facts. For $Q$5 candidate facts from $Q$6, each fact embedding is constructed by concatenating head, relation, and tail embeddings:

$Q$7

The LLM processes $Q$8 to obtain a contextual query vector $Q$9. Multi-head attention scores are computed between this query representation and the fact representations:

$k$0

where $k$1 and $k$2 are linear projections of $k$3 and $k$4 respectively (Khan et al., 10 Aug 2025).

The defining operation is hard sparsification by top-$k$5 selection. Instead of applying softmax over all $k$6 fact scores, the mechanism selects only the top-$k$7 indices:

$k$8

Softmax is then applied only on these selected entries, and the final knowledge vector is formed as a weighted sum of the corresponding values:

$k$9

(Khan et al., 10 Aug 2025). This design is intended both to reduce computational cost and to mitigate noise from irrelevant facts.

The paper also gives the following pseudocode for the sparse attention step:

$\mathbf{h}$7

This suggests that DySK-Attn treats retrieval and attention as a single coupled mechanism rather than as two fully separate modules.

4. Knowledge injection and optimization

After sparse selection, the external knowledge is injected into the LLM by fusing the selected knowledge vector into intermediate hidden states. The fusion rule is

$$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$0

where $$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$1 is the output of self-attention at layer $$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$2 and $$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$3 is a learnable gating scalar (Khan et al., 10 Aug 2025). The formulation makes the external knowledge contribution additive and explicitly gated.

Training uses a standard language modeling loss together with an auxiliary loss that supervises knowledge selection. The total objective is written as

$$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$4

where the auxiliary term is a cross-entropy between the model’s “pre-selection” attention and ground-truth relevant knowledge (Khan et al., 10 Aug 2025). In the paper’s interpretation, this auxiliary supervision improves the quality of top-$$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$5 selection.

A plausible implication is that the framework separates two learning problems: sequence modeling remains governed by the autoregressive objective, while the knowledge access path is directly regularized to become selective rather than diffuse.

5. Empirical evaluation

DySK-Attn is evaluated on time-sensitive question-answering tasks, with the main reported benchmark being TemporalWiki. The comparisons include a standard LLM with no external knowledge, standard RAG, GreaseLM, and model editing with ROME (Khan et al., 10 Aug 2025). On unseen updated knowledge, the reported results are as follows.

Model	EM	F1
Standard LLM	10.3	14.5
Standard RAG	42.1	48.9
GreaseLM	43.5	50.2
Model Editing (ROME)	48.7	55.1
DySK-Attn	59.6	66.8

The same evaluation reports $$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$6-F1 values of $$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$7 for Standard LLM, $$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$8 for Standard RAG, $$f(h, r, t) = -\|\mathbf{h} \circ \mathbf{r} - \mathbf{t}\|_1$$9 for GreaseLM, $\mathbf{h}$0 for Model Editing (ROME), and $\mathbf{h}$1 for DySK-Attn (Khan et al., 10 Aug 2025). The paper states that DySK-Attn achieves the highest factual accuracy and general QA metrics, especially for new facts not included in the original model or training.

The ablation study attributes performance to three specific factors. Removing sparse attention and using dense attention over the subgraph causes a drop of $\mathbf{h}$2 F1 points, from $\mathbf{h}$3 to $\mathbf{h}$4. Removing the dynamic KG causes a dramatic drop of $\mathbf{h}$5 F1. Removing the auxiliary loss yields a smaller but still notable F1 decrease (Khan et al., 10 Aug 2025). These ablations support the claim that real-time update capability, hard sparsification, and direct supervision of knowledge selection are all functional components rather than incidental design choices.

Efficiency is reported in terms of inference latency and update cost.

Model	Inference latency (ms/token)	Knowledge update cost
Standard RAG	35.8	<1 ms
GreaseLM	51.2	N/A (static)
Model Editing (ROME)	28.1	~5 seconds per fact edit
DySK-Attn	38.5	<1 ms

The framework is described as matching RAG in latency while far surpassing editing methods for update efficiency, with new facts incorporated almost instantly (Khan et al., 10 Aug 2025). Human evaluation also rates DySK-Attn highest for factual accuracy, with fluency and coherence equal to or better than other systems (Khan et al., 10 Aug 2025).

6. Position within the knowledge-updating literature

DySK-Attn occupies a distinct position relative to retrieval-augmented generation and model editing. Compared with standard RAG, it uses a structured dynamic Knowledge Graph rather than relying only on textual retrieval, and its sparse knowledge attention is designed to select a small, highly relevant subset of facts from a candidate subgraph rather than broadly retrieving passages (Khan et al., 10 Aug 2025). Compared with model editing approaches such as ROME, it avoids directly rewriting model parameters for each factual change, which the paper characterizes as slower and more prone to side effects under frequent updates (Khan et al., 10 Aug 2025).

The framework therefore differs from parametric editing methods that seek to alter internal knowledge stores. A later paper on unified modular editing argues that attention modules play a substantial role in factual knowledge storage and retrieval, especially in earlier layers, and proposes joint editing of MLP and attention modules for better edit success and preservation (Liu et al., 31 Oct 2025). This comparison is relevant because DySK-Attn takes the opposite operational route: instead of balancing direct updates across internal modules, it externalizes mutable knowledge into a dynamic graph and routes relevant facts into the model at inference time. This suggests two complementary paradigms for factual updating: modifying internal associative memories, or maintaining an external, instantly updateable knowledge substrate.

The limitations stated for DySK-Attn are also structurally important. The method depends on the quality of the Knowledge Graph; if the KG is incomplete or inconsistent, the responses will reflect that incompleteness or inconsistency. The optimal fusion location for external knowledge remains open, and while the framework enables some multi-hop reasoning, there may be limits to chaining long reasoning paths within the sparse attention framework (Khan et al., 10 Aug 2025). These caveats constrain the interpretation of the reported gains: the method improves factual updating under the assumption of a sufficiently maintained external graph.

Future directions listed for the framework include extending to multimodal Knowledge Graphs, exploring learned sparsity patterns beyond top-$\mathbf{h}$6, incorporating richer temporal context or user profiles, and evaluating in open-world, web-scale settings with high KG dynamism (Khan et al., 10 Aug 2025). A plausible implication is that the central contribution of DySK-Attn is less a fixed architecture than a systems pattern: a LLM coupled to a mutable symbolic store through selective, computationally bounded attention.