LMs as Task-Specific Knowledge Bases: An Interpretability Analysis

Abstract: LLMs (LMs) capture large amounts of factual knowledge applicable to a wide range of tasks, motivating the view of their parameters as a knowledge base. An important property of knowledge bases is that different queries for the same fact return consistent results, drawing on a single source of truth. We investigate whether LMs satisfy this property through behavioral and mechanistic analyses. Our results suggest that they encode knowledge in a task-specific manner. Behaviorally, facts acquired on one task frequently fail to co-emerge on others during training. Parameter localization experiments suggest a mechanistic explanation, revealing distinct parameter subsets underlying different tasks for the same fact. Finally, we show that chain-of-thought reasoning draws part of its effectiveness from engaging task-specific parameters beyond those tied to the evaluation task. Our findings suggest that what the model knows and how it is asked are intertwined in parameter space, undermining the "knowledge base" analogy and carrying implications for the reliability and controllability of factual knowledge in LMs.

• The paper reveals that factual knowledge in LMs emerges in a task-specific, format-dependent manner, contradicting the central KB analogy.
• The study employs behavioral analyses and parameter localization techniques to quantify significant task-based differences in knowledge encoding.
• Empirical findings suggest that model interventions for knowledge editing must address distinct, non-transferable task-specific encodings.

Task-Specific Knowledge Encoding in LLMs: An Interpretability Analysis

Introduction and Theoretical Motivation

The paper "LMs as Task-Specific Knowledge Bases: An Interpretability Analysis" (2606.27237) scrutinizes a central analogy in the interpretability of LMs: viewing the parameters as a knowledge base (KB). A robust KB should return consistent answers for any equivalent query, regardless of the query's concrete realization (e.g., cloze, multiple-choice, open-ended), since it stores facts at a central, task-invariant locus. This work provides systematic behavioral and mechanistic evidence that this analogy is not valid for current LMs: factual knowledge is parametrically and functionally entangled with the task (or probe format), such that a fact acquired/represented for one task generalizes only partially, if at all, to other tasks.

This claim is formalized and tested both at the level of model outputs during training and via parameter-localization techniques that dissociate the encoding loci of facts given different downstream tasks. The study spans multiple models and a battery of (subject, relation, object) knowledge probes.

Figure 1: Schematic comparison between a traditional KB (top), where queries all consult a central fact repository, and LLM behavior (bottom), where each task accesses an independent parameter locus, enabling divergent answers to semantically identical queries in different formats.

Behavioral Evidence: Failures of Task-Invariant Knowledge Emergence

A behavioral co-emergence hypothesis is articulated: if knowledge is stored in a task-invariant fashion, a fact that emerges in one task (i.e., is answered reliably by the model at some training checkpoint) should co-emerge in all other tasks for which the model is competent. Formally, the emergence step $E(f,t)$ (first reliable retrieval of fact $f$ on task $t$) should satisfy

$$E(f, t) \leq \max\left(E(f, \bar{t}),\, E(\cdot, t)\right)$$

for all $f$ and $t$, where $E(f,\bar{t})$ is the emergence step on any other task, and $E(\cdot, t)$ is the step at which the task overall achieves competence.

The authors collect performance traces across 105 training checkpoints of OLMo-3-7B IT, tracking 230 facts across 6 probing tasks (ranging from open QA to MCQA to verification). They find frequent and robust failures of co-emergence: 47.9% of (fact, task) pairs do not satisfy the co-emergence criterion—i.e., the fact does not emerge in the target task by the time it is reliably available in the source task and the target task is generally competent.

Figure 2: Consistent co-emergence (top) and inconsistent (bottom): a fact emerges in one task and then, despite the target task being competent, does not emerge there as expected under task-invariant knowledge storage.

A two-way ANOVA formalization of the task-invariant hypothesis finds that the fact-task interaction term is statistically highly significant at every checkpoint, and this interaction effect increases throughout training, contributing 23% of total variance in the final model. These results robustly falsify the idea that LMs store factual knowledge in a task-invariant way.

Mechanistic Interpretability: Parameter Localization of (Fact, Task) Encodings

The mechanistic analysis leverages mask-based localization methods to directly search for sparse subnets of the model (attention heads and MLP neurons) that are both necessary and sufficient for answering a (fact, task) pair, and that are as specific as possible to that pair. Masks are learned for each (fact, task), minimizing a loss that combines necessity (performance drop when ablated on target), sufficiency (performance recovery on corrupted input when only these activations are inserted), specificity (minimal impact on other tasks/facts), and sparsity.

Figure 3: Illustration of the parameter localization protocol for (France, capital-of, Paris) in the OpenQA format—localized parameters must be necessary, sufficient, and specific for this (fact, task) pair.

The results demonstrate that for all models and relations considered, disjoint and task-specific parameter subsets are found:

• Same-task ablation induces large performance drops on the targeted (fact, task) (29–89% relative decrease), with off-target tasks/facts essentially unaffected (≤8% deviation).
• Patch-based sufficiency tests show high task specificity: patching the localized components into a corrupted prompt recovers 69–102% of lost accuracy for the targeted (fact, task), but has negligible effect for others.

Figure 4: Necessity heatmap for (country, official language, language) on OLMo-2-7B IT. The large diagonal drops confirm the localized necessity of the subnetworks for (fact, task) pairs; minimal off-diagonal change demonstrates specificity.

This necessitates the conclusion that the loci of factual storage are format-dependent, contravening the KB analogy.

Quantifying Task Entanglement

The degree of overlap (entanglement) between task-specific (fact, task) encodings varies by task form:

• Discrimination tasks (e.g., MCQA, Verification) exhibit higher average entanglement (mean $\operatorname{Ent}_{\text{task}} = 0.21$), i.e., their fact encodings interfere more with other tasks, compared to generation tasks (mean $= 0.11$).
• Directional analysis of behavioral co-emergence shows that facts learned first in generation tasks frequently co-emerge in discrimination tasks (40–90%), but not vice versa (3–42%).
• The parametric and behavioral entanglement metrics are congruent.

Figure 5: Directional co-emergence rates across task pairs; higher rates along source-generation to target-discrimination pairs, and lower in the reverse direction, highlight the asymmetry and task dependence of knowledge acquisition.

Chain-of-Thought Reasoning: Bridging Task-Specific Encodings

Chain-of-thought (CoT) reasoning is shown to depend, in part, on parameter loci outside the canonical (fact, task) encoding for the evaluation format. Zero-ablation experiments demonstrate that CoT answers can often recover the performance lost when the task-specific subnet is ablated, by routing through alternative (task, fact) encodings; further, CoT is more vulnerable than direct answering to cross-task ablations, indicating wider reliance on multiple task-specific knowledge subnetworks.

Figure 6: Same-task ablation—CoT accuracy drops significantly less than direct answering when a task-specific subnet is ablated, illustrating CoT's broader access to knowledge encodings.

Implications and Future Directions

This work has several direct implications:

• Reliability and Control: Knowledge editing or unlearning methods targeted to a single task format will often not generalize to other query forms for the same fact, undermining confident system control and making it easy for "residual" knowledge to persist.
• Evaluation Protocols: Single-format factual evaluations provide only a lower bound on model knowledge; comprehensive evaluation must span multiple tasks to probe the full factual parametric store.
• Model Development: Achieving more trustworthy and updatable LM knowledge will likely require designing training curricula or architectural interventions that explicitly regularize toward task-invariant or fact-centralized encodings.
• Theoretical Insights: Models organize knowledge not only by content (fact, relation) but also by the input-output interface (task configuration), with functional and parametric redundancy and specialization mirroring cognitive phenomena such as encoding specificity.

Given the observed trends, increasing model scale and capacity may exacerbate task-dependent separation, as overparameterized models further specialize subnetworks. Thus, future models may require explicit architectural or objective-based incentives (e.g., auxiliary invariance losses, format-mixing during training, or reasoning-prompted training) to realize the KB ideal.

Conclusion

The analogy of LMs as task-invariant knowledge bases is empirically and mechanistically untenable for current models. Factual knowledge is intricately tied to both "what" (content) and "how" (task format) models are queried. This format dependence has operational consequences for model reliability, editability, and evaluation. Mechanistically, distinct loci of storage for each (fact, task) emerge, with variable entanglement shaped by the nature of the probing format. Chain-of-thought protocols leverage this structure to bridge otherwise isolated knowledge encodings. These findings direct the interpretability field toward more granular, task-sensitive analyses of parametric knowledge, and suggest a new research agenda for inducing format-agnostic knowledge storage.