Knowledge Injection: Methods and Applications

• Knowledge injection is the process of integrating external factual or domain-specific information into models to update and enrich their capabilities.
• Techniques include parametric fine-tuning, non-parametric retrieval, and plug-and-play plugins, addressing both knowledge adaptation and retention.
• Empirical studies show these methods enhance factual accuracy, prevent catastrophic forgetting, and enable models to adapt to dynamic information.

Knowledge injection is the process by which external factual, procedural, or domain-specific knowledge is incorporated into machine learning models—especially LLMs, vision-LLMs, and multimodal architectures—beyond what is contained in their initial pre-trained weights. It addresses both adaptation to novel or dynamic knowledge (knowledge adaptation) and the challenge of preserving previously acquired knowledge (knowledge retention). Methods for knowledge injection span parametric updating, non-parametric augmentation, plug-and-play interfacing, and hybrid techniques, with significant technical diversity tailored to target modalities, application constraints, and efficiency considerations.

1. Core Principles and Objectives

The principal aim of knowledge injection is to enable machine learning models to internalize new or evolving factual knowledge while minimizing degradation of prior capabilities. Modern LLMs and multimodal models, such as LLaVA or CLIP derivatives, encode static knowledge during pre-training. This knowledge rapidly becomes outdated or incomplete as external information evolves. Effective knowledge injection solves two orthogonal objectives:

• Knowledge Adaptation: Incorporation of new external knowledge such that the injected model reliably responds to updated or newly acquired facts.
• Knowledge Retention: Avoidance of catastrophic forgetting, whereby the introduction of new knowledge overwrites or corrupts knowledge acquired previously.

These objectives set the baseline requirements for any knowledge injection method targeting production-level LLMs or multimodal models (Jiang et al., 22 Oct 2025).

2. Methodological Taxonomy

Techniques for knowledge injection can be broadly classified by the interface they exploit, the degree to which they alter the model’s parameters, and the structure of the injected information.

2.1 Parametric Injection

Classic approaches inject knowledge by updating model parameters through continued pre-training (CPT), supervised fine-tuning (SFT), or adapter-based modifications. For LLMs, this often involves re-training or fine-tuning on domain-specific corpora or QA pairs (Xu et al., 2023, Kang et al., 2024, Kujanpää et al., 2024). Recent advances include latent paraphrasing to maximize knowledge diversity in the hidden state space while reducing costs (Kang et al., 2024); null-space LoRA-based fine-tuning to preserve previous knowledge (Jiang et al., 22 Oct 2025); and prompt distillation for efficient transfer of new factual content with minimal forgetting (Kujanpää et al., 2024).

2.2 Non-Parametric Injection

Non-parametric techniques such as retrieval-augmented generation (RAG) or in-context knowledge editing (ICE) update knowledge at inference time without changing model parameters. Here, new facts are injected by prepending or retrieving relevant passages or facts from an external database, which are then incorporated into the model’s response generation (Abonizio et al., 8 Aug 2025, Wang et al., 31 May 2025). The DecKER framework, for instance, decouples reasoning and knowledge via masked reasoning path planning and hybrid retrieval/model-based validation, mitigating conflicts between newly injected knowledge and a model’s parametric memory (Wang et al., 31 May 2025).

2.3 Plug-and-Play and Modular Injection

Plug-and-play paradigms keep downstream models frozen and inject domain knowledge through lightweight mapping networks (plugins). Map-tuning, for example, learns a mapping from knowledge embeddings (e.g., from Wikidata or UMLS) into the input space of pretrained models, enabling cross-domain adaptation and rapid knowledge updates without any downstream model retraining (Zhang et al., 2023).

2.4 Multimodal and Domain-Specific Injection

Multimodal models face unique challenges owing to the need to coordinate knowledge injection across multiple streams (vision, text, audio). The KORE method for LMMs uses knowledge-oriented augmentations (multi-turn dialogues, instruction-style tasks) and constraints (null-space covariance projection for adapters) to enable robust updating and powerful retention in vision-LLMs (Jiang et al., 22 Oct 2025). In specialized domains (ICD coding, radiology reporting), injection frameworks must also integrate heterogeneous knowledge forms (descriptions, hierarchies, synonyms, weighted concepts, retrieved triplets) through architecture-compatible mechanisms (Zhang et al., 24 May 2025, Li et al., 2023).

3. Representative Algorithms and Mechanisms

Distinct knowledge injection methods operationalize the high-level principles through concrete algorithmic constructs and loss formulations. Selected examples:

Method / Paper	Key Injection Mechanism	Retention/Forgetting Addressed?
KORE (Jiang et al., 22 Oct 2025)	LoRA adapters in null covariance	Yes: null-space direction prevents interference
LaPael (Kang et al., 2024)	Latent-level input-dependent noise	Yes: complement with data-level paraphrasing
Map-tuning (Zhang et al., 2023)	Embedding mapping plugin, frozen model	Yes: supports rapid domain adaptation, cross-model reuse
StructTuning (Liu et al., 2024)	Structure-aware CPT+SFT, mind-map taxonomy	Yes: structural conditioning achieves data-efficient retention

These approaches frequently include explicit augmentation (dialogues, paraphrases, fine-grained synthesis), novel optimization objectives (symmetrized KLs, semantic alignment, mask regularization), and structural constraints to balance adaptation and preservation of knowledge.

4. Empirical Findings and Evaluation Protocols

Successful knowledge injection is measured across multiple axes:

• Knowledge Acquisition: Improvement in accuracy, F1, or EM on new knowledge-specific benchmarks, such as closed-book QA, few-shot NLU tasks, domain-specific ICD coding, or multi-modal captioning.
• Forgetting Quantification: Delta in control task accuracy post-injection; comparison with RAG or instruction-pretraining baselines; visualization of feature space diversity after injection (Abonizio et al., 8 Aug 2025, Jiang et al., 22 Oct 2025, Kang et al., 2024).
• Data Efficiency and Scalability: Several works demonstrate strong gains with limited data; e.g., diverse paraphrasing unlocks near-RAG performance in low-resource LLMs (Abonizio et al., 8 Aug 2025); structure-aware mind-map splitting rivals large-scale CPT with only 5% corpus used (Liu et al., 2024).
• Ablation Analyses: Isolation of augmentation granularity, model architecture, and injection-location effects; crucial for quantifying the complementarity and robustness of injected knowledge (Jiang et al., 22 Oct 2025, Kang et al., 2024, Zhang et al., 24 May 2025).

5. Design Choices and Application-Specific Constraints

Effective knowledge injection demands alignment with model architecture (encoder-decoder, Transformer, multimodal backbone), knowledge types (facts, hierarchical structures, graphical constraints), and practical trade-offs:

• Diversity of injected forms: Instruction-style, dialogue-style, synthetic QA, logical constraints, knowledge map prefixes.
• Retention methods: Null-space projection, masked loss regularization, prototype replay, or alignment objectives are widely adopted to mitigate forgetting (Jiang et al., 22 Oct 2025, Zhou et al., 11 Mar 2025).
• Efficiency: Plug-and-play and latent-perturbation techniques are preferred in resource-constrained or rapidly-changing domains (Zhang et al., 2023, Kang et al., 2024, Abonizio et al., 8 Aug 2025).
• Downstream compatibility: Reusability with parameter-efficient tuning (LoRA, adapter, BitFit), and transferability across tasks or domains (Zhang et al., 2023, Zhang et al., 24 May 2025, Zhou et al., 11 Mar 2025).
• Multimodality: Knowledge must be synchronously injected into all relevant modalities (e.g., both visual and textual branches in CLIP-based CIL frameworks) (Zhou et al., 11 Mar 2025, Jiang et al., 22 Oct 2025).

6. Limitations, Open Challenges, and Future Directions

Current techniques face several documented limitations:

• Forgetting and Overfitting: Despite advanced constraints, retention of rare knowledge (tail codes, unseen modalities) still poses challenges (Zhang et al., 24 May 2025). RAG-based methods risk greater control task degradation relative to parametric methods (Abonizio et al., 8 Aug 2025).
• Knowledge-Conflict Resolution: In-context and parametric memory often conflict, and naive injection can derail multi-hop reasoning pathways. Decoupling reasoning from injected knowledge (as in DecKER) remains nascent (Wang et al., 31 May 2025).
• Scalability and Generality: While plug-and-play and synthetic ingestion methods are promising, their coverage of structured and unstructured domains, ability to deal with hallucinated or noisy facts, and scaling to high-dimensional modalities remain open problems (Zhang et al., 2024, Zhang et al., 2023).
• Evaluation: No universal benchmarks exist for knowledge injection across task families; comparisons often rely on bespoke QA and retrieval sets (Kujanpää et al., 2024, Zhang et al., 2024).
• Extensibility: Structured knowledge (e.g., graphs, table-encoded knowledge) is rarely handled as directly or flexibly as textual knowledge.

Future work is poised to explore tighter integration of structured representations, compositional injection across domains (text, image, graph), continual learning with layered retention, modular evaluation frameworks, and unified approaches that harness retrieval, augmentation, and parameter-efficient updating with minimal trade-off across knowledge retention and adaptation.

7. Representative Impact Across Domains

Knowledge injection is broadly impactful in:

• Domain adaptation and few-shot specialization (ICD coding (Zhang et al., 24 May 2025), radiology reporting (Li et al., 2023))
• Continual and class-incremental learning (multimodal and vision-LLMs (Jiang et al., 22 Oct 2025, Zhou et al., 11 Mar 2025))
• Improvement of QA and factual accuracy in LLMs (RAG, SFT, CPT, synthetic knowledge ingestion (Kujanpää et al., 2024, Zhang et al., 2024))
• Federated learning with domain constraints (personalized models with local knowledge (Fan et al., 2022))
• Interactive and reinforcement learning agents by injecting episodic memory and knowledge graphs (Chhikara et al., 2023)
• LLM robustness and generalization especially for rare, updated, or conflicting knowledge (Zhang et al., 2024, Liu et al., 2024)

Systematic research in knowledge injection continues to be foundational for models that must operate in dynamic, heterogeneous, and knowledge-rich real-world environments.