Horizontal Continual NLP

• Horizontal continual NLP is the process of incrementally adapting a single model across sequential tasks or domains while preventing catastrophic forgetting.
• Methodologies such as regularization, replay buffers, parameter isolation, and meta-learning balance forward/backward transfer with efficiency.
• Empirical results show that modular dynamic architectures and adaptive memory buffers achieve near-zero forgetting and robust performance on multi-domain tasks.

Horizontal continual NLP refers to the incremental adaptation of a single model across a stream of tasks, domains, or data distributions, usually of similar type or shape, with the key constraint that the model must retain its competence on all previously seen distributions without revisiting their data. This framework is motivated by the need for NLP systems that can flexibly accumulate skills, knowledge, or representations as new domains or tasks are encountered, while minimizing catastrophic forgetting. Research in this area encompasses both supervised and self-supervised setups, and spans a broad range of task types including text classification, language modeling, topic modeling, and sequence generation. Methodological advances include regularization techniques, replay buffers, parameter-isolation architectures, dynamic expansion modules, and meta-learning strategies, each exhibiting distinct trade-offs between knowledge retention, forward/backward transfer, efficiency, and scalability.

1. Definition and Scope of Horizontal Continual NLP

Horizontal continual learning in NLP is defined as the sequential, non-repetitive adaptation of a model to a set of datasets or tasks, $\{D_1, D_2, ..., D_T\}$, each arriving in order and, in the canonical case, sharing the same input-output structure (e.g., sentiment analysis across multiple product domains, or masked LLM adaptation across temporal slices of text). At each learning stage $t$, the model observes only the current dataset $D_t$ (or a small buffer of past examples, if allowed), trains to minimize a suitable loss (e.g., supervised, generative or self-supervised), and then discards access to prior full datasets except as allowed by buffer or parameter-isolation policy (Ke et al., 2022, Shi et al., 2024, Ke et al., 2022). The key desiderata are:

• No catastrophic forgetting: Performance on all previously seen tasks/domains is preserved.
• Forward/backward knowledge transfer: Model improvements on future tasks should benefit from, and sometimes contribute back to, earlier learned knowledge.
• Strict/limited data access: Once a domain/task is finished, its full data is (often) no longer accessible, matching realistic deployment and privacy scenarios.

Horizontal continual learning is distinguished from vertical continual learning, which moves from general pretraining to increasingly specific adaptation layers (e.g., from web-scale data to task-specific fine-tuning), and from class-incremental learning, which emphasizes discovery and discrimination of new label classes over time (Shi et al., 2024).

2. Methodological Taxonomy

Approaches to horizontal continual NLP are grouped into four main families, often augmented by hybridization strategies:

• Regularization-based: Attach importance penalties (e.g., Elastic Weight Consolidation (EWC), L2, Synaptic Intelligence) to parameters learned from previous domains, to discourage updates on those critical for past distributions (Ke et al., 2022, Mi et al., 2020, Biesialska et al., 2020, Ke et al., 2022). Mathematically, for EWC:

$$L_{\text{total}}(\theta) = L_t(f(x;\theta),y) + \frac{\lambda}{2} \sum_{i} F_i (\theta_i - \theta_i^*)^2$$

where $F_i$ is the Fisher information from previous tasks.

• Replay-based: Maintain a buffer $\mathcal M$ of exemplars from previous domains, replayed alongside new data to reinforce old knowledge (Experience Replay, generative or prototype replay) (Mi et al., 2020, Biesialska et al., 2020, Ke et al., 2022, Diera et al., 2024). Selection can be prioritized based on loss, diversity, or importance to minimize storage while maximizing utility.
• Parameter-isolation: Allocate domain- or task-specific modules, gates, or masks within a shared backbone (e.g., CL-plugins in CPT (Ke et al., 2022), adapters in HOP (Michieli et al., 2024), capsule routing, multi-domain mixture-of-experts, dynamic expansion with LoRA or prompts (Shi et al., 2024, Su et al., 2020)). Knowledge isolation is achieved by freezing, masking, or gating critical parameters per task:

$m_l^{(t)} = \sigma(e_l^{(t)}/\tau)$

and block gradients accordingly.

• Meta-learning: Frame continual adaptation as an inner/outer loop, learning initialization or adaptation strategies that are robust across a sequence of tasks (MAML, OML, MeLL, meta-MBP) (Wang et al., 2020, Biesialska et al., 2020, Ke et al., 2022). Meta-objective formulations optimize for fast adaptation to each new task while preserving previous performance.
• Hybrid/Other: Integrate curriculum/data-selection, knowledge distillation, lattice routing, dynamic re-weighting, or instruction-based continual learning (Yin et al., 2022).

3. Principal Instantiations and Empirical Results

Several representative frameworks operationalize these methodological principles:

Framework	Model Family/Principle	Key Characteristics	Forgetting Control
CPT (Ke et al., 2022)	Parameter isolation (plugin adapters)	Frozen backbone, per-domain CL-plugins, masks	Gradient/path blocking, hard masks
HOP (Michieli et al., 2024)	Adapters + high-order moments	Frozen backbone, domain adapters, $p$-moment pooling, per-task head	Parameter isolation
ARPER (Mi et al., 2020)	Replay + adaptive EWC	Prioritized buffer by utility/diversity, adaptive regularization	Buffer + Fisher penalty
ERNIE 2.0 (Sun et al., 2019)	Multi-task continual pre-training	Task-specific heads, per-task curriculum, perpetual interleaved sampling	Balanced sampling, head isolation
DKVB (Diera et al., 2024)	Discrete bottleneck + frozen encoder	Frozen encoder/keybook, only value tables updated	Local plasticity (bottleneck)
ProgBERTQA (Su et al., 2020)	Dynamic-architecture	Per-domain adapters within BERTQA, frozen backbone	Adapter isolation

Empirical results show that parameter-isolated or dynamic-expansion approaches (adapters, CL-plugins, mixture-of-experts) typically achieve near-zero forgetting on benchmarks, while flexible replay with adaptive buffer management closely matches them under storage constraints (Ke et al., 2022, Michieli et al., 2024, Mi et al., 2020). In domain- or task-incremental NLU, meta-learned adaptation strategies substantially mitigate forgetting versus standard fine-tuning or static regularization (Wang et al., 2020). In scenario-specific settings (e.g., continual model refinement, streaming OOD), mixed replay+regularization approaches are optimal (Lin et al., 2022).

4. Evaluation Protocols, Metrics, and Benchmarks

Protocols emphasize sequential, single-pass/epoch constraint (no revisiting old data unless using a small buffer), and evaluation after each domain's completion:

• Metrics:
  • Average accuracy or Macro-F1 across all previously seen domains/tasks.
  • Forgetting: $$\mathrm{F}_T = \frac{1}{T-1}\sum_{i=1}^{T-1} [\max_{k}(a_{k,i}) - a_{T,i}]$$.
  • Backward transfer: improvement or degradation in past domains/tasks.
  • Forward transfer: zero- or few-shot performance on new domains given prior learning.
  • Task-specific metrics: BLEU, Slot Error Rate (NLG (Mi et al., 2020)), retrieval precision (topic modeling (Gupta et al., 2020)).
• Benchmarks:
  • GLUE/SuperGLUE: split as sequential single-task or domain streams (Biesialska et al., 2020, Wang et al., 2020).
  • Multi-domain streams: sentiment (Amazon/Yelp), topic classification (20News, AGNews), dialogue (MultiWoZ), MRC (SQuAD, NewsQA, DuoRC, NarrativeQA, etc.) (Gupta et al., 2020, Su et al., 2020, Mi et al., 2020).
  • Continual model refinement: OOD error streams with dynamic cluster sampling (Lin et al., 2022).
  • Temporal/domain drift: synthetic Wikipedia/News/Reddit streams (Shi et al., 2024).

Empirical findings consistently show (1) regularization alone rarely eliminates forgetting for long streams or abrupt domain shifts, (2) replay-based methods retain knowledge best at modest buffer sizes, though performance saturates without buffer diversity selection, and (3) parameter-isolation or modular expansion models excel in near-zero-forgetting settings but introduce parameter growth and architectural complexity (Ke et al., 2022, Michieli et al., 2024, Mi et al., 2020, Biesialska et al., 2020).

5. Challenges, Theoretical Insights, and Open Problems

Major challenges include:

• Catastrophic forgetting versus plasticity: Fixed-capacity models inherently trade memory for adaptability; modular or sparse expansion mitigates this but is subject to parameter blowup (Ke et al., 2022, Biesialska et al., 2020).
• Domain/task similarity and negative transfer: Transfer is non-trivial when domains differ in granularity, label semantics, or distribution. Methods for adaptive transfer (contrastive learning, prompt/capsule routing, meta-initialization) have improved selectivity but remain imperfect (Ke et al., 2022, Biesialska et al., 2020).
• Online, boundary-free, or OOD adaptation: Fully online streams without explicit domain boundaries (e.g., continual model refinement, temporal news) require methods robust to hidden shifts, typically hybrid replay+regularization (Lin et al., 2022, Shi et al., 2024).
• Scalability and efficiency: Replay buffers must balance coverage against size; parameter-isolation approaches require dynamic expansion or slimming; computation and latency constraints in practical deployments are largely unaddressed (Diera et al., 2024).
• Benchmarks and unified protocols: Existing benchmarks are diverse and non-standardized; there is a call for tasks simulating both horizontal domain shift and class emergence, with unified multi-metric dashboards (Biesialska et al., 2020, Ke et al., 2022).
• Theory: Few generalization or forgetting bounds are tailored to large pre-trained, horizontal-CL regimes; e.g., theory for optimal OOD/task-id detection, or guarantee of modular transfer, is underdeveloped (Ke et al., 2022, Shi et al., 2024).

6. Extensions, Innovations, and Future Directions

Ongoing research directions for horizontal continual NLP include:

• Modular dynamic architectures: Lifelong mixtures-of-experts, functional expansions, prompt- or adapter-tuning with dynamic allocation per domain and on-the-fly adaptation (Su et al., 2020, Diera et al., 2024, Shi et al., 2024).
• Efficient memory and buffer management: Smart replay—prioritizing causally salient, high-diversity, or OOD-prone examples, possibly using clustering or task-similarity selection (Mi et al., 2020, Huang et al., 2021).
• Unlabeled and instruction-based learning: Approaches such as ConTinTin demonstrate that even with only textual instructions (no prior labels or data), forward and backward transfer can be realized by careful instruction replay and negative sampling (Yin et al., 2022).
• Temporal and streaming adaptation: Protocols and methods for non-stationary, incrementally drifting text streams (e.g., news, social media, Reddit) with potentially hidden or seamless domain boundaries (Lin et al., 2022, Shi et al., 2024).
• Theoretical frameworks and unified benchmarks: Toward generalization bounds, OOD detection theory, and standardized CL stream benchmarks that combine domain, task, and class-incremental axes (Ke et al., 2022, Biesialska et al., 2020).

In sum, horizontal continual NLP is a rapidly evolving field integrating principles from adaptive neural architectures, self-supervision, lifelong memory management, and meta-learning. Its objectives—maintaining memory and facilitating cumulative adaptation—are foundational for NLP systems deployed in the wild, facing the full variability and dynamics of natural language use. Leading approaches differ in their efficiency, modularity, and transfer-friendliness, but converge on the necessity to balance plasticity and stability in the face of ever-moving targets.