Progressive Multimodal Training Paradigm

• Progressive multimodal training paradigms are defined as structured methods where models incrementally integrate modalities through sequential stages.
• They alleviate optimization hurdles, modality imbalances, and catastrophic forgetting by decomposing learning into modular, curriculum-based phases.
• Empirical studies show significant improvements in video understanding, domain adaptation, and continual learning through progressive integration strategies.

A progressive multimodal training paradigm refers to any structured approach wherein multimodal models incrementally acquire capabilities or representations through a curriculum of increasing complexity, modular expansion, staged alignment, or stepwise integration of modalities, data, or expertise. Such paradigms contrast with monolithic or fully joint training, which attempts to realize all objectives and process all modalities concurrently. Instead, progressive methods decompose learning into sequential or layered phases, facilitating improved optimization, robustness, modularity, and transferability. This entry surveys foundational principles, representative methodologies, mathematical formulations, empirical findings, and established implications of progressive multimodal training from recent literature.

1. Foundational Concepts and Motivations

Progressive multimodal training paradigms are driven by the recognition that naive end-to-end or joint training of multimodal models can lead to optimization hurdles, information fragmentation, catastrophic forgetting, inefficient transfer, or under-optimized submodules. Key motivations include:

• Resource Constraints: End-to-end training with full-scale multimodal architectures incurs prohibitive memory/computation, especially with long videos (Pang et al., 2021) or federated settings (Tun et al., 22 Jul 2024).
• Modality Imbalance: In joint training, more informative modalities may dominate; others can become under-optimized, resulting in poor generalization and limited robustness (Wei et al., 15 Oct 2024).
• Knowledge Transfer and Modularity: Sequentially growing, adapting, or aligning expert modules or adapters enhances continual learning and supports flexible extension to new tasks or data types (Li et al., 2023, Yu et al., 26 Oct 2024).
• Optimization Stability: Progressive exposure (e.g., curriculum, staged vocabulary activation) smooths model adaptation, stabilizes optimization, and enables reliable convergence (Tang et al., 27 Mar 2025, Yuan et al., 30 Jul 2025).

2. Canonical Methodologies

Progressive paradigms adopt several organization principles and mechanisms, which are exemplified below:

Sequential Curriculum and Staging

• Stage-wise Learning: Models are trained via a sequence of stages, each focusing on a subset of the overall objectives or data complexity:
  • Early stages build foundational capabilities (e.g., text-only reasoning (Liu et al., 3 Aug 2025), image-text alignment (Li et al., 2023)).
  • Later stages progressively incorporate more complex or multimodal elements, such as video with temporal-aware policies or compositional experts (Pang et al., 2021, Yuan et al., 30 Jul 2025).
• Curriculum Reinforcement: Progressive curriculum reinforcement learning (PCuRL) guides models from easier to harder examples, modulated by difficulty weighting and dynamic reward design (Yuan et al., 30 Jul 2025).

Modular and Compositional Expert Expansion

• Expert-based Growth: Specialized experts (e.g., for text, image, context, generation) are progressively added and composed. Earlier experts may serve as "teachers" guiding newer ones, facilitating transfer (Li et al., 2023).
• Adapter-in-Adapter Expansion: Incremental addition of modality-specific adapters with cross-modal integration (AnA framework) allows models to grow along new modal paths without joint retraining (Yu et al., 26 Oct 2024).

Progressive Data and Vocabulary Integration

• Activation of Subsets: Begin with limited data or vocabulary (e.g., only text tokens), incrementally exposing the model to visual tokens, which are activated one-by-one or in small batches (progressive vocabulary learning) (Tang et al., 27 Mar 2025).
• Pseudo-label Expansion: In domain adaptation, begin adaptation with "easy" samples (high-confidence pseudo-labels), progressively including more difficult examples using consensus or modality-specific selection (Zhang et al., 24 Jun 2025).

Iterative Alignment and Refinement

• Markov Progressive Propagation: Temporal models for long videos split sequences into fragments, enforcing Markovian, past-to-current dependencies and progressively transmit information through custom Markov convolutional operators (Pang et al., 2021).
• Iterative Evidence or Alignment Refinement: Multi-stage evidence selection or local alignment is refined over multiple iterations, integrating current and previous outcomes for precise, noise-robust mappings (e.g., PLAN for medical local alignment) (Yan et al., 25 Feb 2025, Yang et al., 2023).
• Cross-modal Distribution Alignment: Use of KL-divergence on next-token distributions to align strong (text-rich) and weak (vision-rich) outputs for multimodal reasoning, e.g., Math-PUMA's upward alignment (Zhuang et al., 16 Aug 2024).

3. Mathematical Formulations and Algorithms

Progressive training methods are typically formalized by introducing explicit update rules, loss decompositions, or operator modifications at each stage. Common elements include:

• Progressive Layer Training:

$$\text{At stage } s: \mathbf{F}^{(s)} = [L^1, \ldots, L^s]$$

All parameters up to $L^s$ updated jointly in progressive (vs. layerwise frozen) strategies (Tun et al., 22 Jul 2024).

• Progressive Markov Convolution:

$$f_\text{res} = \text{Conv}(f_\text{past}, f_\text{cur}, 0)$$

Forward pass only depends on present and past, with gradients truncated to block future context flow (Pang et al., 2021).

• KL-Divergence Alignment for Multimodal Output:

$$\mathcal{L}_\text{KL} = \lambda_\text{KL} (\alpha_\text{KL} \mathcal{L}_\text{FKL} + (1-\alpha_\text{KL}) \mathcal{L}_\text{RKL}) \cdot \tau^2 + (1 - \lambda_\text{KL}) \mathcal{L}_\text{hard}$$

With FKL/RKL measuring forward/reverse KL between text/vision output distributions (Zhuang et al., 16 Aug 2024).

• Progressive Vocabulary Activation:

As in UGen (Tang et al., 27 Mar 2025):

Initialize V_A = V_T (text), V_I (pending visual tokens) Every k steps: V_A ← V_A ∪ {next visual token} ; mask inactive tokens in loss

• Dynamic Gating and Arbitration (e.g., for in-network fusion):

$$H_\text{fused} = g \odot H_\text{text} + (1-g) \odot H_\text{cross}$$

Where $$g = \sigma(W_g [H_\text{text}; H_\text{cross}] + b_g)$$ (Wen et al., 20 Aug 2025).

• Iterative Similarity Refinement for Local Alignment:

$$\tilde{S}_t = f_\text{LA}(\delta S_t + (1-\delta) \tilde{S}_{t-1})$$

For $t$-th iteration, yielding convergence to stable (denoised) local alignments (Yan et al., 25 Feb 2025).

4. Empirical Achievements and Comparative Evaluations

Progressive multimodal training paradigms have consistently outperformed baselines across a range of tasks:

• Video and Action Understanding: PGT improves mAP by up to 3.7 on Charades and top-1 accuracy by 1.9 on Kinetics versus fragmentary baselines (Pang et al., 2021). ReasonAct's progressive pipeline lifts a 3B parameter model to 67.2%/94.1%/78.9% accuracy on HMDB51, UCF-101, Kinetics-400, representing 12–18pt gains (Liu et al., 3 Aug 2025).
• Domain Adaptation: PMC outperforms both multi-modality and single-modality domain adaptation methods by robust easy-to-hard pseudo-label expansion and consensus fusion (Zhang et al., 24 Jun 2025).
• Dialogue and Continual Learning: Progressive staged expert pretraining (PaCE) achieves SOTA in intent prediction, retrieval, state tracking, and response generation (Li et al., 2023). ModalPrompt reduces catastrophic forgetting and achieves 20–29% performance gains in continual multimodal task learning by dual-modality prompt selection (Zeng et al., 8 Oct 2024).
• Medical Local Alignment: PLAN yields superior phrase grounding (CNR), retrieval (P@1), object detection (IoU, Dice), and zero-shot classification metrics by iteratively refining word-pixel correspondences (Yan et al., 25 Feb 2025).
• Multimodal Reasoning: VL-Cogito's PCuRL advances SOTA on diverse benchmarks by staged curriculum RL, with dynamic reward shaping (Yuan et al., 30 Jul 2025). Math-PUMA's KL-based alignment reduces the text/vision gap by ≈10% on MathVerse and MathVista (Zhuang et al., 16 Aug 2024).
• Multilingual Pivoting: Progressive cross-lingual curriculum in MPM demonstrates that English-only visual pretraining suffices for high-performing non-English (Chinese) multimodal models (Hu et al., 2023).

The table below summarizes several core strategies and their reported performance/context:

Method	Core Progressive Principle	Empirical Improvement
PGT (Pang et al., 2021)	Markov stepwise propagation	+1.9% top-1, +3.7 mAP (long video)
Math-PUMA (Zhuang et al., 16 Aug 2024)	KL-alignment, staged multimodal curriculum	+10% gap reduction, SOTA on open MLLMs
UGen (Tang et al., 27 Mar 2025)	Progressive vocabulary activation	+13.3% vs vanilla AR, stable convergence
PLAN (Yan et al., 25 Feb 2025)	Iterative local alignment refinement	SOTA CNR, P@1, IoU, Dice in medical datasets
PaCE (Li et al., 2023)	Progressive expert expansion (dialog)	SOTA F1, R@k, BLEU etc. (8 benchmarks)
PMC (Zhang et al., 24 Jun 2025)	Easy-to-hard sample, MSS/MIS pseudo-label	SOTA domain adaptation with/without missing mod.
ModalPrompt (Zeng et al., 8 Oct 2024)	Prompt selection/fusion for new tasks	+20–29% continual learning gain, faster, compact
VL-Cogito (Yuan et al., 30 Jul 2025)	Progressive curriculum RL, dynamic reward	Robust SOTA on reasoning, better length/accuracy

5. Architectural and Implementation Considerations

Progressive multimodal training is realized across numerous architectures and use cases, with various practical implementation details:

• Operator Modification/Insertion: Markov convolutional operators, cross-attention blocks with dynamic fusion, LoRA+adapter PEFT integration (Pang et al., 2021, Wen et al., 20 Aug 2025).
• Curricular Scheduling: Progressive step sizes (number of layers/tokens activated, pseudo-label ratio, difficulty weighting) must be tuned for stability and performance (Tang et al., 27 Mar 2025, Zhang et al., 24 Jun 2025, Yuan et al., 30 Jul 2025).
• Parallelization and Efficiency: Many progressive strategies lend themselves to parallelization by working on submodules, supporting lower peak memory/communication for large-scale or federated learning (Tun et al., 22 Jul 2024).
• Transfer and Compositionality: Modular expert or adapter designs support future extensibility, continual learning, and faster adaptation to new tasks or datatypes without full retraining (Li et al., 2023, Yu et al., 26 Oct 2024).
• Loss Balancing: Regulating the contribution of progressive alignment (e.g., KL vs cross-entropy), gating coefficients, or curriculum weights demands empirical validation for optimal tradeoff between efficiency and generalization.

6. Implications and Future Directions

Progressive multimodal training paradigms have demonstrated their value in building robust, performant, and adaptable systems across domains. The staged or modular approach fosters more interpretable, resource-efficient, and extensible models—attributes key to real-world deployment. Established as a prevailing pattern in state-of-the-art multimodal vision, language, dialogue, medical, and reasoning systems, progressive methodologies will likely underpin further advances in autonomous agents, federated learning, cross-lingual transfer, and lifelong continual learning. Moreover, linking the progressive organization to both optimization theory and human curriculum learning continues to inspire more sophisticated schemes for curriculum design, adapter routing, and staged integration, driving the next wave of research into scalable and trustworthy multimodal intelligence.