Multi-View Teacher Aggregation

• Multi-view teacher aggregation is a method that unifies diverse supervision signals from multiple expert teachers to enhance student model training while mitigating biases and label conflicts.
• It employs architectures such as Teacher2Task, probabilistic operators, and view-attentive fusion to reconcile heterogeneous label spaces and calibrate teacher reliabilities for improved loss landscapes.
• Applications span open-vocabulary image recognition, weakly supervised 3D object detection, and multi-label learning, demonstrating significant gains in metrics like PR-AUC and mAP while addressing scaling challenges.

Multi-view teacher aggregation is a broad and evolving set of methodologies aimed at leveraging the diversity of annotation, supervision, or model expertise available from multiple teachers (“views”) to enhance the training signal delivered to a student model. The approaches span architectures, loss landscapes, and theoretical underpinnings, but share the central goal of making supervision more robust, information-rich, and unbiased by encoding and aggregating heterogeneous sources without resorting to simplistic heuristics. Research in this area encompasses domains including vision, language, multi-label learning, and representation distillation, with frameworks ranging from explicit teacher-task multi-tasking to sophisticated attention or probabilistic aggregation strategies.

1. Problem Definition and Overview

The core challenge in multi-view teacher aggregation is the principled fusion of multiple supervision signals—potentially arising from different models, modalities, human annotators, or artificially constructed “views”—into a unified or coordinated training process. Unlike single-teacher distillation, where a single model’s outputs provide the sole target, multi-view aggregation must manage diversity, conflicts, potentially varying label spaces, and the calibration of teacher reliabilities. Key desiderata include avoidance of heuristic label collapse, robustness to teacher biases, efficient scaling with teacher count, and information-theoretic guarantees on the quality of the aggregated supervision.

2. Architectural Paradigms for Multi-View Aggregation

2.1 Explicit Task Formulations: Teacher2Task

Teacher2Task reformulates multi-teacher learning as a structured multi-task problem by introducing teacher-specific input tokens, effectively treating each teacher as a separate auxiliary supervision head. For an example with $N$ teachers, each training sample is expanded into $N+1$ subtasks: one primary (ground truth) and $N$ auxiliary (teacher-specific) tasks. Each auxiliary task uses the teacher’s identity token, its prediction, and an associated confidence score; the loss summed across all tasks ensures the model simultaneously fits the primary objective and reconstructs each teacher’s labeling tendencies and reliabilities (Nguyen et al., 2024).

2.2 Multi-View Probabilistic Operators

A probability-domain operator-theoretic approach defines aggregation via a mapping $\mathcal{A}$ from $N$ teacher predictive distributions (optionally temperature-scaled and weighted) to a single fused probability $q$. Formal requirements—convexity, positivity, monotonicity, continuity, and temperature coherence—admit multiple concrete implementations, such as the weighted arithmetic mean, geometric mean (log-linear mixture), or entropic projections (Flouro et al., 14 Jan 2026). This perspective provides generalization guarantees (variance reduction, bias attenuation, KL and log-loss bounds) and supports design choices in scenarios with heterogeneous or correlated teachers.

2.3 View-Attentive and Privileged Information Aggregation

Settings with multiple types of “privileged” signals (solutions, final answers, demonstrations) necessitate decomposable aggregation. The Adaptive-View Self-Distillation (AVSD) method constructs a normalized geometric mean consensus from $K$ teacher-view distributions, while measuring and adaptively gating view-specific residuals based on alignment and magnitude factors. This assures the target distribution for the student aligns with the consensus direction and only admits additional mass when cross-view support is strong (Nguyen et al., 20 May 2026).

2.4 Adaptive and Fusion-Based Methods

Label- or feature-level aggregation may be realized through attention weighting schemes, e.g., in TMKD, where semantic weights from a modality-specific teacher (CLIP text encoder) adaptively fuse multiple visual priors, or in SCSD, where correlation-aware weights based on label co-occurrence structures fuse view-specific logits into a fused “teacher” prediction for self-distillation to individual branches (Zhang et al., 25 Mar 2026, yan et al., 5 Apr 2026).

2.5 Single-Teacher Augmentation as View Diversification

When resources limit the availability of truly independent teachers, single-teacher view augmentation generates semantically diverse “pseudo-teachers” through architectural branching and angular-diversity regularization. Multiple linear heads, each orthogonalized and dropout-injected, produce diverse predictive “views,” which are ensembled and distilled, mathematically mimicking the ensemble loss bounds and diversity gains of true multi-teacher aggregation (Yu et al., 26 Oct 2025).

3. Mathematical Formalism and Loss Functions

Method	Aggregation object	Aggregation rule	Loss landscape
Teacher2Task	Tasks/subtasks	$N$ teacher-specific + 1 primary	Cross-entropy (primary), MSE (aux)
Prob. Operators	Probabilities	Weighted avg., geometric mean, etc.	KL, mixture log-loss
AVSD	Next-token distrib.	Geometric mean + gated residual	Reverse-KL (token-level, on-policy)
TMKD/SCSD	Features/logits	Semantic weights, codebook fusion	KL, contrastive, BCE, quantization
Angular-KD	Features/logits	Ensemble of N+1 views (pseudo and orig)	KL, CRD, cross-entropy

Aggregation moves from naive hard label voting/averaging to explicit multi-output loss decomposition, operator-agnostic soft aggregation, attention-based weighting, or learned codebook fusion. Loss functions flexibly combine supervised objectives (cross-entropy), teacher reconstruction, distributional KL, mean-squared error, contrastive regularization, and direct information-theoretic surrogates (Gaussian NLL, mutual information bounds).

4. Empirical Performance and Evidence

Multi-view teacher aggregation generally yields improvements in both robustness and generalization, especially on complex or open-set tasks and in weakly supervised or missing-data scenarios. Teacher2Task outperforms zero-shot and single-teacher baselines on open-vocabulary image/video classification with large PR-AUC margins, and the benefit scales with teacher/model count (Nguyen et al., 2024). MVAT achieves a significant boost in mAP over prior methods for weakly supervised 3D object detection, closing to 81% of the fully supervised oracle using only 2D supervision (Lahlali et al., 9 Sep 2025). AVSD demonstrates +3.1–4.8% Avg@8 advantage over strong single-view and GRPO baselines for math and code LMs, with ablations confirming that consensus-residual gating is critical (Nguyen et al., 20 May 2026). TMKD’s dual-teacher, multi-view design confers +1.19–4.49% accuracy improvements across several visual benchmarks (Zhang et al., 25 Mar 2026). SCSD’s fusion/self-distillation strategy achieves up to +8% AP over contrastive and bottleneck schemes in 50% missing-view/label regimes (yan et al., 5 Apr 2026). The operator-theoretic framework establishes that, even under non-ideal teacher independence or calibrations, multi-teacher aggregation systematically reduces ensemble variance and log-loss (Flouro et al., 14 Jan 2026).

5. Distinctions from Traditional Heuristic Aggregation

Conventional approaches—majority vote, teacher probability averaging, manual weighting, or instance-wise teacher selection—are vulnerable to aggregation errors, propagation of noise, overfitting of instance-agnostic weights, and the necessity to resolve label conflicts via arbitrary rules. Multi-view formulations such as Teacher2Task avoid opaque label collapse entirely; each teacher’s signal remains a separate optimization path. Operator-based strategies are grounded in axioms that preserve convexity, positivity, and control over combination weights and temperature, yielding theoretically justified upper bounds and risk guarantees. Recent adaptive and fusion-based approaches automate teacher selection and weighting, with learned or dynamically computed fusion reflecting instance-level view quality or semantic coherence.

6. Limitations, Extensions, and Open Directions

Key limitations include dependencies on calibrated teacher confidence (Teacher2Task), label-space alignment across teachers, and scaling in auxiliary task count or feature dimension. Some frameworks require access to dense teacher outputs (e.g., all teacher embeddings, feature maps). Operator-theoretic methods require specification of per-teacher weights and temperatures, and may not automatically exploit hierarchical or graph-structured teacher relationships. Current methods do not resolve the challenge of correlated-bias teachers; extensions with graph-structured teacher modeling, task-specific sampling, calibration heads, or zero-shot/open-set teacher integration are active research areas. Efficient continuous onboarding of new teachers and policies for optimal subset selection in regimes with large, heterogeneous teacher pools are also directions of interest (Nguyen et al., 2024, Flouro et al., 14 Jan 2026).

7. Applications and Representative Domains

Multi-view teacher aggregation is now central in weakly supervised 3D vision, open-set/noisy-label classification, self-distillation with privileged information, unsupervised representation learning, and semi-supervised learning with incomplete views/labels. Representative applications include:

• Open-vocabulary image and video recognition with LLM and domain-specific teacher fusion (Nguyen et al., 2024)
• Weakly supervised 3D object detection exploiting multi-frame LiDAR with 2D-3D aggregation (Lahlali et al., 9 Sep 2025)
• LLM fine-tuning with heterogeneous privileged view distillation (Nguyen et al., 20 May 2026)
• Visual representation transfer with multi-prior, cross-modal teachers (Zhang et al., 25 Mar 2026)
• Task-agnostic embedding distillation from diverse model architectures and modalities (Formont et al., 21 Oct 2025)
• Efficient single-teacher ensemble augmentation for resource-constrained settings (Yu et al., 26 Oct 2025)
• Multi-view, multi-label learning under missingness with shared codebooks and fused-teacher distillation (yan et al., 5 Apr 2026)

In sum, multi-view teacher aggregation underpins a new generation of robust, scalable, and theory-aligned distillation architectures, enabling state-of-the-art transfer, generalization, and knowledge fusion across a spectrum of tasks and domains.