Plug-and-Play Logit Fusion for Heterogeneous Pathology Foundation Models

Abstract: Pathology foundation models (FMs) have become central to computational histopathology, offering strong transfer performance across a wide range of diagnostic and prognostic tasks. The rapid proliferation of pathology foundation models creates a model-selection bottleneck: no single model is uniformly best, yet exhaustively adapting and validating many candidates for each downstream endpoint is prohibitively expensive. We address this challenge with a lightweight and novel model fusion strategy, LogitProd, which treats independently trained FM-based predictors as fixed experts and learns sample-adaptive fusion weights over their slide-level outputs. The fusion operates purely on logits, requiring no encoder retraining and no feature-space alignment across heterogeneous backbones. We further provide a theoretical analysis showing that the optimal weighted product fusion is guaranteed to perform at least as well as the best individual expert under the training objective. We systematically evaluate LogitProd on \textbf{22} benchmarks spanning WSI-level classification, tile-level classification, gene mutation prediction, and discrete-time survival modeling. LogitProd ranks first on 20/22 tasks and improves the average performance across all tasks by ~3% over the strongest single expert. LogitProd enables practitioners to upgrade heterogeneous FM-based pipelines in a plug-and-play manner, achieving multi-expert gains with $\sim$12$\times$ lower training cost than feature-fusion alternatives.

• The paper introduces LogitProd, a plug-and-play, logit-level fusion framework that integrates heterogeneous pathology foundation models without feature re-encoding.
• It employs a lightweight, sample-adaptive gating network using logit-derived cues to combine predictions via a product-of-experts, resulting in superior performance across 22 benchmarks.
• Empirical results show LogitProd outperforms the best individual model by ~3% on average, with significant efficiency gains and theoretical guarantees ensuring performance does not fall below the top expert.

LogitProd: Plug-and-Play Logit Fusion for Heterogeneous Pathology Foundation Models

Motivation and Challenges in Pathology FM Integration

The advent of pathology foundation models (FMs) has led to significant advances in computational histopathology, evidenced by improved transfer performance across diagnostic and prognostic tasks. However, the exponential growth of FM architectures—often trained with disparate pretraining datasets, augmentation protocols, and objectives—has engendered a heterogeneous model selection dilemma. Large-scale benchmarking studies demonstrate that no single FM consistently dominates across tasks, with variance arising even within a single endpoint [neidlinger2025benchmarking, ma2025pathbench]. This heterogeneity complicates deployment: exhaustively adapting and validating multiple candidate FMs is computationally intensive, while suboptimal single-model selection sacrifices potential predictive gains.

Existing integration strategies—distillation at pretraining or feature-space fusion and alignment—address this bottleneck but at the expense of substantial computational and storage overheads, especially when working with gigapixel WSIs. Patch- or slide-level feature concatenation and retraining require re-encoding voluminous data with multiple encoders, complicating upgrades and retuning pipelines when new expert models are introduced. Standard prediction-level fusion methods, such as probability averaging or majority voting, are insensitive to expert reliability and inter-expert heterogeneity, lacking adaptive sample-specific weighting.

LogitProd Methodology

LogitProd introduces a logit-level, plug-and-play fusion framework, leveraging independently trained FM-based predictors as frozen slide-level experts. The method operates strictly at the prediction level, avoiding encoder retraining and feature alignment, and is characterized by the following core components:

• Logit-derived Cues: LogitProd extracts per-expert confidence (maximum predicted probability, top-2 probability margin, entropy) and inter-expert disagreement statistics from temperature-calibrated logits. These cues facilitate adaptive gating responsive to sample complexity and predictor reliability.
• Sample-Adaptive Gating Network: A lightweight MLP predicts nonnegative expert weights for each input, using concatenated logit-derived features. The network architecture is minimal, with two layers and 64 hidden units, and outputs weights constrained to the simplex for classification or per-time-bin survival prediction.
• Product-of-Experts Fusion: Predictions are combined via a weighted, normalized product-of-experts (PoE) formulation, enhancing consensus when experts agree and suppressing overconfident errors under correlated failure. This multiplicative aggregation is theoretically motivated, with proof that the optimal fusion achieves cross-entropy risk no worse than the best single expert.

  Figure 1: LogitProd framework. Frozen FM experts output logits; reliability cues drive sample-adaptive weights for efficient product fusion—enabling streamlined multi-task prediction without patch re-encoding or feature-space alignment.

Theoretical Justification

The theoretical analysis ensures that, within the logit-only product-federated model family, there exists a choice of fusion weights (one-hot included) such that the training objective (cross-entropy or log-likelihood) is at least as good as the best individual expert. The proposition leverages the geometric mean's upper-bound properties and demonstrates that learned sample-adaptive weights can further exploit instance-level variance, potentially surpassing simple one-hot gating. This analysis applies to discrete-time survival tasks via decomposition into independent bin-wise cross-entropy losses, generalizing LogitProd's applicability.

Empirical Evaluation and Results

LogitProd was systematically evaluated across 22 benchmarks encompassing WSI-level and tile-level classification, gene mutation prediction, and discrete-time survival modeling. The expert pool consisted of nine state-of-the-art FMs, and downstream predictors were independently trained using standard weak supervision and multiple instance learning (MIL) paradigms. Performance metrics included AUC, ACC, F1, and C-index; efficiency was quantified using EffScore, integrating parameters, FLOPs, and training time.

LogitProd ranked first in 20 of 22 tasks, outperforming the strongest single FM-based expert by an average of ~3%. Notable numerical results included a gain of 5.3% mAUC in ARID1A mutation prediction and improvements in CRC-MSI tile classification (AUC 0.8288 vs. 0.8150). Survival analysis also saw LogitProd consistently at or near the top across six TCGA cohorts, with BRCA AUC reaching 0.7338 compared to 0.6975 for the next strongest model.

Figure 2: Performance on 22 pathology tasks. LogitProd achieves superior mAUC in mutation prediction, highest AUCs in tile/WSI classification, and top C-index distributions in survival analysis. Task-wide rank heatmap shows LogitProd consistently outperforms individual FM-based experts.

Efficiency analysis highlighted LogitProd’s resource advantages: it realized multi-expert gains with a $\sim$12$\times$ reduction in training time compared to patch-level fusion baselines and used fewer trainable parameters (0.77M vs. up to 12.3M). Performance-efficiency trade-off was optimal as indicated by the EffScore metric.

Ablation studies confirmed that LogitProd's improvements are not attributable to generic ensembling; the largest performance drop occurred when sample-adaptive weighting or logit-derived features were disabled, affirming that instance-wise reweighting is critical.

Practical and Theoretical Implications

LogitProd enables practitioners to integrate heterogeneous pathology FMs in a plug-and-play manner, obviating the need for feature extraction, alignment, or head retraining. Its theoretical guarantee ensures that performance cannot degrade below the best expert, facilitating robust deployment in scenarios where expert reliabilities are task- and sample-dependent. The fusion operates on frozen experts, making pipeline upgrades efficient and supporting incremental expert pool expansion via refitting only the gating network on logits.

From a theoretical perspective, LogitProd demonstrates that reliability cues in prediction logits suffice for adaptive fusion, without requiring access to high-dimensional embeddings. Multiplicative product-of-experts aggregation sharpens consensus and mitigates correlated errors.

Future Directions

Extensions of LogitProd should focus on federating experts with differing label spaces, objectives, or modalities. As fusion currently assumes homogeneous endpoint predictors, broader generality is attainable by incorporating multimodal foundation models. Online expert selection under distribution shift and uncertainty-aware gating offer promising avenues for robust deployment in evolving clinical environments.

Conclusion

LogitProd presents a rigorous, efficient fusion framework for heterogeneous pathology FMs, achieving state-of-the-art accuracy and resource utilization across a wide range of tasks. By leveraging logit-derived reliability cues and product-of-experts aggregation, it sidesteps feature-level bottlenecks and delivers robust, plug-and-play inference. Its theoretical guarantees and empirical validation substantiate its practical utility for computational pathology pipelines, while future work promises broader applicability and resilience in multi-modal, shifting domains.