Papers
Topics
Authors
Recent
Search
2000 character limit reached

Late Fusion in Multimodal Decision Systems

Updated 8 July 2026
  • Late Fusion is a decision-level strategy that integrates final predictions from independent modalities, preserving separate processing paths.
  • It employs various methods—from shallow MLPs to weighted averaging and neural network combiners—to fuse outputs in tasks like classification, retrieval, and object detection.
  • Empirical evidence shows that while late fusion can improve performance and modularity, its benefits depend on task-specific architectures and the nature of the fused outputs.

Late fusion is a family of fusion strategies in which heterogeneous modalities, views, descriptors, detectors, or predictors are combined at the decision stage rather than at the raw-input or intermediate-feature stage. In the cited literature it appears under several closely related names—decision-level fusion, late classifier fusion, and, in a Bag-of-Visual-Words setting, visual words integration—but the shared principle is consistent: each branch is processed independently until a terminal representation, score vector, probability distribution, detection set, or trajectory forecast is available, and only then are the outputs combined (Bi et al., 2020, Morvant et al., 2012, Ali et al., 2017). This design makes late fusion a recurrent solution for multimodal classification, retrieval, segmentation, object detection, scientific machine learning, clustering, recommendation, and collaborative autonomy, while also making its advantages and limitations highly task dependent.

1. Definition, terminology, and contrast with earlier fusion

In multimodal product classification, late fusion is explicitly defined as decision-level fusion that fuses predictions from independent text and image classifiers rather than fusing features (Bi et al., 2020). In multimedia indexing, it is late classifier fusion, where the scores of each modality are combined at the decision level (Morvant et al., 2012). In content-based image retrieval, late fusion is described as visual words integration after separate descriptor-specific Bag-of-Visual-Words pipelines have already been formed (Ali et al., 2017). These formulations differ in implementation detail, but all separate per-modality learning from cross-modality combination.

The principal contrast is with early fusion and intermediate or feature-level fusion. Early fusion merges modalities before or at the beginning of the encoder, as in channel concatenation before a UNet or ViT encoder (Remedios et al., 2024, Tziafas et al., 2022). Intermediate fusion merges hidden representations during the network, as in feature-level fusion for ECG or encoder-stage fusion for pancreas segmentation (Oladunni et al., 6 Aug 2025, Remedios et al., 2024). Late fusion, by contrast, delays integration until class probabilities, logits, final embeddings, ranked retrieval scores, object hypotheses, or predicted trajectories have already been computed.

This distinction is not merely architectural. Several papers tie late fusion to modularity, black-box reuse, or communication efficiency. In RGB-D Vision Transformers, late fusion preserves the pretrained patch embedder and encoder and combines the final [CLS][CLS] embeddings after independent encoding (Tziafas et al., 2022). In autonomous-driving object detection, late fusion is motivated precisely because it does not require altering the original detector’s network structure (Yang et al., 2024). In collaborative perception and collaborative trajectory prediction, it reduces the need to exchange high-dimensional features, replacing them with compact object attributes or forecasts (Fadili et al., 3 Jul 2025, Madjid et al., 24 Apr 2026).

2. Canonical mechanisms and mathematical forms

Although the term is singular, the mechanism of late fusion varies by the fused object. The fused object may be a probability vector, a logit vector, an embedding, a histogram, a ranked score list, a partition matrix, a detection cluster, a conformal score, or a trajectory forecast. A common pattern is independent branch training followed by a shallow or structured combiner.

In multimodal e-commerce classification, the text classifier and image classifier each output a probability distribution over 27 classes, PtP_t and PiP_i, and the two vectors are concatenated into a 54-dimensional input for a shallow fusion network (Bi et al., 2020). The one-layer form is

Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),

while the two-layer form is

h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).

In ECG classification, late fusion is a weighted sum of independently produced softmax outputs,

y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,

with the weights selected by grid search on validation data (Oladunni et al., 6 Aug 2025).

In RGB-D ViTs, late fusion operates on the final modality-specific [CLS][CLS] embeddings,

hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),

where ff is averaging, max pooling, or concatenation (Tziafas et al., 2022). In scene classification from synchronized image and audio, the primary networks are trained independently, their softmax layers are removed, the pre-softmax outputs are concatenated, and a tertiary neural network performs the final classification (Bird et al., 2020).

In retrieval, late fusion may occur after descriptor quantization or after ranking. In the FREAK–SIFT CBIR system, separate codebooks are learned, histograms HFREAKH_{FREAK} and PtP_t0 are formed, and the late-fused representation is the concatenation

PtP_t1

which remains late because the two descriptor streams are kept separate until BoVW histograms have already been constructed (Ali et al., 2017). In query-adaptive retrieval, the fused object is not a feature vector but a set of per-feature query-specific scores with adaptive weights,

PtP_t2

where the weights are estimated from sorted score-curve patterns for each query (Wang et al., 2018).

The following examples illustrate how diverse the fused object can be while remaining recognizably late fusion.

Setting Fused object Representative form
E-commerce classification Class probabilities Concatenate PtP_t3 and PtP_t4 into 54 dimensions
RGB-D ViT recognition Final embeddings Fuse PtP_t5 tokens after shared-weight encoding
CBIR with FREAK and SIFT BoVW histograms Concatenate descriptor-specific histograms
ECG classification Softmax outputs Weighted sum of modality predictions
Object detection Detections and class scores Cluster, match, or rescore box hypotheses
Collaborative trajectory prediction Forecasts with uncertainty Fuse track-level future trajectories

These cases suggest that late fusion is best understood structurally: fusion is “late” when branch-specific processing is preserved until a terminal, semantically meaningful output space has already emerged.

3. Empirical behavior across classification, retrieval, segmentation, and recommendation

Several studies report clear improvements from late fusion over unimodal or feature-level baselines, but the empirical pattern is not uniform across tasks.

In e-commerce product classification on the SIGIR 2020 E-Commerce Workshop Data Challenge, the uni-image classifier obtained PtP_t6 macro-F1, the uni-text classifier PtP_t7, feature-level fusion PtP_t8, decision-level fusion PtP_t9 and PiP_i0 on the two test phases, and an ensemble of late-fusion models reached PiP_i1, yielding first place on the final leaderboard (Bi et al., 2020). The comparison is important because feature-level fusion did not surpass text alone, whereas decision-level fusion did.

In scene classification for autonomous machines, the independently trained image and audio models achieved PiP_i2 and PiP_i3 10-fold CV accuracy, while the late-fused tertiary neural network reached PiP_i4 (Bird et al., 2020). On unseen data, the audio classifier obtained PiP_i5, the image classifier PiP_i6, and the multi-modality late-fusion model PiP_i7 (Bird et al., 2020). The examples reported in that work emphasize correction of modality-specific failures: a city scene with a water feature misclassified as river by the audio classifier alone, and a crowded street misclassified as forest by the image classifier alone.

In RGB-D object recognition with pretrained ViTs, late fusion outperformed the early-fusion variants that altered the embedding stage. The paper reports late/cat at PiP_i8 top-1 accuracy in the first ROD trial, compared with PiP_i9 and Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),0 for the dual-embedder and joint-embedder early-fusion variants, and later reports Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),1 for ViT-B late, Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),2 for ViT-L late, and Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),3 for SWIN-B late on the Washington RGB-D Objects dataset (Tziafas et al., 2022). The same study reports ViT-B early at Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),4, materially below the late-fusion counterpart.

In image retrieval, late fusion of FREAK and SIFT improved mean average precision on all three reported datasets: on Corel-1000 the best late-fusion MAP was Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),5 versus Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),6 for SIFT only and Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),7 for FREAK only; on Corel-1500 the corresponding figures were Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),8, Pfusion=Softmax(W[Pt;Pi]+b),\mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W[P_t;P_i]+b),9, and h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).0; on OT-Scene they were h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).1, h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).2, and h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).3 (Ali et al., 2017). Query-adaptive late fusion further addresses a recurrent retrieval problem: some features are discriminative for some queries and distractors for others. The reported method estimates query-specific feature effectiveness from sorted score curves and is described as resilient to distractor features (Wang et al., 2018).

Recommendation provides a different empirical reason for late fusion: scaling long, semantically rich inputs. In GRAM, coarse user prompts and fine-grained item prompts are encoded separately and fused only in the decoder. On the Beauty dataset, the full late-fusion model reached Recall@5 h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).4 and NDCG@5 h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).5, compared with h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).6 and h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).7 for the early-fusion variant; on the Toys dataset the reported figures were h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).8 for late fusion and h=ReLU(W1[Pt;Pi]+b1),Pfusion=Softmax(W2h+b2).\mathbf{h}=\mathrm{ReLU}(W_1[P_t;P_i]+b_1), \qquad \mathbf{P}_{\mathrm{fusion}}=\mathrm{Softmax}(W_2\mathbf{h}+b_2).9 for early fusion (Lee et al., 2 Jun 2025). The paper attributes the difference to preserved item detail and the shift of most fine-grained encoding cost offline.

The counterexamples are equally important. In pancreas segmentation from imperfectly registered T2w and T1w MRI, late fusion by averaging logits produced small, sometimes significant improvements in the basic UNet, but it was not the best fusion approach for either model; the best fusion was early/mid in the basic UNet and naive early input concatenation in nnUNet, with statistically significant Dice gains of y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,0 and y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,1, respectively (Remedios et al., 2024). In multimodal ECG classification, intermediate fusion achieved the peak accuracy of y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,2 percent and outperformed late fusion with Cohen’s y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,3 relative to late fusion (Oladunni et al., 6 Aug 2025). These results directly contradict any universal claim that later fusion is intrinsically superior.

4. Detection, collaborative perception, and trajectory forecasting

Late fusion has a distinct object-level meaning in detection and autonomy. Here the fused entities are often detections, boxes, tracks, or predicted futures rather than class probabilities.

ALFA formulates late fusion for object detection as agglomerative clustering of detector predictions using both bounding box locations and class scores (Razinkov et al., 2019). Each cluster is treated as one object hypothesis, and the fused location is a weighted combination of clustered bounding boxes. On PASCAL VOC 2007 and 2012, ALFA is reported to achieve up to y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,4 lower error than the best individual detectors and up to y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,5 lower error than Dynamic Belief Fusion (Razinkov et al., 2019). Its similarity function combines IoU with the Bhattacharyya coefficient over class distributions, making the fusion simultaneously geometric and semantic.

The Multi-modal Multi-class Late Fusion method for autonomous driving uses projected 3D detections from Complex-YOLO and 2D detections from YOLOv8, models the class scores as evidence parameters for a Dirichlet distribution, fuses beliefs with Dempster–Shafer theory, and predicts an objectness score from IoU, confidence, distance, and fused uncertainty (Yang et al., 2024). The reported gains on KITTI include y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,6 to y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,7 in 2D detection AP and y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,8 to y^=α1y^1D+α2y^2D+α3y^Trans,α1+α2+α3=1,\hat{y}=\alpha_1\hat{y}_{\mathrm{1D}}+\alpha_2\hat{y}_{\mathrm{2D}}+\alpha_3\hat{y}_{\mathrm{Trans}}, \qquad \alpha_1+\alpha_2+\alpha_3=1,9 in 3D detection AP relative to the original 3D detectors (Yang et al., 2024). In that formulation, uncertainty is part of the late-fusion representation rather than a post hoc diagnostic.

A related but distinct design is the hybrid late-cascade fusion network for LiDAR and RGB 3D object detection (Sgaravatti et al., 25 Apr 2025). Its late stage matches projected LiDAR boxes to RGB detections to remove LiDAR false positives, while the cascade stage uses epipolar constraints and frustums from unmatched RGB detections to recover LiDAR false negatives. The study reports significant performance improvements, especially for Pedestrians and Cyclists, with improvements of [CLS][CLS]0 AP over single-modal detectors, and states that the system remains real-time at [CLS][CLS]1 FPS with PointPillars + Faster-RCNN fusion (Sgaravatti et al., 25 Apr 2025).

Collaborative perception pushes late fusion toward communication-constrained multi-agent systems. One framework operates solely on shared 3D bounding box attributes—category, size, position, and orientation—without access to the underlying detection models, and reports up to five times lower position error, a scale error reduced by a factor of [CLS][CLS]2, orientation error reduced by half, and perfect [CLS][CLS]3 precision and recall when fusing detections from heterogeneous perception systems (Fadili et al., 3 Jul 2025). Another collaborative formulation moves fusion from perception to prediction: each vehicle runs its own forecasting stack and shares only future trajectories and associated uncertainty, rather than feature maps (Madjid et al., 24 Apr 2026). On the real-world V2V4Real dataset, collaborative prediction improved the success rate by [CLS][CLS]4 and [CLS][CLS]5 for the two intelligent vehicles relative to individual forecasting (Madjid et al., 24 Apr 2026). In both cases, the rationale for late fusion is not only accuracy but deployability under bandwidth, synchronization, and proprietary-model constraints.

5. Optimization, uncertainty, and formal late-fusion models

A substantial part of the late-fusion literature is not heuristic aggregation but explicit optimization.

The PAC-Bayesian MinCq framework defines a weighted majority vote

[CLS][CLS]6

and upper-bounds its risk through the C-bound

[CLS][CLS]7

where the first moment is the average margin and the second moment captures margin variance (Morvant et al., 2012, Morvant et al., 2014). MinCq minimizes the empirical second moment under a positive-margin constraint, thereby favoring accurate and diverse voters. The ranking extension adds an order-preserving pairwise loss so that positive examples are ranked above negatives, directly targeting Mean Average Precision (Morvant et al., 2014). On PASCAL VOC'07, the kernelized pairwise-average variant is reported with average MAP [CLS][CLS]8, exceeding the reported RBF SVM stacking value of [CLS][CLS]9 (Morvant et al., 2012).

In multi-view clustering, late fusion becomes a consensus-partition problem. OS-LFMVC-CS defines the consensus partition matrix as

hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),0

where hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),1 are base partition matrices, hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),2 are permutation matrices, and hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),3 are learned view weights (Ou et al., 2024). The framework jointly optimizes consensus structure and discrete labels in a compressed subspace, avoids post hoc hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),4-means, and is reported to achieve the best overall performance on Citeseer, Cora, ProteinFold, NUSWIDE, and Reuters, while remaining linear in sample size and deterministic because there is no downstream hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),5-means step (Ou et al., 2024).

Conformal prediction offers another formalization of late fusion: aggregate uncertainty at the score level rather than merge prediction regions directly. The proposed multivariate score-based method constructs a quantile envelope

hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),6

using projected multivariate conformal scores from each view (Rivera et al., 2024). The paper states that coverage is preserved while prediction regions become more efficient than direct region fusion. On a synthetic classification task with hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),7, the average prediction set sizes were hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),8 for the best individual view, hcls=f(hclsrgb,hclsd),\mathbf{h}_{cls}=f(\mathbf{h}_{cls}^{rgb},\mathbf{h}_{cls}^{d}),9 for pre-fused, and ff0 for MVCP (Rivera et al., 2024).

Late fusion can also be a structural prior for extrapolation. The Late Fusion Neural Operator separates state encoding from parameter injection: a neural operator computes hidden states from the system state, and a sparse regression library then incorporates parameters,

ff1

Across advection, Burgers, and 1D and 2D reaction–diffusion equations, the reported average RMSE reduction relative to the second-best method is ff2 in-domain and ff3 out-domain (Tegelen et al., 17 Apr 2026). Here “late” means disentangling parameter effects from learned state dynamics until after latent state computation.

6. Comparative assessment, misconceptions, and open problems

The literature does not support a single universal rule such as “late fusion is always better” or “late fusion is only score averaging.” Rather, it supports a more conditional interpretation.

First, late fusion is often advantageous when the constituent branches are already strong and heterogeneous. The e-commerce study explicitly combines a strong CamemBERT text model with a weaker but complementary ResNet152+DCL image model and improves over text alone (Bi et al., 2020). The RGB-D ViT results are strongest when late fusion allows full reuse of unimodal pretraining (Tziafas et al., 2022). Collaborative perception papers rely on late fusion precisely because the branches may be black-box, cross-vendor systems (Fadili et al., 3 Jul 2025).

Second, late fusion is not synonymous with a trivial weighted average. The surveyed work includes shallow MLPs over class probabilities (Bi et al., 2020), tertiary neural networks over pre-softmax activations (Bird et al., 2020), query-adaptive weighting derived from score-curve geometry (Wang et al., 2018), PAC-Bayesian quadratic programs (Morvant et al., 2012), Dempster–Shafer uncertainty fusion (Yang et al., 2024), graph expansion in retrieval (Park et al., 25 Feb 2026), and Gaussian-process fusion of shared forecasts (Madjid et al., 24 Apr 2026).

Third, late fusion does not eliminate all fusion pathologies. In table-text retrieval, standard late fusion is said to risk missing relevant contexts, because table segments and passages are retrieved individually and aligned online (Park et al., 25 Feb 2026). HELIOS addresses this by harmonizing early fusion, late fusion, and LLM reasoning: it first retrieves edge-based bipartite subgraphs, then performs query-relevant node expansion as the late-fusion stage, and finally applies star-based LLM refinement. The reported improvements on OTT-QA reach up to ff4 in recall and ff5 in nDCG, and removing the node-expansion late-fusion stage reduces AR by ff6 and nDCG@50 by ff7 (Park et al., 25 Feb 2026). This suggests that late fusion may need structured retrieval or reasoning support when relational context is distributed across modalities.

Fourth, the best fusion point remains architecture- and task-specific. In imperfectly registered MRI, the optimal fusion block differed between a basic UNet and nnUNet, and the gains were small (Remedios et al., 2024). In ECG classification, intermediate fusion rather than late fusion produced the best predictive and interpretability outcomes (Oladunni et al., 6 Aug 2025). A plausible implication is that late fusion is strongest when preserving branch autonomy is more valuable than learning deep cross-modal interactions; when cross-modal interactions are themselves the signal, earlier or intermediate fusion may dominate.

Future work in the cited corpus points toward hybridization rather than doctrinal preference. HELIOS explicitly combines early fusion, late fusion, and LLM refinement (Park et al., 25 Feb 2026). The LiDAR–RGB detector combines late fusion with cascade fusion (Sgaravatti et al., 25 Apr 2025). GRAM uses separate encoders and decoder-stage late fusion to preserve detail under long-context constraints (Lee et al., 2 Jun 2025). These designs imply that the central research question has shifted from whether to fuse late toward what to keep separate, what to combine late, and how to formalize the combiner so that modularity, calibration, recall, and cross-modal complementarity are balanced rather than traded off blindly.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (19)

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Late Fusion.