Multi-Factor Gaussian Fusion Overview

• Multi-Factor Gaussian Fusion is a family of probabilistic methods that combine Gaussian factors such as tasks, sensors, and latent processes to model cross-factor dependencies.
• It employs techniques like joint covariance constructions, latent space alignment, and density pooling to effectively integrate diverse information with calibrated uncertainty.
• Empirical studies in fields like geological modeling, image fusion, tracking, and autonomous driving demonstrate significant performance gains and improved prediction accuracy.

Searching arXiv for the cited works and closely related formulations to ground the article. Multi-Factor Gaussian Fusion denotes a family of probabilistic fusion procedures in which multiple factors—such as tasks, fidelities, data sources, sensors, latent processes, or Gaussian primitives—are combined through Gaussian-process, Gaussian-mixture, or Gaussian-state constructions. Across the literature summarized here, the central operation is to encode dependence among factors either in a joint covariance, a latent embedding, a pooling rule over predictive densities, or a multi-criterion selection rule over Gaussian elements. The resulting models are used for heterogeneous information fusion in geological resource modeling, multiband image fusion, distributed tracking, multi-source surrogate modeling, Lie-group state estimation, continuous-time factor graphs, 3D Gaussian Splatting, and end-to-end autonomous driving (Vasudevan et al., 2012, Ravi et al., 2024, Sharma et al., 2024).

1. Joint covariance constructions

A canonical formulation is the multi-task Gaussian-process prior, in which $m$ latent functions $f_1(x),\dots,f_m(x)$ are jointly modeled as

$$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$

The covariance is block-structured, with entries $K_{ij}(x,x')$, and a standard construction is the linear model of coregionalization or intrinsic coregionalization model,

$$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$

Here each $k^{(q)}$ controls correlation at a particular resolution, while the positive-semidefinite matrix $B^{(q)}$ encodes inter-task covariance. In this setting, learning $B^{(q)}$ amounts to learning the information-fusion weights directly from data. Posterior prediction retains the usual GP form, with mean $\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$ and covariance $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$, and hyperparameters are estimated by maximizing the Gaussian marginal log-likelihood. In geological resource modeling, this joint construction was used to fuse heterogeneous element-concentration measurements over large real sensor datasets (Vasudevan et al., 2012).

A related construction appears in non-stationary spatio-temporal multi-fidelity Gaussian processes. There, a base latent process $f_1(x),\dots,f_m(x)$0 is shared across fidelities, and each fidelity-specific deviation is represented by an independent discrepancy process $f_1(x),\dots,f_m(x)$1, so that

$f_1(x),\dots,f_m(x)$2

This yields a decomposed covariance in which cross-fidelity dependence is carried by $f_1(x),\dots,f_m(x)$3, while same-fidelity corrections are carried by $f_1(x),\dots,f_m(x)$4. The same model admits a latent linear representation $f_1(x),\dots,f_m(x)$5 with block-diagonal prior covariance on $f_1(x),\dots,f_m(x)$6, enabling Woodbury-based likelihood evaluation without explicitly forming the dense multi-fidelity covariance matrix. The cited framework combines this decomposition with a Vecchia approximation and generalized least squares mean removal for scalable, likelihood-based inference under stationary and non-stationary fidelity integration (Colombo et al., 5 May 2026).

In multiband image fusion, the Gaussian mechanism is formulated through a forward observation model and a linear mixture model. A target image $f_1(x),\dots,f_m(x)$7 is observed through $f_1(x),\dots,f_m(x)$8 degraded sensors,

$f_1(x),\dots,f_m(x)$9

and $$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$0 itself is modeled as $$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$1, where $$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$2 is the abundance matrix. After vectorization, each observation takes the Gaussian linear form

$$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$3

Under the stated approximation on the blur/down-sampling operators, the joint log-likelihood is Gaussian, the maximum-likelihood estimator is obtained from the normal equations, and the Fisher information matrix

$$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$4

governs uniqueness. Invertibility of $$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$5 requires $$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$6, which formalizes when the fused estimate is unique (Arablouei, 2017).

These formulations share a common principle: fusion is expressed by a structured Gaussian dependency model rather than by post hoc averaging. A plausible implication is that, in this family of methods, the essential modeling decision is not whether to fuse, but how the covariance or linear observation structure should expose cross-factor dependence.

2. Heterogeneous domains, latent source spaces, and augmented factors

A major difficulty in multi-source Gaussian fusion is heterogeneity of the input domains. Standard GPs assume a single common input space, but when $$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$7 the kernel $$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$8 is undefined across different dimensions, and one cannot simply stack the observations. The two-stage framework based on Input Mapping Calibration and Latent-Variable Gaussian Processes addresses this by first learning affine maps

$$[f_1(x),\dots,f_m(x)]^\top \sim GP(\mu(x),K(x,x')).$$9

from each source space into a common reference space, and then training a source-aware GP on the mapped inputs. The LVGP augments each mapped quantitative input with a learnable 2D latent embedding $K_{ij}(x,x')$0 of the source label and uses a kernel of the form

$K_{ij}(x,x')$1

This construction was introduced precisely to fuse sources that share the same output of interest but are parameterized over different input spaces (Comlek et al., 2024).

A closely related formulation treats the source label itself as a categorical variable embedded into a latent space. With observations $K_{ij}(x,x')$2, each source $K_{ij}(x,x')$3 is assigned a latent coordinate $K_{ij}(x,x')$4, typically with $K_{ij}(x,x')$5, and the GP is defined over the latent-augmented input $K_{ij}(x,x')$6. The learned distances $K_{ij}(x,x')$7 then become a dissimilarity metric between sources, and the normalized quantity

$K_{ij}(x,x')$8

measures dissimilarity relative to a reference source fixed at the origin. This makes inter-source similarity interpretable rather than implicit. In the reported demonstrations, the latent map was used not only for prediction but also to diagnose source discrepancy and to identify dissimilar sources whose removal improved performance for a chosen reference dataset (Ravi et al., 2024).

Latent-Map Gaussian Processes generalize this idea further by using a one-hot encoding $K_{ij}(x,x')$9 of the source and a learnable latent map $$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$0 so that $$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$1. The composite GP input becomes $$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$2, with kernel

$$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$3

In this formulation, sources that are mutually predictive are pulled together in latent space, while incompatible sources are pushed apart. The same framework extends to calibration by augmenting the input with calibration parameters $$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$4, and the latent plot can be used for subset selection when only the well-correlated sources should be retained (Oune et al., 2021).

Another branch of multi-factor Gaussian fusion augments the GP input not with source identities but with multiple low-fidelity-derived features. Instead of modeling $$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$5 directly, the high-fidelity target is expressed as

$$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$6

or

$$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$7

A GP with an ARD squared-exponential kernel is then trained in this augmented space. The motivation is explicitly linked to embedology: delayed samples or derivatives of the low-fidelity signal can restore information lost in the projection to $$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$8 alone, making the map to the high-fidelity response smoother and easier to learn from few high-fidelity samples (Lee et al., 2018).

Taken together, these methods show that “factor” need not mean only “task” or “sensor.” It may also mean source identity, fidelity level, calibration state, time-delay coordinate, or derivative-based feature. A common misconception is that Gaussian fusion requires all sources to share a single input parametrization; the heterogeneous-domain formulations above are explicit counterexamples.

3. Pooling, product, and quotient fusion of Gaussian densities

Another major branch of the subject fuses already-formed Gaussian predictive densities rather than constructing a joint covariance over raw observations. In linear pooling, the fused predictive density is

$$K_{ij}(x,x')=\sum_{q=1}^Q B_{ij}^{(q)}\,k^{(q)}(x,x').$$9

with nonnegative weights summing to one. In log-linear pooling, or generalized product of experts,

$k^{(q)}$0

When the experts are Gaussian, the fused density is again Gaussian, with precision $k^{(q)}$1 and mean

$k^{(q)}$2

The cited work places GP priors on the log-weights, samples them with HMC/NUTS, and aggregates the resulting fused Gaussians by Monte Carlo. On the reported synthetic dataset, joint learning methods outperformed stacking methods, and log-linear fusion yielded lower negative log-predictive density than linear pooling across varying numbers of experts or random Fourier frequencies (Ajirak et al., 2024).

Conservative fusion arises when factor correlations are unknown or when double counting must be avoided. The Harmonic Mean Density interpolation defines, for local densities $k^{(q)}$3 and simplex weights $k^{(q)}$4,

$k^{(q)}$5

followed by normalization. For two Gaussians, the denominator mixture is moment-matched by a Gaussian, and the final fused mean and covariance take a closed form after dividing the Gaussian numerator by that approximation. The method is described as consistent and conservative, with the denominator term preventing over-optimistic covariance collapse. The same summary states that naive Kalman fusion is optimistic and often divergent because it ignores cross-correlation, whereas covariance intersection is conservative but cumbersome to extend to mixtures (Sharma et al., 2024).

Theoretical properties of HMD were developed explicitly. The normalization constant $k^{(q)}$6 is convex in $k^{(q)}$7 and satisfies $k^{(q)}$8; the fused density obeys the pointwise bounds $k^{(q)}$9; it is monotone in each input density; and for each $B^{(q)}$0, $B^{(q)}$1 for any $B^{(q)}$2. Computationally, for two densities, the reported cost is approximately $B^{(q)}$3 that of covariance intersection, while retaining consistency and supporting Gaussian-mixture inputs with only modest modifications (Sharma et al., 2024).

In decentralized Gaussian-mixture fusion, the central object is not a product but a quotient:

$B^{(q)}$4

After expanding the numerator into Gaussian mixands, one obtains a mixture of non-Gaussian quotients $B^{(q)}$5. The cited work proposes two importance-sampling approximations: Direct Local Sampling, which approximates each quotient mixand in parallel, and Indirect Global Sampling, which samples from a global proposal and performs a single-shot EM-style responsibility update. This makes exact or approximate Bayesian fusion tractable for recursive decentralized settings in which common information must be removed rather than ignored (Ahmed, 2019).

A recurrent misconception is that Gaussian fusion is simply multiplication of Gaussian densities. The density-pooling and quotient-based literature shows that the main issue is frequently not Gaussian closure, but correlation management: conservative pooling, denominator correction, or explicit common-information removal may be required to maintain consistency.

4. Fusion on manifolds and in continuous time

When Gaussian factors live on nonlinear state spaces, fusion requires a change of reference before Euclidean Gaussian rules can be applied. On a Lie group $B^{(q)}$6, each factor is a concentrated Gaussian around a local reference point $B^{(q)}$7,

$B^{(q)}$8

To fuse several such factors, a common reference $B^{(q)}$9 is selected, offsets $B^{(q)}$0 are computed, and each covariance is transported by the Jacobian of the change-of-coordinate map,

$B^{(q)}$1

The transformed covariance is $B^{(q)}$2, and the fused information matrix and vector become

$B^{(q)}$3

The fused coordinate is $B^{(q)}$4, and the fused group element is $B^{(q)}$5. The paper compares exact Jacobians, first- and second-order Taylor expansions, parallel transport, and parallel transport with curvature correction; the last is reported to achieve similar accuracy to state-of-the-art optimization-based algorithms on $B^{(q)}$6 at a fraction of the computational cost (Ge et al., 2024).

In continuous-time factor-graph fusion, the Gaussian object is the trajectory prior itself. Vehicle state $B^{(q)}$7 is modeled as the solution of a white-noise-on-jerk linear SDE,

$B^{(q)}$8

with $B^{(q)}$9 a GP. The resulting kernel induces exactly sparse prior factors between neighboring discrete-time states and supports interpolation at arbitrary timestamps,

$\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$0

This enables asynchronous fusion of GNSS, IMU, lidar-odometry, and speed-sensor measurements without strict synchronization. The maximum a posteriori objective is a sum of Gaussian prior and measurement residual terms, and the resulting Hessian is sparse because each factor only couples nearby states. The reported system includes both loose and tight GNSS coupling, uses continuous-time GP regression as the trajectory representation, and achieved a mean 2D positioning error of $\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$1 on a $\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$2 route in Aachen when raw GNSS observations were fused with lidar odometry in a tight coupling (Zhang et al., 2023).

These two formulations illustrate that Gaussian fusion is not restricted to Euclidean regression or density averaging. It also includes coordinate transport on manifolds and Gaussian-process priors over trajectories, provided that the local Gaussian structure can be re-expressed in a common coordinate system.

5. Fusion of Gaussian primitives in scene representations

In 3D Gaussian Splatting, the fusion target is a set of scene primitives rather than a latent function or a posterior state. After registering two sub-maps, space is partitioned into non-overlap regions $\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$3 and $\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$4 and an overlap region

$\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$5

For each Gaussian in the overlap, three scores are computed: a skeleton-adherence score

$\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$6

a normalized local-detail score derived from the GA-KPConv response, and a scene-center proximity score

$\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$7

These are combined as

$\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$8

with $\mu_*(x_s)=K_*^\top [K+\Sigma_n]^{-1}y$9. For each overlapping pair, the Gaussian with the higher composite score is retained, subject to a high-confidence override. The chosen Gaussian keeps its original mean, covariance, opacity, and spherical-harmonic coefficients. The stated motivation is to replace hard-threshold merging, which often produces holes, missing object parts, and uneven radiance, by a continuous, structure-aware, detail-preserving, and spatially balanced selection rule (Liu et al., 28 Jul 2025).

A different scene-level use of Gaussian primitives appears in end-to-end autonomous driving. There, a fixed set of $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$0 two-dimensional Gaussians is initialized over the bird’s-eye-view scene as intermediate carriers. Each Gaussian has physical attributes $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$1 and two learned feature vectors $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$2. Sensor features from image and LiDAR backbones are projected onto these Gaussians through deformable cross-attention and standard cross-attention, the Gaussian attributes are refined by an MLP over explicit features, and a cascade planning head repeatedly queries the Gaussians to refine anchor trajectories. In this framework, “fusion” occurs through the progressive refinement of Gaussian carriers that accumulate semantic, spatial, and planning-relevant information from multiple sensors (Liu et al., 27 May 2025).

Although both methods are Gaussian-primitive fusion, their semantics differ. In the 3D-GS case, Gaussian primitives are retained or discarded on the basis of multiple continuous factors. In the autonomous-driving case, Gaussian primitives are persistent latent carriers that mediate sensor aggregation and downstream planning. This suggests that the term “multi-factor” can refer either to multi-criterion scoring or to multiple feature branches acting on a shared Gaussian representation.

6. Empirical behavior, diagnostics, and recurring themes

The empirical literature consistently reports benefits when nontrivial cross-factor dependence exists and is modeled explicitly. In geological resource modeling, three element concentrations were modeled over $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$3 bore-hole samples in a $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$4 region. Ten-fold block cross-validation over block sizes from $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$5 to $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$6 showed that a multi-task GP with a nonstationary neural-network kernel reduced mean squared error by up to approximately $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$7 for small blocks and still by approximately $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$8 for the largest blocks, relative to three independent GPs; negative log-probability was also lower, indicating better-calibrated uncertainty (Vasudevan et al., 2012).

In multi-source surrogate modeling, the same pattern appears under sparse or heterogeneous data. For the Ti6Al4V alloy case, the two-stage IMC+LVGP framework improved test NRMSE from $\Sigma_*(x_s)=K_{**}-K_*^\top [K+\Sigma_n]^{-1}K_*$9 to $f_1(x),\dots,f_m(x)$00 overall and reduced test NRMSE on the scarce FSW source from $f_1(x),\dots,f_m(x)$01 to $f_1(x),\dots,f_m(x)$02, reported as a $f_1(x),\dots,f_m(x)$03 reduction relative to a single-source GP-FSW baseline. In the FeCrAl alloy example, LVGP reduced test NRMSE on the GE reference set from $f_1(x),\dots,f_m(x)$04 to $f_1(x),\dots,f_m(x)$05, and a targeted-source-removal LVGP-Target reached approximately $f_1(x),\dots,f_m(x)$06. In the Ackley synthetic example, LVGP multi-source modeling reduced test NRMSE from approximately $f_1(x),\dots,f_m(x)$07 for GP alone to approximately $f_1(x),\dots,f_m(x)$08 (Comlek et al., 2024, Ravi et al., 2024).

For conservative density fusion in tracking, the Harmonic Mean Density method was reported to outperform other conservative strategies while remaining consistent. In a 3D near-constant-velocity scenario with three bearings-plus-range sensors and $f_1(x),\dots,f_m(x)$09 Monte Carlo runs, HMD achieved position RMSE of approximately $f_1(x),\dots,f_m(x)$10 and velocity RMSE of approximately $f_1(x),\dots,f_m(x)$11, about $f_1(x),\dots,f_m(x)$12–$f_1(x),\dots,f_m(x)$13 better than covariance intersection and about $f_1(x),\dots,f_m(x)$14–$f_1(x),\dots,f_m(x)$15 better than arithmetic averaging; naive fusion diverged late in the run. In a 2D bearings-only passive-tracking scenario, HMD achieved position RMSE of approximately $f_1(x),\dots,f_m(x)$16, about $f_1(x),\dots,f_m(x)$17–$f_1(x),\dots,f_m(x)$18 better than covariance intersection, while arithmetic averaging suffered greater than $f_1(x),\dots,f_m(x)$19 track loss (Sharma et al., 2024).

In Gaussian-primitive scene fusion, the 3D-GS method reported rendering metrics on the ScanNet-GSReg test set of PSNR $f_1(x),\dots,f_m(x)$20, SSIM $f_1(x),\dots,f_m(x)$21, and LPIPS $f_1(x),\dots,f_m(x)$22, compared with PSNR $f_1(x),\dots,f_m(x)$23, SSIM $f_1(x),\dots,f_m(x)$24, and LPIPS $f_1(x),\dots,f_m(x)$25 for GaussReg; the average PSNR gain was $f_1(x),\dots,f_m(x)$26 and approximately $f_1(x),\dots,f_m(x)$27 in some challenging scenes such as $f_1(x),\dots,f_m(x)$28-$f_1(x),\dots,f_m(x)$29. The same paper reported a $f_1(x),\dots,f_m(x)$30 reduction in RRE on complex scenes for registration and a $f_1(x),\dots,f_m(x)$31 PSNR improvement for fusion on the ScanNet-GSReg and Coord datasets (Liu et al., 28 Jul 2025). In autonomous driving, GaussianFusion reported Extended PDMS $f_1(x),\dots,f_m(x)$32 and Original PDMS $f_1(x),\dots,f_m(x)$33 on NAVSIM, and a CARLA Driving Score of $f_1(x),\dots,f_m(x)$34 with Success Rate $f_1(x),\dots,f_m(x)$35 on Bench2Drive; the ablation study attributed gains to explicit Gaussian fusion, the implicit branch, and cascade planning (Liu et al., 27 May 2025).

Two recurring themes unify these otherwise diverse results. First, the gains are strongest when the factors are correlated but not redundant: MTGPs “borrow statistical strength,” LVGPs exploit source similarity encoded in latent space, and HMD removes approximate common information without collapsing uncertainty. Second, the Gaussian assumption does not force a single fusion mechanism. Depending on the problem, fusion may mean joint covariance learning, latent alignment, conservative pooling, quotient approximation, coordinate transport, asynchronous factor-graph inference, or multi-criterion primitive selection. Uncorrelated tasks remain nearly independent in the joint-covariance view, and dissimilar sources can be identified and even removed in latent-source models. This suggests that “Multi-Factor Gaussian Fusion” is best understood not as one fixed algorithm, but as a technical design pattern for coupling multiple information factors under a Gaussian probabilistic calculus.