Heterogeneous Spatial Fusion Overview

• Heterogeneous spatial fusion is the integration of spatial data from diverse sources (modalities, supports, scales) into a unified representation for tasks like prediction and classification.
• It employs methods such as graph-based models, operator-based transformations, and confidence-driven techniques to retain and leverage distinct spatial characteristics.
• Empirical studies show that preserving heterogeneity leads to improved forecasting accuracy, cooperative perception, and image segmentation across various domains.

Searching arXiv for papers related to heterogeneous spatial fusion and the cited works. Heterogeneous spatial fusion denotes the integration of spatial evidence that differs in modality, support, scale, relation type, layer, or acquisition geometry into a representation that remains useful for prediction, reconstruction, localization, or classification. Recent work formulates the problem in markedly different but compatible ways: as a dynamic, multi-relational, multi-type graph learning problem for outbreak forecasting, as a process-based multivariate spatial fusion model over continuous latent surfaces, as a confidence-driven variational model with spatially varying fidelity, as BEV-space intermediate fusion under mixed sensor suites, as distortion-aware volumetric fusion for pinhole–fisheye depth estimation, and as geometry-aware graph fusion over point, line, and area supports (Du et al., 28 May 2025, Wang et al., 2019, Ntouskos et al., 2016, Wei et al., 18 Jul 2025, Zhang et al., 30 Sep 2025, Yu et al., 2 Jul 2026). Across these formulations, the recurring issue is that the relevant spatial signal is not exhausted by a single grid, adjacency, or feature tensor.

1. Conceptual scope and recurring forms of heterogeneity

The literature treats “heterogeneous” as more than a synonym for multimodal. In some settings, heterogeneity is defined by node type, edge type, and layer, as in bi-layer graphs that couple region nodes and case nodes through spatial, genetic, and assignment relations (Du et al., 28 May 2025). In others, it is defined by measurement support, with 0D point, 1D line, and 2D grid observations encoding distinct constraints on the same latent rainfall field (Yu et al., 2 Jul 2026). A process-based statistical view generalizes this further: geostatistical, lattice, and point-pattern data are heterogeneous because their observation operators differ even when the latent scientific process is shared (Wang et al., 2019).

A second recurring form is sensor-configuration heterogeneity. Cooperative perception in vehicle–infrastructure systems must handle nodes that may carry cameras, LiDARs, or both, across nine mixed configurations such as LC+LC, L+C, and L+LC (Wei et al., 18 Jul 2025). Mixed camera geometry yields another version of the problem: pinhole and fisheye cameras differ in distortion, field of view, and depth sensitivity, so fusion in image space is intrinsically misaligned (Zhang et al., 30 Sep 2025). In infrared–visible fusion, heterogeneity arises because thermal salience and visible structure are spatially non-equivalent even when the scene is co-registered (Sun et al., 9 Jan 2026).

A third form is scale heterogeneity. Remote sensing spatiotemporal fusion combines fine-resolution/low-frequency and coarse-resolution/high-frequency observations (Cheng et al., 2016). Road-network representation learning fuses small-, medium-, and large-scale spatial interactions whose meaning changes with road hierarchy and flow patterns (Yang et al., 6 Sep 2025). Medical segmentation work makes a related point within a single image pyramid: coarse semantic features and high-resolution local structure must be fused without collapsing topology (Kabir et al., 19 May 2026).

Representative setting	What is heterogeneous	Fused target
BLUE (Du et al., 28 May 2025)	Location nodes, case nodes, spatial/genetic/assignment relations	Weekly county-level outbreak incidence
Process-based spatial fusion (Wang et al., 2019)	Geostatistical, lattice, point-pattern observation supports	Shared latent spatial surfaces
HeCoFuse (Wei et al., 18 Jul 2025)	Camera, LiDAR, or both at vehicle and infrastructure	Cooperative 3D object detection
PFDepth (Zhang et al., 30 Sep 2025)	Pinhole and fisheye cameras with different projection laws	Multi-view depth maps
Rainfall graph fusion (Yu et al., 2 Jul 2026)	0D point, 1D line, 2D grid supports	Point-support rainfall reconstruction
HetSSNet (Ma et al., 7 Feb 2025)	PAN spatial nodes and LR-MS spectral nodes	HR multispectral image

This suggests that heterogeneous spatial fusion is best understood as a family of methods for reconciling incompatible spatial descriptions rather than a single architectural template.

2. Formal representations of heterogeneity

A common formulation is the heterogeneous graph. In BLUE, the weekly graph snapshot is

$$\mathcal{G}_t=(\mathcal{V}_t,\mathcal{E}_t,\mathcal{T}_V,\mathcal{T}_E),$$

with fixed location nodes $\mathcal{V}^{(c)}$, time-varying case nodes $\mathcal{V}^{(p)}_t$, and three relation types,

$$\mathcal{T}_E=\{\mathcal{E}^{(sp)},\mathcal{E}^{(ge)},\mathcal{E}^{(as)}\},$$

corresponding to spatial, genetic, and assignment edges (Du et al., 28 May 2025). Rainfall reconstruction uses a related graph partition,

$$\mathcal{V}=\mathcal{V}_0\cup\mathcal{V}_1\cup\mathcal{V}_2,$$

where $\mathcal{V}_0,\mathcal{V}_1,\mathcal{V}_2$ are point-, line-, and grid-support layers (Yu et al., 2 Jul 2026). HetSSNet defines an attributed multiplex heterogeneous graph with node-type map $\phi$, edge-type map $\varphi$, PAN nodes, LR-MS nodes, and three relation types encoding PAN spatial structure, LR-MS intra-spectra relations, and PAN–LR-MS spectral relations (Ma et al., 7 Feb 2025).

An alternative formulation is operator-based rather than graph-based. The process-based spatial-fusion framework writes

$$g_j\!\left(\mathbb{E}[\boldsymbol{Y}_j(\cdot)\mid \boldsymbol{\beta}_j,\boldsymbol{Z}_j,\boldsymbol{w}(\cdot)]\right) = \boldsymbol{X}_j(\cdot)\boldsymbol{\beta}_j + B_j\!\bigl(\boldsymbol{Z}_j\boldsymbol{w}(\cdot)\bigr),$$

where $B_j(\cdot)$ is a support-specific operator that subsets, integrates, averages, or otherwise transforms a continuous latent field so it matches the observation support (Wang et al., 2019). This formalism makes heterogeneity a property of the observation operator rather than of node labels.

Variational fusion introduces a third representation. Confidence-driven TGV fusion uses a fused variable $\mathcal{V}^{(c)}$0 and a diagonal confidence matrix $\mathcal{V}^{(c)}$1, with objective

$\mathcal{V}^{(c)}$2

so heterogeneity is encoded by a point-wise confidence field jointly estimated with the fused image (Ntouskos et al., 2016). In this setting, the same spatial location may be trusted differently depending on local residual agreement.

Geometry-aware volumetric methods define heterogeneity through the projection model itself. PFDepth lifts each view-specific feature map into a canonical 3D voxel space via camera-type-specific projections,

$\mathcal{V}^{(c)}$3

$\mathcal{V}^{(c)}$4

so the fused domain is not the image plane but a distortion-aware common 3D volume (Zhang et al., 30 Sep 2025).

These representations are not interchangeable. A plausible implication is that the right formalism depends on where the incompatibility actually resides: in graph relations, in observation support, in local reliability, or in geometry.

3. Fusion operators and architectural patterns

The literature repeatedly rejects the idea that heterogeneous spatial fusion is merely feature concatenation. BLUE performs relation-specific smoothing before graph compression: $\mathcal{V}^{(c)}$5 then constructs region-centered fusion nodes and adaptively fuses relation-specific edge embeddings using a softmax over relations (Du et al., 28 May 2025). The point is explicit: multiple relational channels are smoothed jointly while preserving type semantics.

Cooperative perception in BEV space uses a different operator family. HeCoFuse first brings LiDAR and camera features into a common BEV tensor domain, then applies channel attention and spatial attention: $\mathcal{V}^{(c)}$6

$\mathcal{V}^{(c)}$7

with an Adaptive Spatial Resolution module that downsamples or preserves resolution according to whether a node is camera-only, LiDAR-only, or dual-modality (Wei et al., 18 Jul 2025). Here the fusion variable is a BEV feature map, and heterogeneity is negotiated by per-channel and per-location weighting.

PFDepth separates overlapping and non-overlapping volumetric regions. After identifying voxel visibility masks under pinhole and fisheye projection, it accumulates two branches,

$\mathcal{V}^{(c)}$8

and then combines them by

$\mathcal{V}^{(c)}$9

Fusion therefore distinguishes shared physical support from complementary non-overlap rather than forcing a single view-averaging rule (Zhang et al., 30 Sep 2025).

Variational and state-space models realize fusion differently. Confidence-driven TGV assigns each pixel a closed-form confidence update,

$\mathcal{V}^{(p)}_t$0

so local agreement increases fidelity weight and disagreement lets the regularizer dominate (Ntouskos et al., 2016). DIFF-MF separates channel exchange from spatial exchange. It first builds a discrepancy map,

$\mathcal{V}^{(p)}_t$1

then performs cross-modal state-space scanning on several realigned feature maps in the spatial-exchange module (Sun et al., 9 Jan 2026). WoundFormer, while not multimodal, contributes a decoder-level lesson: topology-preserving coarse-to-fine fusion can be more effective than token-wise MLP decoding when fine spatial boundaries matter (Kabir et al., 19 May 2026).

Remote sensing methods further diversify the operator set. STNLFFM predicts fine-resolution reflectance by transforming reference-date fine pixels and aggregating similar pixels with non-local weights,

$\mathcal{V}^{(p)}_t$2

thereby combining spatial similarity and temporal change consistency (Cheng et al., 2016). HetSSNet generates up to $\mathcal{V}^{(p)}_t$3 basic relationship patterns from three edge types, then aggregates them locally and globally, rather than collapsing all relations into one adjacency (Ma et al., 7 Feb 2025).

A recurring conclusion is that successful heterogeneous spatial fusion usually preserves heterogeneity long enough to compute type-specific or support-specific interactions, and only then compresses to a shared representation.

4. Structure preservation, temporal organization, and optimization

Many methods add explicit constraints to prevent fusion from destroying the structure it is supposed to preserve. BLUE uses a spectral regularizer,

$\mathcal{V}^{(p)}_t$4

to keep the fused graph close to the diffusion geometry of the original heterogeneous graph (Du et al., 28 May 2025). MSRFormer does something analogous across scales through residual accumulation,

$\mathcal{V}^{(p)}_t$5

so small-, medium-, and large-scale interaction features accumulate rather than overwrite each other (Yang et al., 6 Sep 2025). The integrated remote-sensing framework based on DRcGAN imposes a different kind of consistency: the fused HR result must regenerate the observations through a backward degeneration feedback path,

$\mathcal{V}^{(p)}_t$6

which constrains spatial enhancement, temporal transfer, and heterogeneous SAR–optical translation simultaneously (Jiang et al., 2021).

Temporal organization is equally diverse. BLUE keeps graph snapshots static in time and delegates temporal modeling to a graph sequence encoder–decoder over $\mathcal{V}^{(p)}_t$7 (Du et al., 28 May 2025). Air-quality prediction first performs RBF-based spatial fusion and then applies STGCN over a fully connected weighted station graph, with prediction objective

$\mathcal{V}^{(p)}_t$8

and graph convolution based on the weighted adjacency $\mathcal{V}^{(p)}_t$9 (Ma et al., 2021). Fusion-Poly treats time as sensor-frequency heterogeneity: synchronized LiDAR–camera updates and asynchronous camera-only updates are handled differently, with larger observation noise for asynchronous updates,

$$\mathcal{T}_E=\{\mathcal{E}^{(sp)},\mathcal{E}^{(ge)},\mathcal{E}^{(as)}\},$$0

and distinct score updates for synchronized and asynchronous frames (Wu et al., 9 Mar 2026).

Optimization properties also vary sharply. Confidence-driven TGV is biconvex but not jointly convex, and the paper is explicit that the fully joint PDHG extension loses monotonicity-based convergence guarantees (Ntouskos et al., 2016). Drone Referring Localization addresses dense-map imbalance with a Weighted Balance Loss over localization heatmaps, rather than relying on embedding similarity alone (Dai et al., 2022). These differences indicate that heterogeneous spatial fusion is often as much an optimization-design problem as a representation-design problem.

5. Empirical behavior across domains

Empirical evidence consistently favors support-aware or relation-aware fusion over homogeneous baselines, but the magnitude and mechanism vary by domain. In avian influenza forecasting, BLUE reports the best RMSE and F1 on Avian-US with $$\mathcal{T}_E=\{\mathcal{E}^{(sp)},\mathcal{E}^{(ge)},\mathcal{E}^{(as)}\},$$1, and its ablations are unusually sharp: removing cross-layer smoothing worsens RMSE from 0.6255 to 1.5230, removing genetic edges worsens RMSE to 1.7112, and removing the spectral regularizer yields RMSE 2.9998 and F1 0.0014 (Du et al., 28 May 2025). These numbers support the claim that non-spatial biological links and structure-preserving compression are not marginal additions.

In rainfall field reconstruction, the geometry-aware multi-support HGNN reduces RMSE by 23.2% over inverse-distance weighting on Singapore and outperforms convolutional fusion and support-agnostic heterogeneous graph baselines. The same study also reports a negative case: on Sydney, where median gauge spacing is about 3 km and the field appears already well resolved relative to its correlation length, the support-aware model does not outperform IDW (Yu et al., 2 Jul 2026). This directly challenges the misconception that more heterogeneous sources automatically improve reconstruction.

For cooperative perception, HeCoFuse maintains 3D mAP from 21.74% to 43.38% across nine heterogeneous sensor configurations, reaches 43.22% under LC+LC, and attains an even higher 43.38% in the L+LC setting (Wei et al., 18 Jul 2025). PFDepth shows the complementary point for mixed camera geometry: naïve mixed training without proper heterogeneous spatial fusion can collapse pinhole performance, whereas enabling HSF restores strong pinhole and fisheye results; in the full 2 pinhole + 2 fisheye setting, HSF plus 3D Gaussian enhancement yields fisheye AbsRel 8.2 and pinhole AbsRel 7.7 (Zhang et al., 30 Sep 2025).

Image and segmentation tasks show similar patterns. WoundFormer reports 81.9% average Dice on WoundTissueSeg and improves over SegFormer-B5 by 4.3 Dice points by replacing the All-MLP decoder with a spatially-preserving multi-scale fusion head (Kabir et al., 19 May 2026). DIFF-MF reports that removing the spatial-exchange module sharply reduces SF, AG, and SD on M$$\mathcal{T}_E=\{\mathcal{E}^{(sp)},\mathcal{E}^{(ge)},\mathcal{E}^{(as)}\},$$2FD, indicating that simple averaging cannot substitute for cross-modal state-space spatial exchange (Sun et al., 9 Jan 2026). In remote sensing, STNLFFM is reported to be more accurate and robust than STARFM and ESTARFM, especially for heterogeneous landscapes and temporally dynamic areas (Cheng et al., 2016), while HetSSNet achieves the best reduced-resolution and full-resolution pansharpening results across WorldView-3, QuickBird, and GaoFen-2, with its relationship-pattern generation outperforming both meta-path sampling and decoupled adjacency baselines (Ma et al., 7 Feb 2025).

Outside machine learning, the tungsten transmutation study provides a physical-science analogue of why heterogeneous spatial treatment matters. In a 2 cm tungsten shell, the difference between the maximum Re concentration at the back face and the minimum around 1.2 cm depth exceeds 4600 appm after 2 full-power years, while a homogenized treatment would report only a volume-averaged value around 3500 appm (Gilbert et al., 2016). This suggests that “heterogeneous spatial fusion” is not only an algorithmic issue; it is also a warning against collapsing physically distinct local environments into averaged supports.

6. Limitations, misconceptions, and broader implications

A common misconception is that heterogeneous spatial fusion is solved once all modalities are represented as channels in the same tensor. Several papers explicitly argue otherwise. The rainfall study criticizes support-agnostic baselines for reconciling sources in feature space while discarding support geometry (Yu et al., 2 Jul 2026). The process-based statistical framework makes the same point formally through the operator $$\mathcal{T}_E=\{\mathcal{E}^{(sp)},\mathcal{E}^{(ge)},\mathcal{E}^{(as)}\},$$3: change of support is part of the model definition, not a preprocessing nuisance (Wang et al., 2019). PFDepth likewise rejects forced fisheye undistortion into pinhole-like imagery, because camera-model-aware lifting into 3D preserves information that late-stage image-space mixing would distort (Zhang et al., 30 Sep 2025).

A second misconception is that heterogeneity is only a matter of modality. The literature shows otherwise. It can be support type, relation type, temporal frequency, or graph layer. Fusion-Poly’s asynchronous updates show that sensor-rate mismatch is itself a heterogeneous fusion problem (Wu et al., 9 Mar 2026). BLUE’s case nodes and location nodes make granularity mismatch central (Du et al., 28 May 2025). Confidence-driven TGV shows that even within a single image domain, local reliability can vary enough to require spatially varying confidence rather than a global fidelity weight (Ntouskos et al., 2016).

The current literature also has real limitations. Some papers contain notation inconsistencies or malformed equations; BLUE explicitly exhibits both, and some metric tables conflict (Du et al., 28 May 2025). HeCoFuse does not provide dedicated ablations isolating channel attention, spatial attention, and ASR (Wei et al., 18 Jul 2025). The biconvex PDHG extension in confidence-driven TGV lacks a full convergence proof (Ntouskos et al., 2016). WoundFormer reports minimal FLOP overhead relative to SegFormer on one benchmark but noticeably slower inference on another, without reconciling the discrepancy (Kabir et al., 19 May 2026). PFDepth does not provide a closed-form Gaussian splatting kernel or rendering integral, leaving some implementation details implicit (Zhang et al., 30 Sep 2025).

Despite these caveats, the broader implication is stable across domains. Whether the task is infectious-disease forecasting, cooperative perception, field reconstruction, pansharpening, UAV localization, or multimodal classification, the strongest methods do not erase heterogeneity at the input boundary. They model it explicitly—through typed graphs, support operators, geometry-aware projections, relation-specific smoothing, spatially varying confidence, or domain-specific sequence updates—and only then learn a fused representation (Du et al., 28 May 2025, Wang et al., 2019, Wei et al., 18 Jul 2025, Dai et al., 2022, Long et al., 14 May 2026). A plausible implication is that future progress will depend less on adding modalities in bulk and more on preserving the exact way each source constrains space.