FusionStitching: Unified Fusion & Stitching

• FusionStitching is a unified framework combining fusion and stitching methods to optimize deep learning and multimodal image processing.
• It leverages compiler-level optimizations, mesh-based warping, and deep fusion networks to improve GPU execution and alignment fidelity.
• Applications range from accelerated deep learning graph compilation and seamless image stitching to panoramic HDR synthesis and radiance field composition.

FusionStitching encompasses a collection of methods and systems designed to unify or tightly couple fusion and stitching operations for visual data, deep learning workloads, and structured representations such as radiance fields. The term consistently denotes approaches that seek to optimize computation, improve alignment, or fuse content through coordinated algorithmic or architectural strategies. While "FusionStitching" originated in the context of deep learning graph compilation for efficient GPU execution, it now also describes advanced image and modality fusion pipelines, seamless image stitching under large parallax, multimodal segmentation, panoramic HDR generation, and explicit radiance field composition.

1. Compiler-Level FusionStitching for Deep Learning Workloads

Compiler-driven FusionStitching frameworks aim to minimize the end-to-end execution time and resource cost of memory-intensive deep learning operations by unifying fine-grained elementwise, reduction, and small GEMM computations into large, high-occupancy GPU kernels (Long et al., 2018, Long et al., 2019, Zheng et al., 2020). The systems operate as just-in-time (JIT) passes atop TensorFlow’s XLA intermediate representation, exploiting comprehensive fusion plan optimization and advanced data reuse strategies.

Algorithmic Backbone

• Fusion Plan Optimization: For a computation DAG $G=(V,E)$, FusionStitching enumerates candidate "fusion patterns" $P_j$, each a connected subgraph of memory-bound ops. Pattern selection utilizes integer linear programming (ILP) (Long et al., 2019) or dynamic programming with beam search (Zheng et al., 2020), maximizing cumulative fusion gain subject to disjointness and acyclicity constraints:

$$\max_{X_j\in\{0,1\}}\;\sum_j X_j f(P_j), \quad X_u+X_v \leq 1 \quad \text{if}\; V_u\cap V_v\neq\emptyset$$

where $f(P_j)$ is either empirical time savings or bandwidth reduction.

• Kernel Composition Schemes: FusionStitching defines four stitching modes:
  1. Kernel Packing: merge independent loops.
  2. Thread Composition: sequential producer-consumer ops within a thread (register reuse).
  3. Warp Composition: intra-warp (register-shuffle) reuse for reductions; enables sharing high-cost intermediates.
  4. Block Composition: intra-block shared-memory reuse for ops with large intermediate footprints (Batched-GEMM, large reductions).

• Template Scheduling: Fused kernels are generated by traversing a schedule grammar that specifies tiling and buffer allocation. Dominance-tree analyses ensure buffer sharing respects live ranges and memory budgets.

Cost Modeling and Tuning

Two-layer cost models predict fusion gain:

• A fast delta-evaluator approximates bandwidth and launch latency savings.
• An accurate latency-evaluator computes cycles by aggregating macroarchitectural factors: instruction counts, occupancy, shared-memory footprint, and microbenchmark-based CPI values.

Performance

FusionStitching demonstrates substantial E2E speedups:

• Kernel launch reduction rate $L_m$ of up to 55 % relative to XLA (Long et al., 2018), with corresponding reduction of global memory accesses and context switch overhead.
• Reported speedups up to 5.7× versus TensorFlow baseline and 2.21× over XLA, with 1.4×–1.45× geometric mean across industrial models such as BERT, Transformerm DIEN, ASR, and CRNN (Zheng et al., 2020, Long et al., 2019).
• Production deployment in large-scale GPU clusters saves on average 7000 GPU-hours per month for 30,000 tasks without regressions (Zheng et al., 2020).

2. FusionStitching for Seamless Image Fusion and Stitching

FusionStitching methods also denote a category of algorithms in image stitching that unify or tightly couple fusion, seam selection, and geometrically-aware warping, frequently integrating photometric, geometric, and structural information.

Mesh-Warp and Combined Constraints

• Mesh‐based content‐preserving warping (CPW) overlays a regular grid on the input and models pixel correspondence via bilinear interpolation. The global energy jointly optimizes sparse geometric matches (points and lines), dense photometric constraints (intensity and gradient), and regularization (smoothness, collinearity) (Chen et al., 2018):

$$E(V) = \alpha E_f(V) + \beta E_l(V) + \gamma E_p(V) + \delta E_c(V) + \eta E_s(V)$$

• Optimization proceeds via linearization of the photometric residual, sparse least-squares solvers, and multi-level (coarse-to-fine) refinement.
• Quantitative RMSE analysis confirms that methods leveraging both constraint types systematically outperform point-only or photometric-only pipelines in alignment, especially under low texture or large parallax.

Depth-Supervised Fusion and Soft Seam Modeling

• The Depth-Supervised Fusion Network (DSFN) extends FusionStitching to robust, parallax-tolerant image synthesis (Jiang et al., 24 Oct 2025).
• Coarse homography and mesh-based residual warping are regularized with monocular depth maps, preserving global and local alignment across disparate depth ranges.
• A graph-inspired "soft-seam" module learns blending masks via U-Net architectures with energy-style losses reflecting terminal constraints, pixel difference cost, smoothness, and depth consistency:

$$\mathcal{L}^f = \rho\mathcal{L}_{\rm terminal} + \tau\mathcal{L}_{\rm cost} + \iota\mathcal{L}_{\rm smooth} + \sigma\mathcal{L}_{\rm reg}$$

• Efficiency is enhanced by dynamic reparameterization in shift regression blocks.
• DSFN achieves state-of-the-art PSNR, SSIM, SIQE, and LPIPS metrics, completing stitching in 67 ms for 512×512 images (Jiang et al., 24 Oct 2025).

3. FusionStitching in Multimodal and Panoramic Image Synthesis

Advanced FusionStitching strategies target complex multimodal and high dynamic range scenarios, emphasizing flexible, parameter-efficient fusion schemes.

Multimodal Semantic Segmentation via StitchFusion

• StitchFusion employs frozen vision transformer encoders as both feature extractors and fusers, interleaving MultiAdapter modules for cross-modal, multi-directional sharing (Li et al., 2 Aug 2024).
• Information from each modality is injected into every other at multiple transformer stages:

$$\hat F_m^{(l)} = F_m^{(l)} + \sum_{n\neq m} \text{DropPath}(F^{\text{Ada}}(\text{LN}(F_n^{(l)})))$$

• The framework achieves state-of-the-art mIoU on MCubeS, DeLiVER, FMB, MFNet, and PST900 datasets, matching or exceeding traditional heavy fusion modules with minimal parameter overhead ($<1$M per added modality).

Neural Augmentation for HDR Panoramic Stitching

• Panoramic HDR FusionStitching combines physics-driven intensity mapping via weighted histogram averaging (WHA) and locally refined CNN augmentation (MEAN) (Zheng et al., 7 Sep 2024).
• Multi-scale exposure fusion is performed via edge-preserving pyramid blending, and further detail is recovered by solving a gradient-domain quadratic optimization in the log-space.
• This coupling enables dynamic-range consistent panoramas with seamless exposure transitions. Metrics on VETHDR-Nikon report PSNR 34.38 dB and SSIM 0.9153, outperforming histogram-matching and CRF baselines. MEF-SSIM reaches 0.970 over held-out test panoramas.

4. FusionStitching as Unified Inpainting and Seamless Synthesis

Recent methods conceptualize FusionStitching as the unification of fusion and rectangling stages in image stitching through mask-guided diffusion-based inpainting (Xie et al., 23 Apr 2024).

Problem Reformulation and Mask Construction

• Both overlap fusion and border rectangling are posed as a single mask-guided pixel modification problem:

$I'(x) = I(x)(1 - M(x)) + F(I(\cdot),x)M(x)$

• Weighted guide masks regulate the strength of pixel resynthesis during the reverse diffusion process, encoding spatial transitions (content, seam, gradient masks).

Diffusion Model Implementation

• A frozen Stable-Diffusion-2 (latent DDPM) model conducts guided denoising over 50 diffusion steps, leveraging a variable mask schedule.
• No model retraining or fine-tuning is required; spatial interpretability is maintained via explicit mask schedules.

Efficacy and Limitations

• The approach outperforms UDIS++ + Stable-Diffusion-2 baselines across hyperIQA/CLIPIQA metrics and is favored by 80 % of human evaluators for seam visibility and misregistration robustness.
• Generalization derives from the underlying large-scale diffusion model, though limitations arise for very large holes or severe color mismatch (Xie et al., 23 Apr 2024).

5. FusionStitching for Structured Scene Representation

FusionStitching extends to the fusion of multiple radiance field (RF) representations, enabling compositional rendering for extended reality applications (Goel et al., 2023).

Distillation-Based RF Composition

• Multiple teacher RFs (TensoRF grids + small MLPs) are distilled into a single student RF by sampling rays through all teachers and allocating responsibility by maximum opacity $\alpha$.
• Per-point L₂ loss aligns the student's density and color fields to those of the owning teacher; optional pixel-space loss further sharpens rendering consistency:

$$L_{\text{total}} = L_{\text{distill}} + \lambda_{\text{rgb}}L_{\text{rgb}} + \lambda_{\theta}\|\theta_S\|^2$$

• Time and memory complexity post-fusion become independent of the teacher count.

Manipulation and Flexibility

• Addition or deletion of component RFs is supported by iterative redistillation.
• Overlap and disjoint subscene handling are naturally managed via the assignment and composition of teacher densities at each sample point.

6. Impact and Practical Significance

FusionStitching methods consistently demonstrate notable improvements in computational efficiency, memory bandwidth reduction, and alignment fidelity across domains:

FusionStitching Variant	Efficiency Metric	Fidelity/Accuracy Metric	Application Domain
DL Compiler	1.45–2.21× speedup	Kernel launch −55%	Deep Learning (TF/XLA)
Mesh/Depth Stitching	~67 ms per pair	PSNR/SSIM SOTA	Image Alignment
StitchFusion (Modal)	SOTA mIoU (4+ sets)	Param Eff. <1M/modality	Multimodal Segmentation
HDR Panoramas	MEF-SSIM 0.970	PSNR 34.38 dB	High Dynamic Range
Unified Inpainting	SOTA IQA metrics	User preference 80 %+	Seamless Stitching
RF Distillation	O(1) eval complexity	Rendering equiv.	XR/Scene Synthesis

The term "FusionStitching" therefore spans technical contexts but shares the unifying principle of joint optimization of fusion and stitching operations, leveraging both algorithmic depth and architectural integration to push the limits of performance and quality across vision, representation, and computational domains.