Per-Frame Source Blending

• Per-frame source blending is a family of frame-level operations that fuse multiple source representations into a common output under conditions of misalignment and unequal reliability.
• Techniques include spatially variant reconstruction in exposure fusion, asymmetric blending in video interpolation, and dual-stream generative compositing for 3D editing.
• Its applications span video codecs, generative compositing, and even radio astronomical calibration, addressing both designed fusion strategies and unwanted blending artifacts.

Per-frame source blending denotes a family of frame-level operations in which multiple source representations are fused, selectively preserved, or forcibly collapsed into a common output. Across recent arXiv literature, the term spans several distinct technical settings: multi-exposure fusion modulates pyramid reconstruction depth patch-by-patch; generative compositing conditions a diffusion model on parallel source and target streams; video frame interpolation assigns asymmetric roles to two reference frames; stream-switching codecs merge several side-information frames into one identical reconstruction; and SKA Epoch of Reionization analysis treats “source blending” as an adverse overlap of astronomical objects whose conflation biases calibration (2002.01425, Chen et al., 20 Jun 2025, Wu et al., 2024, Dai et al., 2015, Shan et al., 2024).

1. Conceptual Scope

The cited literature does not present a single canonical formulation of per-frame source blending. Instead, it exposes a common problem class: a frame-level system must decide how information from more than one source should contribute to one output under imperfect alignment, unequal reliability, or a hard reconstruction constraint.

In image fusion, the challenge is spatially adaptive smoothing. Standard Laplacian Pyramid Blending uses one fixed pyramid depth everywhere, even though saturated highlights, deep shadows, and strong edges require different reconstruction behavior than flat regions. In generative compositing, the central issue is whether the source context should be preserved or masked when an edited target render specifies a scene change. In interpolation, the problem is that symmetric averaging of two imperfectly aligned sources produces blur and ghosting. In stream switching, multiple decoded source versions of the same picture must be mapped to a single reconstruction regardless of which side-information frame is present at the decoder. In radio astronomy, by contrast, source blending is not a designed fusion rule but a defect: independent sky objects are treated as one source, producing spatial and spectral model errors that propagate into calibration (2002.01425, Chen et al., 20 Jun 2025, Wu et al., 2024, Dai et al., 2015, Shan et al., 2024).

A common misconception is to treat source blending as uniformly synonymous with seamless averaging. The literature shows a broader spectrum. Some methods reduce blending depth near difficult regions, some impose sparse quasi-binary gating, some use piecewise constant merging to guarantee identical output, and one line of work seeks to suppress blending altogether because it corrupts inference (2002.01425, Wu et al., 2024, Dai et al., 2015, Shan et al., 2024).

2. Spatially Variant Reconstruction in Multi-Frame Exposure Fusion

“Spatially Variant Laplacian Pyramids for Multi-Frame Exposure Fusion” identifies a specific weakness of conventional Laplacian Pyramid Blending: the use of a single, fixed pyramid depth over the entire image (2002.01425). In exposure fusion, adjacent frames can differ strongly in intensity near saturated highlights, deep shadows, and high-contrast edges. If too few levels are used, the blend is too local and produces sharp transition artifacts, gradient reversal, and obvious halos. If too many levels are used, the blend becomes overly diffuse: halos spread, fine details weaken, and dynamic range can be reduced because blending behaves too much like a low-frequency average.

The proposed remedy is to alter only the reconstruction stage and make it spatially variant. In addition to the standard Laplacian blend pyramid, the method constructs a Gaussian blend pyramid and reconstructs on overlapping $K \times K$ patches. For each patch $p$ at level $l$, reconstruction is defined as

$$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$

When $\alpha \approx 0$, the patch behaves like standard Laplacian reconstruction; when $\alpha \approx 1$, reconstruction collapses toward the Gaussian-blended result, effectively reducing the contribution of deeper pyramid levels. This local reduction in effective blending depth is the paper’s mechanism for halo suppression (2002.01425).

The control factor $\alpha$ is derived from a local measure of intensity variation: $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$ with $I$ normalized to $[0,1]$ and $p$0. The paper evaluates two patch-wise summaries $p$1: mean Laplacian magnitude and patch variance. Patch variance performs slightly better than average Laplacian weighting and is used for subsequent comparisons. Operationally, high variation increases $p$2, reducing blending depth near difficult edges; low variation keeps $p$3 small and preserves standard multilevel reconstruction (2002.01425).

The method uses 50% patch overlap and merges local reconstructions with a modified raised cosine filter to avoid patch boundaries. For multi-frame exposure fusion, blending is sequential: $p$4 so an exposure stack is fused iteratively rather than being restricted to a two-image case (2002.01425).

Quantitative evaluation on the public MEF dataset uses the no-reference perceptual metric MEF-SSIM. The reported mean scores are as follows.

Method	Mean MEF-SSIM
Gu et al.	0.909
Raman et al.	0.842
Shen et al. (Boosted Laplacian Pyramid)	0.895
Proposed	0.932

Qualitatively, the reported improvements target the canonical failure modes of exposure fusion: halos, dark halos, gradient reversal, and loss of detail. The paper states that the proposed approach is more effective than Boosting Laplacian Pyramid and the method of Gu et al. at eliminating halos and dark patches while maintaining detail and dynamic range at roughly the same level as the boosted Laplacian approach (2002.01425). This suggests that per-frame source blending in exposure fusion is best understood not as a single global fusion scale, but as a locally modulated reconstruction policy.

3. Dual-Stream Source–Target Compositing in 3D-Grounded Editing

“BlenderFusion: 3D-Grounded Visual Editing and Generative Compositing” places per-frame source blending inside a layering–editing–compositing pipeline (Chen et al., 20 Jun 2025). A source image or frame $p$5 is segmented into editable entities, using Grounding DINO to refine 2D boxes, SAM2 to extract object masks, and Depth Pro to estimate depth before back-projection into 3D object reconstructions. These entities are imported into Blender as a source scene $p$6, edited through object translation, rotation, scaling, removal, insertion, replacement, attribute changes, non-rigid deformation, camera motion, or background replacement, and rendered as both a source scene render $p$7 and a target scene render $p$8. Each render contains an RGB image and an object index mask from Blender’s Object Index Pass (Chen et al., 20 Jun 2025).

The compositing stage is a diffusion-based generative compositor that extends Stable Diffusion v2.1 to process source and target scenes in parallel. The architecture is dual-stream: a single weight-shared denoising UNet processes the two streams independently, while allowing interaction through self-attention. The first UNet layer is expanded from 4 to 15 input channels, with zero-initialized weights for the new channels. The channel partition is explicit: 4 channels for the VAE-encoded image or noise, 5 channels for Blender render information consisting of 4 rendering-image channels and 1 instance-mask channel, and 6 channels for camera parameters encoded with Plücker embeddings. Each stream also receives text tokens built from CLIP-embedded class labels and MLP-projected positional encodings of 3D bounding boxes or poses (Chen et al., 20 Jun 2025).

The core per-frame source blending mechanism is source masking. During training, each foreground object is independently masked with probability $p$9, and the mask is applied to both the source image $l$0 and the source Blender render $l$1. The appendix specifies that masking uses the 2D bounding box derived from the 3D object bounding box, with dilation, and that masked objects have their corresponding source bounding-box information dropped for the source stream. Background regions are also randomly masked using boxes with similar aspect ratio to foreground objects. The stated purpose is to reduce object inpainting bias and support object removal and background replacement. At test time, the model can mask both the source image and source render, or only the source image, depending on the editing task (Chen et al., 20 Jun 2025).

A second training strategy, simulated object jittering, addresses entanglement between object motion and camera motion. The key symbolic change is to replace $l$2 and $l$3 with $l$4 and $l$5, creating a reconstruction-style setup in which object positions are jittered while camera motion is fixed. Random source masking is then applied separately to $l$6 and $l$7. The paper states that this significantly improves disentangled control of both object and background in ablations (Chen et al., 20 Jun 2025).

BlenderFusion does not introduce a dedicated temporal loss or explicit recurrent temporal model. Temporal consistency is described as implicit, arising from 3D-consistent conditioning, rendering both source and target in Blender, shared diffusion weights, and training on paired video frames (Chen et al., 20 Jun 2025). In this formulation, per-frame source blending is not explicit pixel propagation from prior frames; it is conditional preservation or suppression of source content under 3D-grounded target guidance.

4. Asymmetric Blending in Video Frame Interpolation

“Perception-Oriented Video Frame Interpolation via Asymmetric Blending” argues that blur and ghosting in video frame interpolation arise from unavoidable motion errors and misalignment in supervision (Wu et al., 2024). The paper’s response is PerVFI, which rejects symmetric averaging in favor of an Asymmetric Synergistic Blending (ASB) module. One reference frame is treated as the primary content carrier, while the other contributes complementary information, especially for occluded or missing regions. This asymmetry is the explicit design principle: the two sources are not given equal contribution everywhere (Wu et al., 2024).

Given two input frames $l$8, the method extracts multiscale features $l$9 and bidirectional flows $$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$0 and $$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$1, then constructs intermediate pyramid features $$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$2. ASB is built from a Pyramid Alignment Module (PAM) and an Adaptive Dilation Module (ADM). PAM forward-warps the first branch using softmax splatting, while the second branch is aligned relative to the first using reverse-flow priors and a multiscale deformable convolution network. The paper’s stated intent is to align first and blend only afterward, rather than averaging misregistered features (Wu et al., 2024).

The blend control is a self-learned sparse quasi-binary mask. At pyramid level $$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$3, an initial binary mask $$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$4 is obtained from the flow through a thresholding operator with default $$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$5. ADM expands this mask using three bias-free convolution layers of kernel sizes $$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$6, which together form a $$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$7 dilation field, computes sample-dynamic attention from the aligned features, and projects the result to obtain $$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$8. The quasi-binary mask is then

$$I_{p_{Recon}^l} = \begin{cases} (1-\alpha)\,\big(\mathcal{G} \circledast (I_{p_{Recon}^{l+1}\uparrow}) + I_{p_{LapBlend}^l}\big) + \alpha\, I_{p_{GaussBlend}^l}, & \text{if } l < M \ I_{p_{LapBlend}^l}, & \text{if } l = M. \end{cases}$$9

where $\alpha \approx 0$0, $\alpha \approx 0$1 during training and $\alpha \approx 0$2 during inference, and $\alpha \approx 0$3 by default (Wu et al., 2024).

The final blending equation is

$\alpha \approx 0$4

If $\alpha \approx 0$5, the output inherits mostly from $\alpha \approx 0$6; if $\alpha \approx 0$7, it inherits mostly from $\alpha \approx 0$8. Intermediate values are limited to uncertain or adaptively dilated regions. The paper explicitly argues that a fully adaptive mask tends to center around $\alpha \approx 0$9 and yields blur, whereas the sparse quasi-binary design prevents widespread competition between slightly misaligned structures (Wu et al., 2024).

PerVFI then decodes the blended feature $\alpha \approx 1$0 with a conditional normalizing flow-based generator $\alpha \approx 1$1, modeling $\alpha \approx 1$2 and training with negative log-likelihood plus a perceptual loss weighted by $\alpha \approx 1$3. The paper positions this as a practical generative alternative to deterministic regression, GAN instability, and iterative diffusion latency (Wu et al., 2024). In this setting, per-frame source blending is explicitly asymmetric: the method preserves a dominant structure and borrows complementary content only where the mask authorizes it.

5. Piecewise Constant Merge Functions for Stream Switching

“Merge Frame Design for Video Stream Switching using Piecewise Constant Functions” addresses a different form of per-frame source blending: the decoder must reconstruct one identical frame from any one of several possible side-information (SI) frames (Dai et al., 2015). This requirement arises in bitrate adaptation and view switching. Rather than relying on distributed source coding with bit-plane and channel coding, the paper introduces a merge frame whose parameters force multiple SI transform coefficients onto the same reconstruction value through a piecewise constant (PWC) mapping (Dai et al., 2015).

For each block and transform coefficient, the paper defines

$\alpha \approx 1$4

where $\alpha \approx 1$5 is the step size and $\alpha \approx 1$6 is a horizontal shift. If all SI q-coefficients for a given coefficient fall in the same interval of this function, they all reconstruct to the same value. The merge information consists essentially of one step size $\alpha \approx 1$7 per frequency group and one shift $\alpha \approx 1$8 per block and coefficient coded in merge mode (Dai et al., 2015).

The paper distinguishes two scenarios. In fixed-target merging, a target frame $\alpha \approx 1$9 is given in advance and the goal is exact reconstruction from any SI frame. A constructive rule is provided: if

$\alpha$0

then choosing

$\alpha$1

and an associated shift $\alpha$2 guarantees

$\alpha$3

To reduce overhead, the paper also uses a group-wise step size $\alpha$4 with per-block shifts (Dai et al., 2015).

In the second scenario, the merged reconstruction and merge parameters are jointly optimized under a rate-distortion criterion: $\alpha$5 The practical choice is a smaller group step size

$\alpha$6

followed by selection of $\alpha$7 through

$\alpha$8

To make $\alpha$9 entropy-friendly, the paper proposes a “spike + uniform” distribution learned by a rate-constrained Lloyd-Max procedure (Dai et al., 2015).

The codec supports intra, skip, and merge block modes. Complexity is low because the decoder only applies the transmitted PWC mapping; no bit-plane channel decoding is required. Reported experimental findings include that block size $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$0 performed best among the tested sizes $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$1, $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$2, and $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$3; that the “Spike + Uniform” $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$4 outperformed a naive learned distribution; and that the optimized M-frame outperformed both DSC-based D-frames and SP-frames. The reported BD-rate savings include up to $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$5 over D-frame for static view switching, up to $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$6 over D-frame and $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$7 over SP-frame in Scenario 2 average case, up to $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$8 over D-frame and $$\mathcal{M}(I) = \exp\left(-\frac{(I - 1.0)^2}{2\sigma^2}\right),$$9 over SP-frame in Scenario 2 worst case, up to $I$0 over D-frame and $I$1 over SP-frame in Scenario 3 average case, and up to $I$2 over D-frame and $I$3 over SP-frame in Scenario 3 worst case (Dai et al., 2015).

Here, per-frame source blending is exacting rather than perceptual. The objective is not visual fusion between sources but source-invariant reconstruction: any permissible SI frame must decode to the same output.

6. Source Blending as a Calibration Defect in SKA EoR Analysis

In “An evaluation of source-blending impact on the calibration of SKA EoR experiments,” source blending means that independent radio sources overlap or lie sufficiently close on the sky that they are treated as one source under the telescope resolution (Shan et al., 2024). The paper distinguishes stacked, overlapping, and proximity cases. It identifies a two-fold impact: spatial-domain errors, in which flux-density distribution and position are wrong because two sources are combined into one apparent source, and frequency-domain errors, in which the blended object acquires an incorrect integrated spectral index. Because sky-based calibration is solved per frequency channel, the latter becomes a chromatic gain error (Shan et al., 2024).

The paper introduces a blending estimate

$I$4

where $I$5 is projected source area, $I$6 is source density per unit area, and $I$7 is a dimensionless blending threshold with $I$8. Using the W08/S$I$9 sky model and SKA1-Low observing conditions, the estimated blending level is roughly $[0,1]$0 to $[0,1]$1 across the sky, with one representative example giving a total blending ratio of $[0,1]$2. For controlled tests, the paper constructs sky models with blending ratios of $[0,1]$3, $[0,1]$4, and $[0,1]$5, corresponding to 200, 20, and 2 blended source pairs in a 4000-source sky model (Shan et al., 2024).

The methodological framework is the HEVAL pipeline, which combines end-to-end simulation and analytic calibration analysis. Sky models are generated in ideal and blended forms, visibilities are synthesized with OSKAR, relative gain errors are derived by sky-based calibration using the logarithmic PWL method, and residuals are propagated into visibilities, imaged with WSClean, and Fourier transformed into 3D and cylindrical 2D power spectra. The visibility model is

$[0,1]$6

with

$[0,1]$7

Linearized residual gain relations are written as

$[0,1]$8

and solved by SVD (Shan et al., 2024).

The main quantitative conclusion is that additive residuals from calibration against sky models with blending ratios of $[0,1]$9 and $p$00 significantly contaminate the EoR window, whereas $p$01 leaves little residual imprint within it. The inferred practical tolerance threshold is approximately $p$02, described as roughly 5 blended source pairs per 10,000 sources (Shan et al., 2024). At $p$03 blending, the paper also finds some excess residual power in the EoR window for the EoR signal itself, but this excess is below one-tenth of the EoR signal and is therefore not presented as the dominant limitation (Shan et al., 2024).

This usage reverses the sign of the concept. In the previous sections, blending is a designed operator. Here it is a pathology that must be mitigated through de-blending in sky-model construction, higher-resolution or multiwavelength cross-matching, frequency-dependent error mitigation such as spectrally smoothed antennas, baseline weighting, and simultaneous multi-channel calibration, and imaging-weight optimization, for which Briggs robustness $p$04 gives a better trade-off in the studied case (Shan et al., 2024). A plausible implication is that “per-frame source blending” should be interpreted relative to task semantics: in some domains it is the mechanism that produces a desired frame, while in others it is precisely the defect that prevents reliable inference.