Random Poisson Blending

• Random Poisson Blending is a technique that solves gradient-domain Poisson equations while incorporating stochastic variations in region selection, guidance composition, and sampling.
• It bridges classical discretized formulations with modern INR-based methods to balance deterministic constraints with randomization for improved image blending.
• Empirical evaluations show that INR-based random blending reduces artifacts and color discrepancies compared to traditional methods, though challenges remain with mask accuracy and extreme content mismatches.

Searching arXiv for the cited papers to ground the article in current paper metadata and context. Random Poisson Blending denotes a family of gradient-domain compositing procedures in which Poisson constraints are retained while stochasticity is introduced into the blending region, guidance construction, sampling process, or optimization schedule. In the classical formulation, blending reconstructs an unknown composite image inside a region $\Omega$ by matching a guidance field under Dirichlet boundary conditions. In more recent neural variants, the same variational principle is implemented either as a differentiable loss on pixel images or as a continuous optimization problem over Implicit Neural Representations (INRs). Within this literature, "Deep Image Blending" formalizes a deterministic differentiable Poisson blending loss coupled to deep style and content objectives, and explicitly does not define a randomized Poisson method (Zhang et al., 2019). By contrast, the Neural Poisson Solver framework presents a continuous INR-based solver and describes random Poisson blending as stochastic variation in region selection, guidance composition, and coordinate sampling while still solving Poisson-domain constraints (Wu et al., 2024).

1. Classical Poisson formulation and the role of randomness

Classical Poisson blending is posed on a blending region $\Omega \subset \mathbb{R}^2$ with boundary $\partial \Omega$, a target image $t$, a source image $s$, and an unknown composite image $f$ reconstructed inside $\Omega$. The standard variational objective minimizes

$E(f) = \iint_{\Omega} \|\nabla f - v\|^2$

subject to the Dirichlet boundary condition

$f|_{\partial \Omega} = t|_{\partial \Omega}.$

The associated Euler–Lagrange condition yields

$$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$

On a regular pixel grid, discretization produces a sparse SPD linear system with the standard 4-neighbor discrete Laplacian on the left-hand side and the discrete divergence of the guidance field plus boundary injections on the right-hand side (Zhang et al., 2019).

This formulation is deterministic once the source, target, mask, and guidance field are fixed. Consequently, randomness is not intrinsic to classical Poisson image editing. The data describe "Random Poisson Blending" as an extension in which stochasticity is introduced while preserving the Poisson objective. In the INR-based account, this includes randomly sampling $\Omega \subset \mathbb{R}^2$0, randomly mixing source and target gradients in the guidance field, and using stochastic coordinate minibatches during optimization (Wu et al., 2024). This suggests that randomness operates at the level of problem instantiation or solver dynamics rather than altering the underlying variational principle.

2. Deterministic deep image blending and its Poisson loss

"Deep Image Blending" replaces the classical linear-system solve with a differentiable loss that emulates Poisson image blending and jointly optimizes it with deep style and content objectives (Zhang et al., 2019). The method constructs a blended image

$\Omega \subset \mathbb{R}^2$1

where $\Omega \subset \mathbb{R}^2$2 is the optimizable image, $\Omega \subset \mathbb{R}^2$3 is the target image, and $\Omega \subset \mathbb{R}^2$4 is a coarse binary mask aligned to $\Omega \subset \mathbb{R}^2$5.

Its Poisson gradient loss is

$\Omega \subset \mathbb{R}^2$6

where $\Omega \subset \mathbb{R}^2$7 denotes a Laplacian filter applied channel-wise. Because $\Omega \subset \mathbb{R}^2$8 equals $\Omega \subset \mathbb{R}^2$9 outside $\partial \Omega$0, applying the Laplacian over the whole $\partial \Omega$1 implicitly enforces the target boundary condition, while the interior term emulates gradient-domain matching (Zhang et al., 2019).

The method is explicitly non-random with respect to the Poisson formulation itself. The paper states that it does not define any randomized variant of Poisson blending and does not introduce randomness into the guidance field, boundary conditions, or the Poisson constraints. The only explicit randomness is the initialization of $\partial \Omega$2 as 2D random noise at the start of stage one, which affects the optimization trajectory but is unrelated to the Poisson formulation itself (Zhang et al., 2019). This distinction is central to the concept of random Poisson blending: stochastic initialization alone is not equivalent to randomizing the Poisson problem.

3. Full deep objective, masks, and optimization dynamics

The deep formulation combines the Poisson gradient loss with VGG-16-based content, style, histogram, and total variation terms. Features $\partial \Omega$3 and Gram matrices $\partial \Omega$4 are computed from VGG-16 pretrained on ImageNet. The style loss uses layers conv1_2, conv2_2, conv3_3, and conv4_3, while the content loss uses conv2_2 (Zhang et al., 2019).

In stage one, the content loss is masked to preserve source semantics inside the object region,

$\partial \Omega$5

and the style loss compares the blended image to the target style,

$\partial \Omega$6

The histogram loss is

$\partial \Omega$7

and the total variation loss is

$\partial \Omega$8

The full stage-one objective is

$\partial \Omega$9

Stage two refines the result using content-to-$t$0 and style-to-$t$1 losses, but omits the Poisson term (Zhang et al., 2019).

Optimization proceeds in two stages with L-BFGS over image pixels. Stage one initializes $t$2 as 2D random noise, computes $t$3, evaluates the losses, and updates $t$4 to minimize $t$5. Stage two initializes $t$6 and optimizes $t$7 using content, style, histogram, and TV losses. The reported setting is $t$8 iterations in both stages, with runtime of approximately 5 minutes for $t$9 on a single NVIDIA GTX 1080Ti (Zhang et al., 2019).

The mask and boundary treatment are integral to the method’s interpretation. Boundary consistency is ensured by constructing $s$0 with the target outside $s$1 and applying the Laplacian over the whole image. No explicit feathering or edge-aware filtering is used; smoothness is enforced by the gradient loss and TV, while interior style adaptation is driven by deep style and content features (Zhang et al., 2019). Coarse masks are acceptable, and the content loss helps retain object identity even with coarse delineation.

4. Continuous INR-based Poisson solving and boundary-condition-free blending

The Neural Poisson Solver reformulates Poisson blending in a continuous function space using INRs (Wu et al., 2024). An INR models a signal as

$s$2

with $s$3. The blended signal is parameterized as another INR $s$4 and optimized to satisfy Poisson constraints continuously rather than on a fixed pixel grid.

The primary loss is the direct variational form,

$s$5

combined with a global color-consistency term over the complement $s$6,

$s$7

so that

$s$8

The framework also gives an optional PDE-residual form,

$s$9

but emphasizes the boundary-condition-free design based on global $f$0 (Wu et al., 2024).

This design is presented as a response to limitations of grid-based Poisson blending. The paper attributes distortions, artifacts, and color shifts to discretization and boundary handling in traditional methods, and argues that strict enforcement only on $f$1 makes solutions sensitive to mask shape and placement (Wu et al., 2024). In the INR setting, $f$2 and $f$3 are obtained by automatic differentiation at arbitrary coordinates, which avoids finite-difference staircasing and extends naturally to higher-dimensional signals. A plausible implication is that random Poisson blending becomes easier to define in continuous space, because randomness can be injected into spatial sampling and region geometry without first redesigning a grid solver.

5. Randomization mechanisms in Poisson-domain blending

Within the provided material, random Poisson blending is described most explicitly in the Neural Poisson Solver framework (Wu et al., 2024). The stochastic components are all external to the Poisson constraints themselves and concern the choice of region, guidance, and sampling.

Component	Randomized mechanism	Source framing
Region $f$4	Random masks, blob-like or irregular shapes, splines, random size, aspect, and placement; in 3D, random boxes or spheres	Randomly sample $f$5
Guidance field $f$6	Random weights $f$7, per-direction or per-channel mixtures, stronger-gradient selection	Guidance composition
Optimization samples	Random coordinate minibatches in $f$8 and $f$9; in NeRF, random camera poses	Stochastic sampling

For 2D INR-based blending, the guidance field may be chosen as

$\Omega$0

The paper evaluates three deterministic guidance constructions that can also serve as ingredients in randomized pipelines: feature blending, where $\Omega$1 is taken from the stronger gradient at each point,

$\Omega$2

feature overlay with $\Omega$3, and feature smoothing with $\Omega$4 (Wu et al., 2024).

The paper further states that random Poisson blending "simply introduces stochasticity in region selection, guidance composition, and sampling, while still solving the Poisson constraints" (Wu et al., 2024). In this sense, the adjective "random" does not denote a different PDE, but a randomized policy for instantiating and training the same Poisson-domain problem. This differs from the discussion in "Deep Image Blending," where randomized guidance, randomized boundary conditions, stochastic masking, or dropout in deep style features are presented only as speculative ways a random variant could be constructed and are explicitly not part of the published method (Zhang et al., 2019).

6. Two-dimensional and three-dimensional realizations

For 2D images in the INR formulation, the inputs are a target INR $\Omega$5, a source INR $\Omega$6, a blending region $\Omega$7, and a balance parameter $\Omega$8 (Wu et al., 2024). The blending INR may be initialized as $\Omega$9 or as a trainable MLP initialized near $E(f) = \iint_{\Omega} \|\nabla f - v\|^2$0. The reported 2D backbone is DINER, using hash indexing plus an MLP to capture higher-frequency detail. Optimization uses Adam with cosine annealing on a single NVIDIA RTX 3090 GPU, with blending time of approximately 50–100 seconds per blend (Wu et al., 2024).

The optimization loop samples coordinates in $E(f) = \iint_{\Omega} \|\nabla f - v\|^2$1 for gradient-domain terms and coordinates in $E(f) = \iint_{\Omega} \|\nabla f - v\|^2$2 for global color consistency. The reported total objective in the paper focuses on

$E(f) = \iint_{\Omega} \|\nabla f - v\|^2$3

with $E(f) = \iint_{\Omega} \|\nabla f - v\|^2$4 when the optional boundary and PDE residual terms are omitted (Wu et al., 2024). The parameter $E(f) = \iint_{\Omega} \|\nabla f - v\|^2$5 controls gradient preservation versus edge smoothness: smaller $E(f) = \iint_{\Omega} \|\nabla f - v\|^2$6 preserves detail inside $E(f) = \iint_{\Omega} \|\nabla f - v\|^2$7 at the risk of edge color mismatch, while larger $E(f) = \iint_{\Omega} \|\nabla f - v\|^2$8 improves edge transitions with slight detail smoothing.

For 3D scenes, the same idea is adapted to NeRFs with 5D input (Wu et al., 2024). The source, target, and blending models are denoted $E(f) = \iint_{\Omega} \|\nabla f - v\|^2$9, $f|_{\partial \Omega} = t|_{\partial \Omega}.$0, and $f|_{\partial \Omega} = t|_{\partial \Omega}.$1, and the blending region becomes a random 3D ROI $f|_{\partial \Omega} = t|_{\partial \Omega}.$2. To save memory, optimization is restricted to points within $f|_{\partial \Omega} = t|_{\partial \Omega}.$3, and density is set to zero outside $f|_{\partial \Omega} = t|_{\partial \Omega}.$4 during training-time rendering:

$f|_{\partial \Omega} = t|_{\partial \Omega}.$5

In practice, the paper blends density and color inside $f|_{\partial \Omega} = t|_{\partial \Omega}.$6 via

$f|_{\partial \Omega} = t|_{\partial \Omega}.$7

and

$f|_{\partial \Omega} = t|_{\partial \Omega}.$8

with $f|_{\partial \Omega} = t|_{\partial \Omega}.$9 and $$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$0 (Wu et al., 2024). Training uses random camera poses and $$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$1 rays per pose on a single RTX 3090, takes several hours, and produces final renders at $$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$2 due to memory and time limits.

7. Empirical behavior, comparisons, and limitations

The 2019 deep image method evaluates against copy-and-paste, Poisson Blending, Poisson Blending + Style Transfer, Style Transfer + Poisson Blending, Deep Image Harmonization, and GP-GAN (Zhang et al., 2019). The reported qualitative findings are consistent across the provided summary: copy-and-paste produces obvious boundary artifacts; Poisson Blending smooths boundaries but exhibits inconsistent style and texture together with color seep; Poisson Blending + Style Transfer yields more style consistency but unpleasant illumination and color during the Poisson step; Style Transfer + Poisson Blending sacrifices source content and structure; Deep Image Harmonization improves illumination but fails to hide the boundary and needs precise foreground; GP-GAN is supervised and shows poor generalization on the reported test cases (Zhang et al., 2019). In a user study with 30 participants and 20 tasks, the method is preferred in 9/10 real-world tasks and 8/10 painting tasks, corresponding to 80–90% preference in those cases (Zhang et al., 2019).

The Neural Poisson Solver reports average pixel-wise errors for gradient-domain fidelity and color smoothness, measured as $$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$3 over pixels, and states that across multiple scenes it reduces Poisson-PDE-related errors by $$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$4 to $$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$5 compared to discrete PIE (Wu et al., 2024). Qualitatively, the classical discrete method is reported to suffer from color shifts and residual artifacts, whereas the continuous solver preserves global colors and blends details seamlessly. In 3D NeRF blending, the demonstrations include heterogeneous blending such as text in ice or water and large-scale blending such as a chessboard encased in ice, with the Neural Poisson Solver producing coherent blends from multiple viewpoints relative to direct ROI replacement (Wu et al., 2024).

The limitations are also distinct. For deep image blending, the paper does not enumerate failure modes explicitly; the provided summary states that heavy texture or style mismatch and extreme lighting inconsistency may require careful tuning of $$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$6, $$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$7, and $$\Delta f = \operatorname{div} v \quad \text{in } \Omega, \qquad f|_{\partial \Omega} = t|_{\partial \Omega}.$$8, and runtime can be several minutes per image depending on L-BFGS convergence (Zhang et al., 2019). For the Neural Poisson Solver, sharp-edged masks may reveal the mask through unnatural transitions, 5D blending requires three NeRF models of identical size and joint ray tracing, GPU memory use is high, training is slow, render resolution is limited by memory and time, and extreme color or semantic mismatch remains challenging (Wu et al., 2024).

A recurrent misconception is that any Poisson-based compositor with random initialization is therefore a random Poisson method. The literature summarized here does not support that equivalence. The differentiable method in "Deep Image Blending" is deterministic in its Poisson loss and uses randomness only for initialization (Zhang et al., 2019). By contrast, the INR-based account defines random Poisson blending through stochastic region selection, guidance mixing, and sampling while maintaining the original Poisson-domain objective (Wu et al., 2024).