View-Coherent Remapping

• View-coherent remapping is a set of operations that transfer scene information, latent state, or execution order across viewpoints while maintaining structural consistency.
• It is applied in dynamic novel view synthesis, multi-camera enhancement, light-field rendering, and SLAM by incorporating 3D geometric fusion, packet-level denoising, and attention-based modulation.
• The approach relies on explicit geometric representations, residual blending, and joint processing techniques to address challenges like large viewpoint changes and depth discontinuities.

View-coherent remapping denotes, across the cited literature, a family of operations that transfer scene information, latent state, or execution order across viewpoints while preserving compatibility on shared content. In dynamic-scene novel view synthesis, the central requirement is that depths support geometrically stable remapping of background across all views and of foreground through time-specific geometry (Yoon et al., 2020). In multi-camera enhancement, coherence is obtained because multiple target and reference views are jointly processed under explicit 3D and pose conditioning rather than enhanced one at a time (Zanjani et al., 12 Jan 2026). In light-field rendering, the same phrase names a GPU-side permutation that reorders thread-to-subpixel assignment so neighboring threads process subpixels with the same or nearby viewpoint indices, restoring warp-level memory efficiency without changing the final interlaced image layout (Sim et al., 6 May 2026). This suggests that the term is best understood not as a single algorithm, but as a coherence criterion applied to remapping in geometry, latent representations, image editing, rendering, and systems design.

1. Concept and coherence criteria

A precise geometric statement of coherent remapping appears in dynamic novel view synthesis, where source images $\mathbf{I}^{r_t}$, camera projection matrices $\Pi$, and depth maps $\mathbf{D}^{r_t}$ are used to define target synthesis by

$$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$

with static-scene warpings from all source views, a time-specific warp of the dynamic content, a target-view dynamic mask, and a learned blending function (Yoon et al., 2020). In that formulation, coherence means that remapping must keep the resulting target view structurally consistent even though source frames belong to different camera poses and different scene states.

A broader formal interpretation is provided by the theory of remapping and navigation in embedding spaces, which defines remapping as “representing and/or updating new information acquired via experience into the internal latent representation formalized by the problem space (i.e., constantly refining a navigable embedding space).” In the same framework, coherence is enforced by requiring, roughly speaking, that assignments of data to different regions agree wherever the regions overlap, and that any way of building up the data for a region by assigning data to subregions agrees with every other way of building up the data for that region by assigning data to subregions (Hartl et al., 20 Jan 2026). This provides a general overlap-consistency criterion that is directly compatible with multi-view reasoning.

The cited work also makes clear that coherent remapping is not identical to pairwise reprojection alone. In ViewMorpher3D, coherence is explicitly said to emerge because the model jointly denoises multiple target and reference views under explicit 3D and pose conditioning, so the same scene content is restored in a mutually compatible way across views; the framework does not rely on an explicit pairwise post-hoc consistency correction (Zanjani et al., 12 Jan 2026). In CoherentRaster, by contrast, coherence is operational rather than photometric: neighboring threads are made coherent in viewpoint space rather than in raster order so that data access matches the organization of tile- and cluster-indexed Gaussian lists (Sim et al., 6 May 2026).

2. Representations and mathematical formulations

The dominant geometric substrate in view-coherent remapping is explicit correspondence under camera change. For static scenes, source-view warping is written as

$$\mathbf{y} = W_{r\rightarrow v}(\mathbf{x};\mathbf{D}^r,\Pi^r, \Pi^v),$$

while in dynamic scenes the analogous warp is

$$\mathbf{y} = W_{r_t\rightarrow v}(\mathbf{x};\mathbf{D}^{r_t},\Pi^r, \Pi^v).$$

The distinction is decisive: in the dynamic case, source frames cannot generally be cross-warped into each other as if they were snapshots of a single static 3D scene, so coherence requires time-specific geometry rather than a single shared reconstruction (Yoon et al., 2020).

The same work formalizes the complementarity between depth from single view and depth from multiview stereo. DSV is complete but scale-inconsistent and view-variant; DMV is view-invariant yet incomplete. The desired corrected depth is required to satisfy three properties: agreement with DMV on static regions, preservation of the relative depth ordering and shape implied by DSV on dynamic regions through the scale-invariant relative gradient operator

$$g(\mathbf{D};\mathbf{x},\Delta\mathbf{x})= {\frac{\mathbf{D}(\mathbf{x}+\Delta\mathbf{x})-\mathbf{D}(\mathbf{x})}{|\mathbf{D}(\mathbf{x}+\Delta\mathbf{x})|+|\mathbf{D}(\mathbf{x})|}},$$

and minimal smooth 3D scene flow between neighboring views or times (Yoon et al., 2020). In this setting, view coherence is a property of the fused depth field.

A second representational family replaces depth with dense 3D coordinates. ViewMorpher3D uses rendered coordinate maps, or C-maps, whose per-pixel value is

$\mathbf{X}(u,v) \in \mathbb{R}^3,$

computed by visibility-aware accumulation: $$\mathbf{X}(u,v) = \sum_{i=1}^{N} \alpha_i(u,v)\prod_{j=1}^{i-1}\big(1-\alpha_j(u,v)\big)\cdot \mathbf{x}_i.$$ These C-maps are paired with dense Plücker ray embeddings

$$(\mathbf{o}\times \mathbf{d}, \mathbf{d}) \in \mathbb{R}^6,$$

and all camera poses and C-maps in a packet are transformed into the coordinate system of the first target camera, with camera positions normalized so that the maximum pairwise camera distance in the packet is $\Pi$0 (Zanjani et al., 12 Jan 2026). The representation is therefore explicitly image-aligned, occlusion-aware, and pose-aware.

A third line of work addresses remapping of stored latent maps rather than images. In neural implicit SLAM, a local map is represented as

$\Pi$1

and remappability under pose correction is defined by an equivariance relation

$\Pi$2

Under $\Pi$3, voxel centers are transformed by $\Pi$4, features are rotated by

$\Pi$5

and regridding is handled by Jacobian-based interpolation

$\Pi$6

In this formulation, view coherence is geometric consistency after pose updates rather than photometric agreement between rendered views (Yuan et al., 2022).

3. Algorithmic mechanisms

One recurring mechanism is depth-anchored geometric fusion followed by learned residual completion. In dynamic-scene synthesis from a monocular moving camera, DFNet takes monocular depth $\Pi$7, MVS depth $\Pi$8, and RGB image $\Pi$9, and outputs a complete corrected depth

$\mathbf{D}^{r_t}$0

trained with

$\mathbf{D}^{r_t}$1

Rendering then separates static background and dynamic foreground, constructs a global background by shortest-baseline selection, warps foreground from the chosen source time, and applies a learned residual blender

$\mathbf{D}^{r_t}$2

Here coherence is achieved by fusing global metric consistency, local relative-shape preservation, and a scene-flow prior before warping (Yoon et al., 2020).

A second mechanism is packet-level joint denoising. ViewMorpher3D receives a packet of reference and target views, encodes the geometric priors with

$\mathbf{D}^{r_t}$3

adds these condition features to the image latents, stacks the conditioned latents from all views, and processes them with a 2D UNet based on SD-Turbo using full self-attention across flattened spatial dimensions of all views (Zanjani et al., 12 Jan 2026). The model’s coherence is therefore architectural: multiple targets are restored in one forward pass, C-maps and camera rays align restoration with scene geometry, and cross-view attention allows context to flow not only from reference images to targets but also among targets.

A third mechanism is warp-guided modulation of diffusion inference. WAVE computes warped target views from a single source image and monocular depth, converts them into binary support masks, and uses them to alter batch self-attention: $\mathbf{D}^{r_t}$4 It also injects low-frequency structure from warped target-view latents into the initial noise through Pose-Aware Noise Initialization, combining warped low frequencies with random high frequencies before DDIM sampling (Park et al., 30 Jun 2025). The remapping is therefore image-space, attention-level, and latent-space at once.

A fourth mechanism separates geometry-backed transfer from semantics-backed propagation. In view-consistent 3D scene editing, the previous edited view is first warped into the current viewpoint,

$\mathbf{D}^{r_t}$5

and then injected as structural guidance through residual features

$\mathbf{D}^{r_t}$6

Semantic continuity is handled separately by reference-guided attention: $\mathbf{D}^{r_t}$7 with

$\mathbf{D}^{r_t}$8

The method explicitly treats structural correspondence and semantic continuity as different cross-view cues (Li et al., 20 Apr 2026).

A fifth mechanism is purely computational. CoherentRaster keeps the interlaced light-field image fixed, but sorts each tile’s subpixels by viewpoint index and stores the permutation in a lookup table $\mathbf{D}^{r_t}$9 so that thread rank $$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$0 processes

$$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$1

During alpha blending, the thread retrieves viewpoint index $$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$2, cluster ID $$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$3, the Gaussian list range $$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$4, and blends contributions at $$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$5 (Sim et al., 6 May 2026). The remapping is a permutation of subpixel sample order rather than a transformation of image content.

4. Domain-specific realizations

In dynamic novel view synthesis, view-coherent remapping is tied to the failure of epipolar geometry on moving foregrounds. The method in “Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera” studies a deliberately difficult setting in which a handheld or moving monocular camera observes a dynamic scene, and coherent synthesis requires depths that are globally coherent across viewpoints and time rather than merely plausible in each frame (Yoon et al., 2020).

In autonomous driving, the term is tied to enhancement of already rendered novel views. ViewMorpher3D operates as a post-processing enhancer for 3D Gaussian Splatting renderings, accepts variable numbers of cameras and flexible reference/target configurations, and uses temporally adjacent or spatially overlapping views in the same packet so that target outputs become coherent across cameras and across time (Zanjani et al., 12 Jan 2026). The remapped entity is not a single source image but a jointly denoised set of rendered observations.

In text-driven 3D scene editing, the problem is formulated as a joint conditional distribution over edited views rather than a product of independent single-view edits. The framework based on FLUX Kontext edits the first view normally and then conditions each subsequent view on a projected structural cue and patch-level semantic features derived from the previous edited view, thereby remapping editing state sequentially across viewpoints (Li et al., 20 Apr 2026).

In single-image diffusion-based novel view synthesis, WAVE uses monocular depth and target poses to compute view-guided warps, then leverages those warps to guide attention manipulation and noise reinitialization. The resulting remapping is explicitly training-free and does not require additional modules beyond the pretrained diffusion model, the depth estimator, and the warping stack (Park et al., 30 Jun 2025).

In SLAM and loop closure, the relevant object is the latent map itself. “An Algorithm for the SE(3)-Transformation on Neural Implicit Maps for Remapping Functions” makes previously fused local neural implicit maps transformable under pose correction, allowing removal of stale contributions and reintegration under corrected poses without returning to the original raw points (Yuan et al., 2022).

In light-field rendering, view-coherent remapping addresses an interlaced subpixel layout in which neighboring raster-order subpixels often belong to different viewpoints. Sorting subpixels by viewpoint index restores coherence relative to how Gaussian lists are stored, thereby improving memory coalescing and cache locality during alpha blending (Sim et al., 6 May 2026).

A much earlier physical analogue appears in pupil-remapping interferometry, where a single telescope pupil is fragmented into sub-pupils, injected into single-mode optical channels, transported coherently, and recombined in a non-redundant geometry. The coherence-critical constraints are optical path-length matching, polarization behavior, and modal purity, with path-length matching tolerance stated as tens to hundreds of microns for most reasonable configurations (Tuthill et al., 2010). This suggests that the underlying notion of coherent remapping predates current neural rendering formulations and already involved preservation of inter-channel relations under geometric rearrangement.

5. Empirical evidence

The literature evaluates view-coherent remapping with heterogeneous metrics because the remapped object differs by domain: depth stability, image fidelity, cross-view consistency, loop-closure reconstruction quality, or kernel runtime. The table summarizes representative evidence.

Setting	Reported evidence	Paper
Dynamic-scene NVS	DFNet+B3W: perceptual similarity 0.15, flow error 5.3 pixels; MVS+B3W: 6.8-pixel flow error; MonoDepth+B3W: 10.8 pixels	(Yoon et al., 2020)
Multi-camera driving enhancement	nuScenes: OmniRe 17.15 / 0.645 / 0.448; DiFix3D++ 18.74 / 0.696 / 0.375; ViewMorpher3D 21.58 / 0.716 / 0.303	(Zanjani et al., 12 Jan 2026)
Light-field 3DGS rasterization	Synthetic Blender 2K, Ours w/o Remap vs Ours: blend time 10.00 ms $$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$6 4.45 ms; Mip-NeRF 360 2K: 31.07 ms $$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$7 18.28 ms	(Sim et al., 6 May 2026)
View-consistent 3D editing	DINO: GaussCtrl 0.7278, DGE 0.7501, EditSplat 0.7353, ViP3DE 0.7488, Ours 0.7768	(Li et al., 20 Apr 2026)
Single-image diffusion NVS	DTU, MegaScenes vs MegaScenes + WAVE: LPIPS-next 0.379 $$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$8 0.130, CLIPSIM-next 0.898 $$\mathbf{J} = \phi\left(\left\{\mathbf{J}^v\left(W_{r\rightarrow v}\right)\right\}_r, \mathbf{J}^{v,t}\left(W_{r_t\rightarrow v}\right); \mathcal{M}^{v}\right),$$9 0.956, angular consistency 13.04 $$\mathbf{y} = W_{r\rightarrow v}(\mathbf{x};\mathbf{D}^r,\Pi^r, \Pi^v),$$0 6.32	(Park et al., 30 Jun 2025)
Neural implicit SLAM remapping	ICL-NUIM lr-kt3 surface error: DI-Fusion 4.5 cm, Ours 1.2 cm; remapping runs around 50 Hz	(Yuan et al., 2022)

The evidence in dynamic-scene synthesis is particularly explicit about coherence: the mean optical-flow magnitude between synthesized and ground-truth views is interpreted as a measure of view invariance, with the statement that ideally it should be close to $$\mathbf{y} = W_{r\rightarrow v}(\mathbf{x};\mathbf{D}^r,\Pi^r, \Pi^v),$$1 with the perfect depth map (Yoon et al., 2020). In WAVE, consistency is also tied to recoverable camera motion: generated image sets are passed through COLMAP, and improvements in Frobenius norm, rotation angle difference, and angular consistency are treated as evidence that structural coherence supports better pose recovery (Park et al., 30 Jun 2025).

By contrast, ViewMorpher3D does not introduce a dedicated cross-view consistency metric; its evidence is partly indirect and partly qualitative, relying on PSNR, SSIM, LPIPS, packet-composition ablations, and multi-camera visual comparisons (Zanjani et al., 12 Jan 2026). CoherentRaster likewise treats remapping as a performance optimization rather than a quality-changing approximation, and the measured effect is concentrated in alpha-blending time rather than image quality (Sim et al., 6 May 2026).

6. Limitations, tradeoffs, and conceptual boundaries

The cited work repeatedly shows that coherence is conditional on the reliability of the representation being remapped. In dynamic-scene synthesis from a monocular moving camera, failure modes include neighboring view angles becoming too large, such as rotations over $$\mathbf{y} = W_{r\rightarrow v}(\mathbf{x};\mathbf{D}^r,\Pi^r, \Pi^v),$$2, highly cluttered scenes with many foreground/background depth discontinuities, poor camera calibration, weak static SfM anchors, and severe foreground-mask failure causing afterimages and object fragmentation (Yoon et al., 2020). In WAVE, large viewpoint changes, disocclusions, inaccurate monocular depth, weak geometry cues, thin structures, and repetitive structures under large viewpoint changes all reduce the usefulness of warp guidance (Park et al., 30 Jun 2025).

The literature also makes clear that view-coherent remapping is not always enforced by an explicit consistency loss. ViewMorpher3D states that there is no explicit reprojection, depth, normal, adversarial, or temporal consistency loss, and that coherence comes from packet-level latent denoising and shared attention (Zanjani et al., 12 Jan 2026). The 3D editing framework based on FLUX Kontext similarly does not define explicit cycle-consistency or global all-to-all multi-view constraints; it uses a first-order neighboring-view dependency, so coherence is local and propagated sequentially rather than globally solved (Li et al., 20 Apr 2026). A plausible implication is that these methods trade formal global guarantees for scalability and flexibility.

In latent-map remapping, the transformation is approximate rather than exact. Translation is handled by first-order linearization, the transform-encode versus encode-transform test still shows about $$\mathbf{y} = W_{r\rightarrow v}(\mathbf{x};\mathbf{D}^r,\Pi^r, \Pi^v),$$3 cm error, and the method is specific to a blockwise latent voxel map with SO(3)-equivariant local features rather than a generic exact $$\mathbf{y} = W_{r\rightarrow v}(\mathbf{x};\mathbf{D}^r,\Pi^r, \Pi^v),$$4 action on arbitrary neural fields (Yuan et al., 2022). In CoherentRaster, remapping improves repeated reads during blending but introduces scattered final writes, a tradeoff accepted because each subpixel is written once whereas Gaussian attributes are read repeatedly inside the blending loop (Sim et al., 6 May 2026).

Finally, the conceptual literature places a boundary around the term itself. The theory of remapping and navigation provides a unifying account of coherence, overlap agreement, and structure-preserving embeddings, but it does not provide a dedicated mathematical treatment of “view-coherence,” a benchmark, or an implementation for coherent remapping across views (Hartl et al., 20 Jan 2026). This suggests that view-coherent remapping remains a cross-cutting research concept whose exact operational meaning depends on whether the remapped object is depth, coordinates, latent state, edited semantic content, a Gaussian list access pattern, or a coherently transported optical field.