Papers
Topics
Authors
Recent
Search
2000 character limit reached

Pose-Informed Extrapolative View Synthesis

Updated 14 July 2026
  • Pose-informed extrapolative view synthesis is a technique that leverages explicit camera pose conditioning to generate views beyond the support of observed images.
  • Methods employ diverse pose representations and conditioning mechanisms—from explicit coordinates to geometric projections—integrated with diffusion and inpainting.
  • Empirical evaluations demonstrate enhanced pose recovery and view synthesis quality in sparse-view scenarios, despite challenges in occlusion handling and calibration.

Searching arXiv for the primary and closely related papers to ground the article in current literature. Pose-informed extrapolative view synthesis denotes a class of novel-view-synthesis methods in which camera pose is a primary conditioning variable and the target view may lie outside the support of the observed images, so that synthesis must account for occluded regions, out-of-field-of-view content, or weak geometric support rather than only interpolate between nearby observations. In current literature, the topic spans object-centric generation, scene-level sparse-view synthesis, articulated human rendering, and driving-data conversion; across these settings, pose enters as explicit camera parameters, projected geometry, ray embeddings, canonicalization transforms, or articulated body controls (Kwak et al., 13 Jun 2025, Yu et al., 2023).

1. Conceptual scope and problem boundaries

A useful formalization of extrapolation appears in work on aligned image-and-geometry generation, where a target pose πt\pi_t is called extrapolative when it cannot be written as a convex combination of the reference camera poses, namely when

πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 1

is impossible (Kwak et al., 13 Jun 2025). In that regime, visible support from the input views is incomplete by construction, and the synthesis problem becomes partly generative rather than purely reconstructive.

The literature distinguishes several neighboring but non-identical problem formulations. Some methods are best described as view-synthesis-assisted pose estimation rather than pure view synthesis: Zero123-6D uses pose-conditioned novel-view generation to densify a reference bank and then estimates a category-level RGB 6D pose by feature matching and geometric refinement (Felice et al., 2024). Others are best understood as pose-aware support methods for sparse-view NVS: PoseProbe does not focus on arbitrary free-viewpoint rendering itself, but makes few-shot NeRF reconstruction feasible from 3 ⁣ ⁣63\!\sim\!6 unposed images by using a rigid scene object as a “pose probe” (Gao et al., 2024). Still others are directly aimed at extrapolative synthesis, such as long-horizon diffusion over camera trajectories (Yu et al., 2023), joint image-and-geometry inpainting from sparse views (Kwak et al., 13 Jun 2025), or geometry-conditioned virtual-camera generation for autonomous driving (Lan et al., 8 Apr 2026).

The boundary of the topic is equally important. “Multi-View Image-to-Image Translation Supervised by 3D Pose” learns view-specific person translators coupled by a 3D pose loss, but it is restricted to a fixed discrete set of trained, calibrated camera views and therefore is not arbitrary novel-view synthesis in the usual sense (Diamant et al., 2021). Conversely, UpFusion is explicitly framed as a pose-free alternative: it still requires a query viewpoint π\pi, but it avoids using input-view camera poses at inference and learns to fuse unposed observations implicitly inside a diffusion model (Kani et al., 2023). This contrast clarifies that “pose-informed” need not mean “all camera poses are externally known,” but it does require that pose remains structurally central to synthesis.

2. Pose representations and conditioning mechanisms

A major axis of variation in the field is how pose is parameterized and injected. Object-centric methods often adopt explicit viewpoint parameterizations. Zero123-6D represents camera location in spherical coordinates around an object-centered frame as

(θ,ϕ,r),(\theta, \phi, r),

with relative viewpoint changes encoded by

(Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),

and uses these variables to condition EscherNet for reference-view extrapolation (Felice et al., 2024). Real-time single-image synthesis in PLFNet+ uses only translation coordinates Pt=(x,y,z)P_t=(x,y,z), normalizes them, applies a NeRF-style sinusoidal embedding, maps the result through an MLP, and reshapes the output into a dense pose feature map that conditions a dual-encoder CNN (Gond et al., 2024). P2I-NET, by contrast, learns a scene-specific direct mapping from a 7D camera pose vector to an image, with pose entering both the generator and a projection-style discriminator (Kang et al., 2023).

Scene-centric generative models increasingly avoid raw pose tokens and instead encode pose geometrically. Long-Term Photometric Consistent Novel View Synthesis conditions on camera intrinsics K\mathbf K, extrinsics c=[Rt]c=[\mathbf R \mid \mathbf t], and dense ray embeddings

ru,v=[du,v,τ],r_{u,v} = [d_{u,v}, \boldsymbol{\tau}],

with frequency encoding injected into cross-attention between conditioning and target streams (Yu et al., 2023). MoAI replaces direct pose-token conditioning by target-view geometric projection: off-the-shelf pointmaps are merged into a point cloud πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 10, projected to the target pose through

πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 11

and then used as the basis for joint image and geometry inpainting (Kwak et al., 13 Jun 2025). NVComposer likewise uses dense per-pixel Plücker ray embeddings πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 12 and camera-to-world matrices πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 13 inside a dual-stream diffusion model that jointly predicts target images and missing condition-view poses (Li et al., 2024). OpenLongTail pushes this idea further by constructing token-wise Plücker line features

πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 14

and broadcasting learned geometry embeddings into the main DiT stream, control branch, and cross-view memory bank (Liu et al., 10 Jul 2026).

Human-specific pipelines use articulated pose rather than camera pose as the dominant structural variable. Neural Actor conditions a canonical radiance field on SMPL pose and inverse skinning, with the posed-to-canonical transformation

πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 15

while view direction remains a separate rendering input (Liu et al., 2021). Human Pose Manipulation and Novel View Synthesis using Differentiable Rendering uses 3D joints, transforms them across cameras by

πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 16

and converts limb segments into anisotropic Gaussian primitives rendered into a latent image (Rochette et al., 2021). Older single-image human view extrapolation instead predicts a depth map, analytically derives forward flow by perspective projection, and learns only the residual flow and mask completion (Zhu et al., 2018).

A distinct line of work argues that the pose representation itself should be learned. “Learning Neural Representation of Camera Pose with Matrix Representation of Pose Shift via View Synthesis” encodes each scalar pose coordinate into a high-dimensional latent vector and models local motion by a skew-symmetric matrix πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 17, so that finite pose shifts act through

πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 18

This is not a full geometric renderer, but it explicitly treats camera motion as a structured latent action rather than a raw coordinate concatenation (Zhu et al., 2021).

3. Methodological families

The field now contains several recurring design patterns.

Family Representative methods Pose role
Geometry-grounded projection and inpainting Geo-EVS, MoAI, OpenLongTail Target pose defines reprojection support, holes, and visibility (Lan et al., 8 Apr 2026, Kwak et al., 13 Jun 2025, Liu et al., 10 Jul 2026)
Joint pose-and-image generation NVComposer, PoseCrafter Missing poses or trajectories are inferred inside a generative pipeline (Li et al., 2024, Mao et al., 22 Oct 2025)
View-synthesis-assisted downstream geometry Zero123-6D, PoseProbe Synthesized or probe-guided views support pose estimation or few-view reconstruction (Felice et al., 2024, Gao et al., 2024)
Canonical articulated rendering Neural Actor, differentiable human Gaussian rendering, unsupervised reference-pose canonicalization Pose controls deformation, primitive placement, or canonicalization (Liu et al., 2021, Rochette et al., 2021, Liu et al., 2021)

Within this taxonomy, diffusion-based scene synthesis has split into two broad strategies. One strategy uses autoregressive pose-conditioned generation, as in the long-horizon latent diffusion model that approximates

πt=n=1Nαnπnwithαn0 and n=1Nαn=1\pi_t = \sum_{n=1}^{N} \alpha_n \pi_n \quad \text{with} \quad \alpha_n \geq 0 \text{ and } \sum_{n=1}^{N} \alpha_n = 19

and relies on pose-augmented cross-attention to preserve visible content while hallucinating unseen regions (Yu et al., 2023). Another strategy uses warping-and-inpainting under explicit geometric projection, as in MoAI, where projected pointmaps or mesh renderings define the observed target support and the diffusion model fills the remainder (Kwak et al., 13 Jun 2025). Geo-EVS follows the same high-level principle for driving, but its condition maps are sparse reprojected colored point clouds

3 ⁣ ⁣63\!\sim\!60

and training deliberately corrupts them with reprojection-derived artifact masks to mimic out-of-trajectory defects (Lan et al., 8 Apr 2026).

A separate trend couples pose inference to synthesis rather than treating calibration as external preprocessing. NVComposer constructs a masked sequence of image-pose bundles

3 ⁣ ⁣63\!\sim\!61

where condition-view poses are hidden and target-view images are hidden, and trains a dual-stream denoiser to recover both jointly (Li et al., 2024). PoseCrafter is not a renderer in the same sense, but its Hybrid Video Generation combines DynamiCrafter with ViewCrafter to synthesize intermediate views along an inferred camera trajectory, then selects frames with the highest endpoint matchability for final pose estimation (Mao et al., 22 Oct 2025). This suggests a broader methodological point: in sparse-overlap regimes, synthesized views can serve as geometric bridges even when their primary purpose is not final rendering.

Feed-forward pose-to-image models occupy another corner of the design space. P2I-NET learns 3 ⁣ ⁣63\!\sim\!62 for scene-specific indoor environments, using a 7D pose input and auxiliary discriminator constraints in latent feature space and pose space (Kang et al., 2023). PLFNet+ similarly avoids explicit geometry, warping, or depth and instead maps target translation coordinates into a learned spatial conditioning field, reaching real-time rates while remaining limited to position-only control (Gond et al., 2024). These methods show that pose-informed synthesis need not always be mediated by an explicit 3D scene scaffold, though the literature repeatedly reports that geometry becomes more important as extrapolation becomes more severe.

4. Pose recovery and alignment under sparse or unposed inputs

A defining challenge in contemporary pose-informed synthesis is that the conditioning images are often unposed or only weakly posed. PoseProbe addresses this by turning a rigid object visible in all images into a calibration surrogate. A probe is segmented with SAM, initialized from a cube represented as an SDF, and used to recover initial poses via PnP on 2D–3D correspondences derived from the current object geometry. The resulting object and scene branches share camera poses and are jointly optimized, which makes the method particularly effective in feature-sparse, large-baseline, and few-view settings where COLMAP is unstable (Gao et al., 2024). On a sparse 3-view split, the paper reports that standard COLMAP has 3 ⁣ ⁣63\!\sim\!63 success in recovering all camera poses with only 3 ⁣ ⁣63\!\sim\!64 matches, while the probe-based PnP approach succeeds on 3 ⁣ ⁣63\!\sim\!65 of cases with 3 ⁣ ⁣63\!\sim\!66 matches and remains at 3 ⁣ ⁣63\!\sim\!67 success even when only 3 ⁣ ⁣63\!\sim\!68 of matches are used (Gao et al., 2024).

NVComposer internalizes this alignment problem. Rather than requiring external multi-view registration, it hides the condition-view poses and trains the denoiser to infer them jointly with target-view images. On RealEstate10K, its pose estimates from generated Plücker rays substantially outperform DUSt3R in both easy and hard settings: on the hard subset, DUSt3R reports 3 ⁣ ⁣63\!\sim\!69 and π\pi0, while NVComposer reports π\pi1 and π\pi2 (Li et al., 2024). The hard split is defined by conditioning-view angular separation π\pi3 and target-view separation π\pi4, so this is specifically evidence for pose recovery under weak overlap (Li et al., 2024).

UpFusion represents the opposite decision: it avoids explicit input-pose usage entirely and instead treats multiple unposed images as context for a transformer-plus-diffusion model that learns to infer query-aligned features and scene tokens implicitly (Kani et al., 2023). The method still needs a target viewpoint π\pi5, but it does not require input extrinsics at inference. Empirically, this allows it to outperform pose-dependent SparseFusion when the latter relies on predicted rather than ground-truth camera poses, while still trailing the same baseline under perfect poses (Kani et al., 2023). This is an instructive contrast: accurate poses remain highly valuable, but noisy external calibration can be worse than a strong pose-free fusion prior.

Driving-oriented pipelines add another layer: they must recover a temporally coherent ego trajectory before any extrapolative view synthesis can be meaningful. OpenLongTail estimates a metric-scale monocular trajectory π\pi6 using MapAnything and then stabilizes it with Kalman filtering and Rauch–Tung–Striebel smoothing,

π\pi7

thereby reducing jitter while preserving metric ATE (Liu et al., 10 Jul 2026). Geo-EVS assumes calibrated target poses but addresses the same deployment issue from the rendering side: because no dense RGB supervision exists at extrapolated virtual views, it uses the same projection operator at training and inference and treats the shift from observed poses to virtual poses as a condition-distribution problem rather than a missing-label problem (Lan et al., 8 Apr 2026).

5. Empirical behavior, benchmarks, and operating regimes

The empirical record supports a consistent conclusion: pose conditioning is most valuable when viewpoint support is sparse, baselines are wide, or calibration is unreliable. In category-level RGB 6D pose estimation, Zero123-6D shows that augmenting a sparse reference bank with synthesized views materially improves downstream pose retrieval and refinement on CO3D. Averaged over five categories, with 5 reference views, median rotation error decreases from π\pi8 to π\pi9, accuracy@15 increases from (θ,ϕ,r),(\theta, \phi, r),0 to (θ,ϕ,r),(\theta, \phi, r),1, and accuracy@30 increases from (θ,ϕ,r),(\theta, \phi, r),2 to (θ,ϕ,r),(\theta, \phi, r),3. In the 1-view regime, median error improves from (θ,ϕ,r),(\theta, \phi, r),4 to (θ,ϕ,r),(\theta, \phi, r),5, accuracy@15 from (θ,ϕ,r),(\theta, \phi, r),6 to (θ,ϕ,r),(\theta, \phi, r),7, and accuracy@30 from (θ,ϕ,r),(\theta, \phi, r),8 to (θ,ϕ,r),(\theta, \phi, r),9 (Felice et al., 2024). These results isolate a concrete use case for extrapolative synthesis: densifying viewpoint support when reference acquisition is minimal.

For sparse-view scene synthesis, MoAI provides direct evidence that explicit target-view geometric projection plus diffusion inpainting is especially effective under extrapolation. On DTU, in the 2-view extrapolative setting, it reports PSNR/SSIM/LPIPS of (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),0, compared with PixelSplat (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),1, MVSplat (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),2, and NoPoSplat (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),3. On RealEstate10K extrapolation with 2 views, it reports (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),4 PSNR, (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),5 SSIM, and (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),6 LPIPS, outperforming PixelSplat, MVSplat, and NoPoSplat (Kwak et al., 13 Jun 2025). Its geometry metrics further separate “Recon” and “Inpainting” regions, showing directly that geometry completion in the unseen parts of the target view remains the harder subproblem (Kwak et al., 13 Jun 2025).

NVComposer addresses a related regime but without external alignment. On RealEstate10K hard, its advantage grows with the number of unposed inputs: with 4 views it reaches PSNR (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),7, SSIM (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),8, LPIPS (Δθ,Δϕ,Δr)=(θ2θ1, ϕ2ϕ1, r2r1),(\Delta \theta, \Delta \phi, \Delta r) = (\theta_2-\theta_1,\ \phi_2-\phi_1,\ r_2-r_1),9, and DISTS Pt=(x,y,z)P_t=(x,y,z)0, versus ViewCrafter Pt=(x,y,z)P_t=(x,y,z)1 (Li et al., 2024). The trend is notable because ViewCrafter does not improve monotonically with additional views in the hard subset, whereas NVComposer does, which supports the paper’s claim that external alignment can become a liability when overlap is weak (Li et al., 2024).

Driving benchmarks make the role of pose-conditioned geometry especially explicit. Geo-EVS evaluates extrapolated target views under LiDAR-Projected Sparse-Reference supervision and reports sparse-PSNR Pt=(x,y,z)P_t=(x,y,z)2 and sparse-SSIM Pt=(x,y,z)P_t=(x,y,z)3, exceeding 3DGS, EmerNeRF, Street Gaussians, and FreeVS (Lan et al., 8 Apr 2026). OpenLongTail measures cross-view geometric consistency with GeoKPM and reports a mean of Pt=(x,y,z)P_t=(x,y,z)4, compared with TrajectoryCrafter Pt=(x,y,z)P_t=(x,y,z)5, Gen3C Pt=(x,y,z)P_t=(x,y,z)6, ReCamMaster Pt=(x,y,z)P_t=(x,y,z)7, and Vista4D Pt=(x,y,z)P_t=(x,y,z)8 on unseen-scene extrapolative synthesis (Liu et al., 10 Jul 2026). It also shows a downstream effect: fine-tuning Alpamayo R1 with synthesized long-tail data improves average AlpaSim Score from Pt=(x,y,z)P_t=(x,y,z)9 to K\mathbf K0 with K\mathbf K1 collision rate, approaching the K\mathbf K2 achieved with ground-truth multi-view data (Liu et al., 10 Jul 2026).

Human rendering results show a different pattern: explicit pose and body structure primarily improve perceptual realism and controllability. Neural Actor reports, on sequence K\mathbf K3 novel-view synthesis for training poses, PSNR K\mathbf K4, SSIM K\mathbf K5, LPIPS K\mathbf K6, and FID K\mathbf K7, outperforming NeRF+pose, Neural Volumes, Neural Body, and NHR in FID and matching or exceeding them in other metrics (Liu et al., 2021). On novel-pose synthesis across eight sequences, it repeatedly achieves much lower FID than Neural Body while remaining competitive in PSNR, which the paper interprets as better sharpness and reduced blur under unseen poses (Liu et al., 2021). Earlier single-image human view extrapolation reports best image MSE K\mathbf K8, SSIM K\mathbf K9, flow MSE c=[Rt]c=[\mathbf R \mid \mathbf t]0, flow c=[Rt]c=[\mathbf R \mid \mathbf t]1, flow NCC c=[Rt]c=[\mathbf R \mid \mathbf t]2, and mask IoU c=[Rt]c=[\mathbf R \mid \mathbf t]3 on its synthetic human benchmark, showing that explicit geometry helps most when articulation makes direct appearance-flow prediction unstable (Zhu et al., 2018).

Efficiency results define yet another operating regime. PLFNet+ reports c=[Rt]c=[\mathbf R \mid \mathbf t]4 s per frame at c=[Rt]c=[\mathbf R \mid \mathbf t]5 and c=[Rt]c=[\mathbf R \mid \mathbf t]6 s per frame at c=[Rt]c=[\mathbf R \mid \mathbf t]7, corresponding to about 135 FPS and 116 FPS on an RTX 2070 Super, while remaining competitive on synthetic translational view synthesis (Gond et al., 2024). P2I-NET reports scene-specific inference at c=[Rt]c=[\mathbf R \mid \mathbf t]8 FPS on CP2Vc=[Rt]c=[\mathbf R \mid \mathbf t]9 and ru,v=[du,v,τ],r_{u,v} = [d_{u,v}, \boldsymbol{\tau}],0 FPS on 7 Scenes, while outperforming several NeRF-family baselines in PSNR and LPIPS on CP2Vru,v=[du,v,τ],r_{u,v} = [d_{u,v}, \boldsymbol{\tau}],1 (Kang et al., 2023). These results do not invalidate geometry-heavy methods, but they show that explicit pose conditioning can also support extremely fast feed-forward renderers when the domain is sufficiently constrained.

6. Limitations, misconceptions, and likely directions

A recurrent misconception is that pose conditioning alone solves extrapolation. The literature consistently argues otherwise. Zero123-6D still needs NeuS reconstruction and online reprojection-based refinement because synthesized category-level reference views only provide a coarse pose and cannot resolve intra-category geometry differences by themselves (Felice et al., 2024). MoAI finds that even joint image-and-geometry generation benefits from cross-modal attention transfer and mesh-based conditioning because shared conditioning alone does not guarantee image-geometry alignment (Kwak et al., 13 Jun 2025). Geo-EVS explicitly attributes failure under out-of-trajectory synthesis to weak geometric support and trains the diffusion model on reprojection-derived artifact masks precisely because clean pose-conditioned inputs are not representative of extrapolated inference conditions (Lan et al., 8 Apr 2026). This suggests that extrapolation is best treated as a joint geometry-and-generation problem rather than as pose-token control alone.

A second misconception is that “pose-free” means “pose-agnostic.” UpFusion does not require input-view poses, but it still models

ru,v=[du,v,τ],r_{u,v} = [d_{u,v}, \boldsymbol{\tau}],2

so a query viewpoint remains essential (Kani et al., 2023). MoAI is “pose-free” only in the sense of not requiring user-supplied reference poses; it still depends critically on inferred reference camera poses and an explicit target pose ru,v=[du,v,τ],r_{u,v} = [d_{u,v}, \boldsymbol{\tau}],3 for projection (Kwak et al., 13 Jun 2025). NVComposer similarly removes external alignment for condition images, but target camera poses remain mandatory (Li et al., 2024). The deeper distinction is therefore between externally supplied calibration and internally inferred pose structure, not between “using pose” and “not using pose.”

The principal limitations are also consistent across domains. Object-centric pipelines often rely on object-centered inputs and may weaken in cluttered scenes or under strong symmetry (Felice et al., 2024). Human pipelines remain limited by body-model topology, hand tracking, and unseen-side appearance; Neural Actor explicitly notes difficulty with loose garments such as skirts and degraded behavior for very extreme or totally different poses, while the differentiable Gaussian-rendering pipeline cannot fully generalize to unseen identities without fine-tuning (Liu et al., 2021, Rochette et al., 2021). Single-image human view extrapolation becomes implausible for rotations larger than ru,v=[du,v,τ],r_{u,v} = [d_{u,v}, \boldsymbol{\tau}],4 (Zhu et al., 2018). Diffusion-based scene and driving systems incur substantial inference cost and may still produce temporal artifacts or geometry drift over long horizons (Mao et al., 22 Oct 2025, Liu et al., 10 Jul 2026). PoseCrafter additionally notes failure under severe illumination change and in low-texture or repetitive-texture scenes, where feature-based frame selection becomes unreliable (Mao et al., 22 Oct 2025).

A plausible implication is that the field is converging toward hybrids that separate responsibilities more cleanly. Several papers already instantiate this pattern: generation for coverage, geometry for refinement in Zero123-6D (Felice et al., 2024); dual object/scene optimization in PoseProbe (Gao et al., 2024); projected-geometry conditioning plus diffusion completion in MoAI and Geo-EVS (Kwak et al., 13 Jun 2025, Lan et al., 8 Apr 2026); and articulated canonicalization plus local appearance latents in Neural Actor (Liu et al., 2021). This suggests that future pose-informed extrapolative view synthesis systems will likely combine internal pose inference, explicit target-view geometry, and generative completion, rather than relying exclusively on any one of these components.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (17)

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Pose-Informed Extrapolative View Synthesis.