View-Dependent Projection (VDP)

• View-Dependent Projection (VDP) is a set of methods that condition image formation on viewing angles, scene geometry, and angular light-field coordinates.
• It underpins applications in light-field displays, neural rendering, panoramic synthesis, and point-cloud segmentation, optimizing quality and efficiency.
• The approach balances spatial and angular resolutions while navigating trade-offs in hardware constraints and rendering speed, guiding future research.

Searching arXiv for the cited papers and related uses of “View-Dependent Projection.” arXiv search: "View-Dependent Projection" View-dependent projection (VDP) denotes viewpoint-conditioned mappings in which the projection from a scene, light field, or intermediate representation to a displayed or rendered image is explicitly modulated by viewing direction, view plane, or camera-space geometry. In the display-theoretic formulation of Horstmeyer et al., view dependence is expressed in ray space, where spatial and angular light transport from thin optical elements can be analyzed within a common framework for parallax barriers and holograms (Horstmeyer et al., 2010). In later computer-vision and graphics work, the same phrase is used for learned per-plane projection rays in point-cloud segmentation, view-conditioned opacity in Gaussian splatting, view-conditioned splatting kernels, content-aware spherical-to-planar viewport generation, and optical-flow-based pseudo-3D warping for virtual viewpoints (Chen et al., 9 Jul 2025, Nowak et al., 29 Jan 2025, Ding et al., 25 May 2026, Jabar et al., 2024, Bai et al., 2023).

1. Scope of the term across research areas

Current arXiv usage spans several technically distinct mechanisms. What unifies them is that the projection is not fixed globally: it is conditioned on angular light-field coordinates, camera-space primitive attributes, user viewing direction, or per-plane point distributions.

Domain	View-dependent mechanism	Representative paper
Light-field displays	Ray-space / Wigner-space emission control	(Horstmeyer et al., 2010)
Gaussian and splatting renderers	View-conditioned opacity or 2D kernel formation	(Nowak et al., 29 Jan 2025, Ding et al., 25 May 2026)
360° viewport rendering	Viewing-direction-dependent Pannini projection and local warp	(Jabar et al., 2024)
Image-based view synthesis	Interpolated optical-flow warping for virtual viewpoints	(Bai et al., 2023)
Point-cloud segmentation	Learned per-plane projection rays	(Chen et al., 9 Jul 2025)

This distribution of meanings matters because VDP is not a single canonical algorithm. In display theory it addresses the spatial-angular degrees of freedom of emitted light; in neural rendering it modulates splat opacity or kernel shape as a function of view; in image formation for panoramic or close-up synthesis it selects or optimizes a projection conditioned on content and view direction; in 3D semantic understanding it learns 3D-to-2D mappings from local geometry.

2. Ray-space foundations and the rank-1 constraint

In the light-field formulation, a one-dimensional slice of a display is written as a light field

$L(x,\theta),$

where $x$ is spatial position on a reference plane and $\theta$ is ray direction. An equivalent two-plane parameterization uses intercepts $(x_1,x_2)$ on two planes separated by $d$, with

$$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$

Free-space propagation over distance $d$ shears the light field,

$$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$

while an amplitude mask $M(x)$ multiplies it pointwise,

$$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$

Under coherent illumination, the corresponding ray-space quantity is the Wigner distribution $x$0, with $x$1, defined from a complex transmission $x$2 by

$x$3

Propagation and mask interaction then become shear and convolution in $x$4-space (Horstmeyer et al., 2010).

A central result is that both parallax barriers and single holographic masks are rank-1 generators of view-dependent light distributions. For a two-plane amplitude system with a screen $x$5 and mask $x$6,

$x$7

and after discretization the light field is an outer product,

$x$8

so $x$9. For a fully coherent hologram with transmission $\theta$0, the mutual intensity is

$\theta$1

which is likewise rank 1; the Wigner distribution is a linear transform of $\theta$2 and inherits the same effective limitation (Horstmeyer et al., 2010).

The practical implications are explicit. A general $\theta$3 light field can have rank up to $\theta$4, but parallax barriers and single holographic masks can only generate rank-1 fields. In the barrier case,

$\theta$5

so increasing angular resolution forces down spatial resolution. Rank-1 displays produce discrete viewpoints rather than continuous parallax, exhibit banding and aliasing when the target light field has rank greater than 1, and have limited support for occlusion and focus cues. Horstmeyer et al. further show that partially coherent light escapes this restriction through the Mercer decomposition

$\theta$6

yielding

$\theta$7

so a single mask under $\theta$8-mode partial coherence can generate a rank-$\theta$9 field. The stated consequences are that angular resolution can increase by factor $(x_1,x_2)$0 without sacrificing spatial sampling, depth cues and focus variation improve, and light efficiency can rise (Horstmeyer et al., 2010).

3. View-conditioned projection in splatting and Gaussian rendering

In neural rendering, VDP appears as a per-view modification of the projected footprint or opacity of volumetric primitives. In “Learning View-Dependent Splatting Kernels,” each primitive is a 3D ellipsoid with center $(x_1,x_2)$1, scale vector $(x_1,x_2)$2, rotation quaternion $(x_1,x_2)$3, covariance $(x_1,x_2)$4, and a learnable 3D-kernel latent $(x_1,x_2)$5 with $(x_1,x_2)$6 by default. Projection to screen space uses

$(x_1,x_2)$7

while a projection network

$(x_1,x_2)$8

maps camera-space primitive attributes to a 2D-kernel latent. A decoder then outputs a radially symmetric 2D kernel as a function of Mahalanobis distance

$(x_1,x_2)$9

with $d$0 and truncation to $d$1 inside the projected ellipse. The typical architecture is a 4-layer fully connected $d$2 with hidden width 64 and leaky-ReLU, and a 3-layer fully connected $d$3 with hidden width 4 and sigmoid output; for acceleration, $d$4–$d$5 presampled radii are sufficient for interpolation (Ding et al., 25 May 2026).

The same paper trains the full differentiable splatting pipeline end-to-end with

$d$6

using a pre-training phase of $d$7K iterations, a stabilization phase of $d$8K iterations, and joint optimization to $d$9K iterations. On Mip-NeRF360, Tanks & Temples, Deep Blending, and NeRF Synthetic, the reported averages are PSNR $$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$0–$$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$1 dB for 2D splatting and $$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$2–$$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$3 dB for 3D splatting on Mip-NeRF360, SSIM $$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$4–$$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$5, and LPIPS $$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$6–$$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$7; removing $$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$8, scale, or rotation inputs to $$L(x_1,x_2)=L\bigl(x=(x_1+x_2)/2,\;\theta=(x_2-x_1)/d\bigr).$$9 degrades quality by $d$0 dB PSNR, and replacing $d$1 with a view-independent decoder increases LPIPS by $d$2–$d$3 (Ding et al., 25 May 2026).

A closely related formulation appears in VoD-3DGS, which augments each Gaussian with a symmetric positive-definite matrix $d$4. Standard 3DGS uses

$d$5

and scalar opacity

$d$6

VoD-3DGS instead defines

$d$7

and

$d$8

so the splatting weight becomes view dependent. This allows some Gaussians to be suppressed or emphasized depending on perspective, targeting specular highlights and reflections. The reported performance includes image-quality gains of up to $d$9 dB PSNR and $$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$0 SSIM relative to 3DGS, frame rates above $$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$1 FPS on a 48 GB NVIDIA L40, and a memory overhead of approximately $$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$2–$$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$3 (Nowak et al., 29 Jan 2025).

4. View-conditioned projection for panoramic and image-based synthesis

For spherical images rendered on planar displays, the projection itself may be optimized as a function of viewing direction and image content. In the Pannini-based method of “Globally and Locally Optimized Pannini Projection for High FoV Rendering of 360-degree Images,” a point with longitude $$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$4 and latitude $$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$5 is mapped by

$$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$6

$$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$7

where $$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$8 interpolates between rectilinear and stereographic style and $$L_{\rm out}(x,\theta)=L_{\rm in}(x-d\theta,\theta),$$9 is a vertical-compression factor. The global parameters $M(x)$0 are selected by grid search over $M(x)$1 with step $M(x)$2,

$M(x)$3

using $M(x)$4. Local conformality is then improved by semantic segmentation, dual meshes $M(x)$5 and $M(x)$6, and optimization of

$M(x)$7

with $M(x)$8, Adam at learning rate $M(x)$9, and 100 iterations. In a crowdsourcing study with 30 naïve observers and pairwise comparisons, the method was preferred over every benchmark in $$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$0 by $$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$1–$$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$2 of votes and in $$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$3 by $$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$4–$$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$5, and it won $$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$6 and $$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$7 against the previous best content-aware MOP/GAP in the two groups (Jabar et al., 2024).

A different view-dependent mechanism appears in “Close-up View synthesis by Interpolating Optical Flow,” where virtual viewpoints are synthesized without explicit depth. Given bidirectional flows $$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$8 and $$L_{\rm after}(x,\theta)=L_{\rm before}(x,\theta)\times M(x),\quad 0\le M(x)\le1.$$9 between source images $x$00 and $x$01, an intermediate view at parameter $x$02 uses

$x$03

or equivalently

$x$04

The warped images

$x$05

are blended with per-pixel weights. The method further assumes

$x$06

to obtain pseudo-depth from flow magnitude and introduces lens stretching

$x$07

for close-up magnification. The reported evaluation cites PSNR, SSIM, LPIPS, and a custom distortion error, with improvements of $x$08–$x$09 dB PSNR, SSIM gains of $x$10–$x$11, and perceptual-distortion reduction of $x$12–$x$13 relative to a naïve Street-View-style zoom (Bai et al., 2023).

5. Learned per-plane VDP for point-cloud segmentation

In PointVDP, VDP is defined explicitly as a learned alternative to view-independent projection (VIP) for point-cloud segmentation. VIP uses fixed straight or “mirage” rays with hand-chosen parameters, often requiring multiple rendered images per plane. PointVDP instead learns a small set of per-plane, data-driven rays from the local 3D point distribution and applies them once to obtain a single informative 2D image per plane (Chen et al., 9 Jul 2025).

For a point $x$14 in plane $x$15, the learned projection is

$x$16

where $x$17 and $x$18 are predicted distortion coefficients. The framework first selects the plane-specific subset $x$19, performs farthest-point sampling and BallQuery-based aggregation, partitions pooled features into $x$20 and $x$21, applies self-attention and cross-attention, fuses them with weight $x$22, and predicts $x$23 through MLP and pooling. A mutation operator then perturbs the coefficients with a lightweight Gaussian rule and clamps them within $x$24 (Chen et al., 9 Jul 2025).

The paper also introduces color regularization through a space-utilization term

$x$25

and a Gaussian penalty

$x$26

summed across planes as $x$27, with the full loss

$x$28

The efficiency claim is explicit: VIP often uses 3 preset $x$29 values per plane across 4 planes, giving 12 images, whereas PointVDP uses exactly 1 learned image per plane across 4 planes, giving 4 images and a 66.7% reduction in 2D forwards. On S3DIS with Swin-T, training time drops from approximately 5,400 min to approximately 3,720 min and inference from approximately 322 ms/scene to approximately 184 ms/scene; on S3DIS Area 5, mIoU rises from $x$30 to $x$31 with Swin-T and from $x$32 to $x$33 with SG-Former-S, while on ScanNet validation it rises from $x$34 to $x$35 with Swin-T and from $x$36 to $x$37 with SG-Former-S (Chen et al., 9 Jul 2025).

6. Common constraints, misconceptions, and research directions

A recurring misconception is that view dependence automatically implies unrestricted control over the projected image. The ray-space analysis shows the opposite: canonical parallax barriers and single holographic masks remain rank-1 systems unless partial coherence introduces multiple modes, so view dependence alone does not remove the spatial-angular degree-of-freedom bottleneck (Horstmeyer et al., 2010).

A second misconception is that adding view-conditioned parameters always improves fidelity without trade-offs. In VoD-3DGS, scenes with almost purely diffuse content can see a slight drop in accuracy, rendering speed falls from the $x$38 FPS regime to $x$39–$x$40 FPS, and memory grows by $x$41–$x$42 (Nowak et al., 29 Jan 2025). In learned splatting kernels, the quality gains depend on supplying camera-space center, scale, and rotation to the projection network; ablations that remove these inputs degrade PSNR by $x$43 dB, indicating that the learned projection is not merely a generic smoothing prior (Ding et al., 25 May 2026). In optical-flow-based close-up synthesis, the method is highly dependent on accurate flow, cannot handle very large viewpoint changes, and requires additional checks for dynamic scenes (Bai et al., 2023).

These results suggest that VDP is best understood as a family of conditional projection operators whose benefit depends on how effectively the conditioning variable captures the latent cause of appearance change: angular light transport in displays, viewing direction in specular rendering, local geometric distribution in point projection, or flow-derived parallax in image-based synthesis. A plausible implication is that future work will continue to hybridize these views, combining explicit geometry, learned conditioning, and controlled coherence or regularization rather than treating projection as a fixed camera model.