Projection Adjustment Overview

• Projection adjustment is a process that refines projected quantities—by eliminating nuisance variables, enforcing physical models, or regularizing optimization—to yield more stable and computationally tractable results.
• It transforms data into a reduced or geometry-aware space, enabling enhanced parameter estimation and improved inference in applications such as bundle adjustment, causal modeling, and optical calibration.
• Real-world applications include dense structure-from-motion, nullspace projection in pose estimation, and mitigating semantic drift in representation learning.

Projection adjustment denotes a family of procedures in which projected quantities are modified, constrained, or reinterpreted after a projection step in order to improve fidelity, numerical stability, statistical efficiency, or physical realism. The phrase is used heterogeneously across computer vision, optimization, causal inference, optical projection mapping, and representation learning. This suggests not a single canonical method, but a recurring pattern: a problem is first expressed in a projected, reduced, or geometry-aware space, and the subsequent adjustment step is used to eliminate nuisance variables, stabilize inference, or correct distortions that are difficult to handle in the original parameterization (Tang et al., 2018, Witte et al., 2020, Erel et al., 2023, Jeong et al., 11 Jun 2025).

1. Conceptual scope

In geometric estimation, projection adjustment often refers to modifying poses, depths, landmarks, or line parameters so that projected observations agree with image evidence. In causal inference, it refers to performing covariate adjustment after passing from the original graph to a latent projection that removes forbidden variables. In contrastive learning, it refers to modifying class embeddings through projection functions and compensating the resulting positive–negative mismatch with an explicit adjustment term. In optical systems, it can mean redirecting projector rays, recalibrating projector geometry, or reprojecting image pixels into a physically meaningful atmospheric or scene coordinate system (Demmel et al., 2021, Witte et al., 2020, Terai et al., 2021, Jeong et al., 11 Jun 2025).

Across these uses, three recurring motifs appear. First, projection is used to remove nuisance structure, as when landmarks or forbidden nodes are eliminated. Second, projection is used to impose a more faithful physical model, as in rolling-shutter line projection, projector calibration, or geospatial reprojection. Third, projection is used to regularize optimization by restricting updates to a low-dimensional, sparse, or otherwise structured space. This suggests that “adjustment” is typically the step that turns a projected representation from a formal reduction into a practically usable estimator or control mechanism.

2. Dense and initialization-free bundle adjustment

One major use of projection adjustment appears in bundle adjustment and structure-from-motion. BA-Net formulates dense structure-from-motion as a differentiable bundle-adjustment module with a feature-metric residual rather than a sparse geometric reprojection residual or a raw photometric residual. Its central residual is

$$e_{i,j}^f(\mathcal{X})=F_i(\pi(\mathbf{T}_i,d_j\mathbf{q}_j))-F_1(\mathbf{q}_j),$$

and dense depth is not optimized per pixel directly, but through learned basis depth maps,

$$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$

The resulting Levenberg–Marquardt update remains in the bundle-adjustment form, but the adjustment is performed in feature space and over a low-dimensional depth code rather than over unconstrained dense depth variables (Tang et al., 2018).

Initialization-free bundle adjustment uses a different projection logic. Power Variable Projection first eliminates landmarks through Variable Projection, reducing the first-stage problem to camera parameters, and then approximates the inverse VarPro Schur complement by a truncated power expansion,

$$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$

The same inverse-expansion idea is then lifted to a Riemannian tangent-space formulation for projective refinement. Here projection adjustment does not merely mean changing image projections; it means eliminating structure variables, solving in a reduced pose space, and using that reduced space to enlarge the convergence basin of bundle adjustment without a good initialization (Weber et al., 2024).

3. Nullspace projection, pose-only constraints, and row-dependent line projection

A second cluster of methods uses projection to remove variables directly from the optimization state. In square-root sliding-window bundle adjustment, landmarks are eliminated “by projecting the Jacobians onto the nullspace $Q_2$ of the landmark Jacobian using QR decomposition.” For each landmark track, the projected residual and Jacobian become

$$\tilde r_\ell = Q_2^\top r_\ell,\qquad \tilde J_{x,\ell}=Q_2^\top J_{x,\ell},$$

and the marginalization prior is updated directly in square-root form by QR rather than by repeated Hessian Schur complements. The method is algebraically equivalent to Schur complement in exact arithmetic, and in rank-deficient cases yields the Moore–Penrose-equivalent prior without explicitly computing a pseudoinverse (Demmel et al., 2021).

A related structure-eliminating strategy appears in pose-only adjustment for generalized multi-camera systems. There, a scene point is represented implicitly by two base observations and their associated poses, producing a pose-only geometric constraint

$$\mathcal{S}_{\mathbf{X}^W}^{(l, r)} = \left\{ s_i \mathbf{f}_i + \mathbf{v}_i = \mathbf{R}_{li}\left(\frac{\lambda_{lr}}{\theta_{lr}}\mathbf{f}_l + \mathbf{v}_l\right) + \mathbf{t}_{li} \right\}.$$

The optimization then minimizes spherical normalized direction residuals over poses alone, eliminating explicit 3D points from the parameter space while preserving projection geometry in ray form (Liang et al., 26 Apr 2026).

Rolling-shutter line bundle adjustment pushes projection adjustment into a row-dependent setting. Under the constant-velocity rolling-shutter model,

$\mathbf{P}_{v}=\mathbf{P}_{0}+v\mathbf{Q},$

a 3D straight line no longer projects to a single image line but to a polynomial curve $\boldsymbol{\iota}(u,v)=0$. RSL-BA avoids direct curve-to-curve alignment by sampling points on the observed rolling-shutter curve, computing the instantaneous projected line at the same row, and minimizing a distance residual together with a tangent consistency term. The preferred residual is

$$\mathbf{e}_1 = \mathbf{e}_{d1} + \lambda_1 \mathbf{e}_{t}.$$

This line-based projection adjustment is used not only for accuracy, but also to prevent three rolling-shutter degeneracies, including one first identified in that work (Zhang et al., 2024).

4. Optical projection mapping, projector calibration, and geospatial reprojection

In optical projection mapping, projection adjustment often refers to changing where projected light lands. One approach addresses non-uniform sampling on non-planar surfaces by redirecting rays with a phase-only spatial light modulator. The phase image is divided into blocks corresponding to projector pixels, the phase-pattern/shift relation is measured empirically and stored in a lookup table, and a global phase image is generated with $C^1$ continuity. The goal is not to add pixels, but to redistribute the available projector samples so that pixel density on the target surface becomes more uniform (Terai et al., 2021).

Projector calibration uses another form of adjustment: extracting a corrected ray geometry from a physically imperfect optical system. A compact device composed of a flat-bed scanner and pinhole-array masks directionally decomposes structured light. For each pinhole, the method extracts a chief ray passing through the optical center of the projector, treats the projector as a pinhole projector for those rays, and applies a standard camera calibration technique. The practical importance is that the device requires only a minimal working volume directly in front of the projector lens, regardless of focusing distance and aperture size, thereby avoiding the shallow-depth-of-field limitations of conventional projector calibration (Sugimoto et al., 2021).

Neural projection mapping internalizes the projector itself into a differentiable reflectance-field pipeline. The projector is modeled as an inverse pinhole camera,

$$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$0

and scene geometry, material, transmittance, projector intrinsics, projector extrinsics, gain, and gamma are optimized jointly. After training, the projector image $$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$1 becomes the optimization variable for compensation, XRAY-style appearance editing, or text-driven projection design. Projection adjustment here is literally optimization of the projected texture under a learned scene and lighting model (Erel et al., 2023).

A further geometric variant appears in ground-based sky imaging. Two geospatial reprojections convert image pixels into coordinates on the atmospheric cross-section where the field of view intersects a cloud layer: a flat-Earth model and a curved-Earth “great circle” model. In the flat approximation,

$$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$2

whereas the great-circle construction replaces this planar intersection by a chord–sagitta–arc geometry on a spherical cloud layer. The reported conclusion is that the flat reprojection is only adequate when the elevation angle is above $$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$3; below $$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$4, the discrepancy between the two reprojections is in the order of kilometers (Terrén-Serrano et al., 2021).

5. Projection adjustment in causal graphical models

In causal inference, projection adjustment has a precise graphical meaning. The problem is to estimate a total causal effect $$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$5 from observational data by adjusting for a valid covariate set $$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$6,

$$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$7

Because valid adjustment sets are not unique, the paper studies the optimal adjustment set, or $$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$8-set, and characterizes it through a projection operation on the graph itself.

The key construct is the forbidden projection

$$\mathbb{D}=\mathrm{ReLU}(\mathbf{w}^{\top}\mathbf{B}).$$9

obtained by marginalizing exactly the forbidden nodes other than $$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$0 and $$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$1. The main identity is

$$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$2

The forbidden projection preserves all information relevant to total causal effect estimation via covariate adjustment; bidirected edges between $$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$3 and $$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$4 in the projected graph characterize failure of adjustment identification; and the same projection perspective supports an optimal version of IDA and clarifies why backward selection can recover the $$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$5-set under strong assumptions (Witte et al., 2020).

6. Representation-space and generative uses

In supervised contrastive learning, projection adjustment enters through class-embedding design. ProjNCE introduces two projection functions, $$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$6 and $$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$7, and defines

$$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$8

where

$$\Delta x_p \approx - \sum_{i=0}^{m} \left(U_\lambda^{-1} W V_0^{-1} W^\top\right)^i U_\lambda^{-1} \left(b_p - W V_0^{-1} b_l\right).$$9

The projection functions generalize the class centroid used implicitly by supervised contrastive learning, while the extra term adjusts the negative-pair contribution so that the resulting objective remains a valid mutual-information lower bound. Within this framework, centroid, conditional-mean, and median projections become interchangeable design choices rather than fixed components of the loss (Jeong et al., 11 Jun 2025).

A related but distinct use appears in generative personalization. One recent formulation describes semantic collapsing problem as a failure mode in which a learned personalized concept overpowers the rest of the text prompt. The abstract reports that semantic drift is concentrated within “a specific low-dimensional subspace,” that the personalization process perturbs the embedding of the original base concept, and that “Test-time Embedding Adjustment with Adaptive Subspace Projection (AdaptSP)” uses “the stable, pre-trained embedding as an anchor,” isolates the semantic drift, and “projects it onto the identified subspace,” thereby mitigating semantic collapsing while maintaining subject identity (Nguyen et al., 8 May 2026). This suggests a latent-space analogue of geometric projection adjustment: the projected quantity is not a ray or a landmark, but the drift component of an embedding.

7. Algorithmic and optimization interpretations

Some uses of projection adjustment are primarily algorithmic. In the Projection and Rescaling framework for conic feasibility, the basic projection step is refined by maintaining each iterate as a convex combination of only a small affinely independent set of columns, using a modified Carathéodory reduction. The modified Incremental Representation Reduction procedure updates that representation in $Q_2$0, keeps support size at most $Q_2$1, and supports the limited-support Perceptron, Von Neumann, and away-step schemes. The reported complexity improvement is from $Q_2$2 in the earlier analogues to $Q_2$3 when the subspace dimension is $Q_2$4 (Gutman, 2018).

In credit-portfolio optimization, quadratic nonlinear projection is a local path-following method under CVaR. The portfolio is adjusted through infinitesimal reallocations,

$Q_2$5

subject to the quadratic adjustment budget

$Q_2$6

After first-order linearization of the objective and constraints, each local step is solved by a Lagrange multiplier system with a closed-form direction. The “projection” is therefore the constrained directional projection of local objective improvement onto an ellipsoidal feasible set defined by the adjustment budget and optional revenue, return, or risk constraints (Kim et al., 2014).

Bayesian Projection Pursuit Regression uses the phrase in yet another sense: projection directions $Q_2$7 are stochastic parameters that must themselves be adjusted. Each ridge function depends on a one-dimensional projection $Q_2$8, active coordinates are selected through $Q_2$9, and direction updates are proposed directly on the sphere with the power spherical distribution. Here projection adjustment is inferential rather than geometric: it is the posterior learning of projection directions, ridge functions, and even the number of projected components $$\tilde r_\ell = Q_2^\top r_\ell,\qquad \tilde J_{x,\ell}=Q_2^\top J_{x,\ell},$$0 by reversible-jump MCMC (Collins et al., 2022).

In markerless 3D tracking, filling-based object projection feature estimation shifts adjustment from rays to regions. Pixel-filling computes centroid and area from the filled interior, but is too sensitive to a one-pixel edge error; $$\tilde r_\ell = Q_2^\top r_\ell,\qquad \tilde J_{x,\ell}=Q_2^\top J_{x,\ell},$$1-grid-filling retains the connected-region logic while reducing sensitivity to edge miscalculations and computational cost. The adjustment step then moves the inner point toward the estimated centroid and re-centers it by ray averaging, stabilizing projection-feature estimation when the inner point is poorly located in a non-convex object projection (Quesada et al., 2012).

Taken together, these lines of work show that projection adjustment is best understood as a recurrent methodological device rather than a single algorithm. Whether the projected object is a landmark Jacobian, a forbidden subgraph, a class embedding, a projector ray, a portfolio direction, or a cloud-layer footprint, the adjustment step serves the same high-level role: it enforces consistency after projection in a space where the original problem becomes more identifiable, more stable, or more computationally tractable.