Residual Projection-Image Formulation

• Residual Projection-Image Formulation is a structural principle where discrepancies from one domain are projected into a target space for direct use in estimation, reconstruction, compression, or discrimination.
• It encompasses methodologies ranging from variable projection in optimization to backprojection in CT and super-resolution, emphasizing the use of projection operators to convert raw errors into informative signals.
• Its applications span pose estimation, imaging, eigenvalue computation, and quantization, demonstrating the versatility and impact of residual projection across diverse computational tasks.

In the literature represented here, Residual Projection-Image Formulation denotes a family of constructions in which a residual defined in one space—projection space, canonical object space, low- and high-resolution feature space, latent space, path space, or a Krylov/projection subspace—is mapped into another space where it becomes directly usable for estimation, reconstruction, compression, or discrimination. Depending on the application, the residual may be a bounding-box projection displacement, a reduced least-squares residual, a backprojected sinogram mismatch, a high-frequency image residual, a quantized coding residual, or a correlated difference between two rendered frames (Zhang et al., 2022, Chen et al., 2024, Feng et al., 2019, Liu et al., 2019, Sidky et al., 2022, Shu et al., 2022, Xu et al., 2024, Bai et al., 2021, Xiang et al., 25 May 2026, Ren et al., 2020, Palitta et al., 2016, Ravibabu, 2019, Popa, 2017).

1. Cross-domain structure

Across these works, the formulation recurrently has three parts: a residual definition, a projection or backprojection operator, and a target representation in which the projected residual is either minimized, fused, or used as an update. In variable projection, the reduced residual is obtained by orthogonal projection onto the complement of a column space; in CT reconstruction, a projection-domain residual is backprojected into image space; in super-resolution, low-resolution consistency errors are projected back to high-resolution features; in compression, a lossy image is complemented by a coded residual; and in quantization, a vector is projected onto a primary basis plus an orthogonal residual basis (Chen et al., 2024, Sidky et al., 2022, Feng et al., 2019, Bai et al., 2021, Xiang et al., 25 May 2026).

Domain	Residual	Projection–image linkage
Category-level pose (Zhang et al., 2022)	Bounding-box projection displacement residual	Canonical/box projections mapped to camera-frame pose and size
Separable nonlinear least squares (Chen et al., 2024)	$r_2(a)=(I-\Phi(a)\Phi(a)^\dagger)y$	Orthogonal projector onto the complement of $\mathrm{col}(\Phi(a))$
MRI and SR (Feng et al., 2019, Liu et al., 2019)	LR/HR feature inconsistency or high-frequency residual	Error-feedback up/down projections and residual image fusion
CT reconstruction (Sidky et al., 2022, Shu et al., 2022, Popa, 2017)	$b-Ax$ or $A^\top(g-Ax)$	Backprojection or row-wise Kaczmarz projection into image space
Matrix equations and eigensolvers (Palitta et al., 2016, Ravibabu, 2019)	Residual matrix or projected subspace residual	Projection-subspace images and reduced operators
Compression and quantization (Bai et al., 2021, Jacobellis et al., 27 May 2026, Xiang et al., 25 May 2026)	Residual image, latent residual, or orthogonal projection error	Additive reconstruction or dual-basis projection
Rendering (Xu et al., 2024)	Frame-to-frame image difference	Correlated path-space mapping with Jacobian correction
Image-set classification (Ren et al., 2020)	Related/unrelated regression residuals	Projection into a discriminant residual subspace

This repeated structure suggests that the phrase does not identify a single operator shared verbatim across fields; rather, it identifies a common design pattern in which residuals become more useful after a projection, backprojection, or correlated mapping.

2. Geometry-guided formulations in pose estimation

In category-level object pose estimation, the formulation is explicitly geometric. RBP-Pose considers a category-level 9-DoF pose consisting of $T=[R,t]\in SE(3)$ and metric size $s=(w,h,d)\in\mathbb{R}^3$, with a canonical bounding box

$$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$

For face $j$ with sign $\sigma_j\in\{+1,-1\}$ and half-extent $a_j\in\{w/2,h/2,d/2\}$, the face-wise projection operator is

$\mathrm{col}(\Phi(a))$0

and in the camera frame the real surface projection becomes

$\mathrm{col}(\Phi(a))$1

The corresponding displacement from a point to its face projection is

$\mathrm{col}(\Phi(a))$2

RBP-Pose then defines a hypothesis displacement from the shape prior,

$\mathrm{col}(\Phi(a))$3

with $\mathrm{col}(\Phi(a))$4, and a residual vector

$\mathrm{col}(\Phi(a))$5

Under the stated assumptions—unbiased prior reconstruction and box, unbiased row-normalized assignment map, unbiased $\mathrm{col}(\Phi(a))$6 and $\mathrm{col}(\Phi(a))$7, and zero-mean symmetric observation noise—the paper gives $\mathrm{col}(\Phi(a))$8 and argues that $\mathrm{col}(\Phi(a))$9 is relatively small, making the residual easier to regress than raw displacements (Zhang et al., 2022).

The network jointly predicts $b-Ax$0, $b-Ax$1, and $b-Ax$2, and couples them with geometry-aware consistency. The principal residual supervision is

$b-Ax$3

implemented in practice through Laplacian uncertainty weighting,

$b-Ax$4

$b-Ax$5

and a geometry consistency term

$b-Ax$6

The paper also gives a zero-mean regularizer

$b-Ax$7

and an optional image-plane residual variant,

$b-Ax$8

with small-residual linearization

$b-Ax$9

RBP-Pose explicitly contrasts this formulation with prior shape-prior-adaptation pipelines that estimate canonical correspondences and recover pose by Umeyama alignment. In that comparison, the residual projection formulation bypasses Umeyama alignment at test time and is reported to achieve 25 Hz inference. On REAL275, the reported results include IoU_75: 67.8%, 5°2cm: 38.2%, 5°5cm: 48.1%, and 10°5cm: 79.2%; on CAMERA25, the reported results include IoU_75: 89.0% and 10°5cm: 89.5% (Zhang et al., 2022). The formulation is therefore not merely residual regression; it is residual regression tied to explicit 3D box-face geometry.

3. Backprojection and residual-image coupling in imaging

In super-resolution and CT, the formulation takes the form of a direct loop between measurement residuals and image-domain updates. CPRN uses a shallow coupled-projection network and a deep residual network. In the shallow branch, one up-projection step is

$A^\top(g-Ax)$0

followed by feedback

$A^\top(g-Ax)$1

and a symmetric down-projection step with

$A^\top(g-Ax)$2

The deep branch learns the high-frequency residual, and the final fusion is

$A^\top(g-Ax)$3

CPRN-S further injects shallow feedback features into the deep residual path through

$A^\top(g-Ax)$4

The model is trained with an $A^\top(g-Ax)$5 reconstruction loss. Reported results include, for Brats at $A^\top(g-Ax)$6, 36.50/0.9911 for CPRN and 36.543/0.9918 for CPRN-S; at $A^\top(g-Ax)$7, CPRN-S reaches 29.445/0.9501 on Brats (Feng et al., 2019).

HBPN states the classical single-image SR measurement model

$A^\top(g-Ax)$8

and embeds iterative back-projection at the feature level. Its up-sampling back-projection block is

$A^\top(g-Ax)$9

while the down-sampling back-projection block is

$T=[R,t]\in SE(3)$0

Multiple SR-HG modules are fused by Softmax-based weighted reconstruction,

$T=[R,t]\in SE(3)$1

The paper reports Set5 $T=[R,t]\in SE(3)$2: 32.55/0.900, Urban100 $T=[R,t]\in SE(3)$3: 23.04/0.647, and NTIRE2019 Real SR validation: 33.88/0.920 for HBPN + WR (Liu et al., 2019).

RBP-DIP uses the CT forward model

$T=[R,t]\in SE(3)$4

and defines the backprojected residual

$T=[R,t]\in SE(3)$5

Rather than updating the image directly, it updates the network input,

$T=[R,t]\in SE(3)$6

with the paper writing the normalization as

$T=[R,t]\in SE(3)$7

Weights are optimized by minimizing

$T=[R,t]\in SE(3)$8

The paper notes that $T=[R,t]\in SE(3)$9 is proportional to the negative image-domain gradient of the least-squares loss. Reported few-view and limited-angle results include 24.92 dB for few-view parallel-beam, 22.87 dB for limited-angle parallel-beam, and 47.96 dB for full-view parallel-beam, each exceeding the listed ASD-POCS and DIP baselines (Shu et al., 2022).

In iterative CT with unmatched projector/backprojector pairs, the residual-image linkage becomes algorithmic rather than architectural. The projection residual is

$s=(w,h,d)\in\mathbb{R}^3$0

and BA-GMRES minimizes

$s=(w,h,d)\in\mathbb{R}^3$1

whereas AB-GMRES minimizes

$s=(w,h,d)\in\mathbb{R}^3$2

The paper emphasizes that unmatched operators break the assumptions of CGLS but do not prevent GMRES convergence in the chosen square system. In the 3D cone-beam experiment, the reported minimum image RMSE is 0.0201 at iteration 29 for $s=(w,h,d)\in\mathbb{R}^3$3 and 0.0210 at iteration 4 for $s=(w,h,d)\in\mathbb{R}^3$4, versus 0.0347 for 180-view FBP (Sidky et al., 2022).

A row-action version of the same pattern appears in Kaczmarz reconstruction. With residual sinogram

$s=(w,h,d)\in\mathbb{R}^3$5

the remotest-set rule selects

$s=(w,h,d)\in\mathbb{R}^3$6

and updates

$s=(w,h,d)\in\mathbb{R}^3$7

The cited result proves that, for underdetermined full row rank systems and under stated CT-relevant assumptions, every row index is selected at least once (Popa, 2017). In all of these cases, the essential device is the same: a projection-space discrepancy is turned into an image-space correction by a learned or analytic projection/backprojection operator.

4. Reduced residuals in optimization and projection methods

In separable nonlinear least squares, variable projection makes the residual itself a projected image. For

$s=(w,h,d)\in\mathbb{R}^3$8

the linear parameters are eliminated by

$s=(w,h,d)\in\mathbb{R}^3$9

The reduced residual and reduced objective are then

$$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$0

The paper gives the Golub–Pereyra Jacobian

$$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$1

the Kaufman approximation

$$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$2

and the exact-Hessian decomposition

$$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$3

VPLR augments the Gauss–Newton curvature by a secant-like correction

$$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$4

and solves

$$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$5

The paper states that Kaufman’s VP achieves a local rate similar to Golub–Pereyra’s VP under the stated smoothness and rank assumptions, and that VPLR improves convergence in large-residual regimes (Chen et al., 2024).

For symmetric Lyapunov and Sylvester equations, the same theme appears in the residual norm itself. With a projected approximation $$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$6 for the Lyapunov equation, the classical residual is

$$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$7

Instead of solving the reduced problem explicitly, the paper derives

$$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$8

where $$B(s)=\{Y\in\mathbb{R}^3: |\langle n_x,Y\rangle|\le w/2,\ |\langle n_y,Y\rangle|\le h/2,\ |\langle n_z,Y\rangle|\le d/2\}.$$9 and $j$0, $j$1, and $j$2 are built from the projected eigensystem and the last block coupling. For the Sylvester case with two projected operators, the residual norm becomes

$j$3

The paper’s interpretation is that, in symmetric settings, the residual depends only on low-dimensional projected images and boundary couplings, so one can avoid the $j$4 reduced solve (Palitta et al., 2016).

In eigenvalue computation, Rayleigh–Ritz and refined projection define two different images of the same trial subspace under $j$5. With $j$6 and Ritz vector $j$7, the Rayleigh–Ritz residual is

$j$8

where the Galerkin condition enforces $j$9. The refined vector instead minimizes the residual norm over $\sigma_j\in\{+1,-1\}$0,

$\sigma_j\in\{+1,-1\}$1

equivalently via the smallest singular vector of

$\sigma_j\in\{+1,-1\}$2

The cited analysis derives computable bounds for the ratio of refined to Rayleigh–Ritz residual norms,

$\sigma_j\in\{+1,-1\}$3

without computing the refined vector directly (Ravibabu, 2019). In all three settings, the projected residual is not a heuristic artifact; it is the central object through which convergence, curvature, or error is analyzed.

5. Residual projection in rendering, compression, and quantization

In re-rendering, the residual is the image difference between two scene states,

$\sigma_j\in\{+1,-1\}$4

with each image written as a path integral

$\sigma_j\in\{+1,-1\}$5

Introducing a path-space mapping $\sigma_j\in\{+1,-1\}$6 with Jacobian $\sigma_j\in\{+1,-1\}$7 yields the correlated residual formulation

$\sigma_j\in\{+1,-1\}$8

The paper decomposes path space into static and dynamic parts, traces from dynamic objects or ghost objects, and combines six dynamic-path sampling techniques with MIS. Reported gains are 2×–5× vs correlated PT and 4×–16× for material edits (Xu et al., 2024).

FRAPPE moves the formulation into learned compression. The $\sigma_j\in\{+1,-1\}$9-th channel is trained on the residual

$a_j\in\{w/2,h/2,d/2\}$0

while still analyzing the full input through projection-pursuit channels

$a_j\in\{w/2,h/2,d/2\}$1

After scale adaptation and quantization, the final reconstruction is

$a_j\in\{w/2,h/2,d/2\}$2

The per-channel loss is

$a_j\in\{w/2,h/2,d/2\}$3

with $a_j\in\{w/2,h/2,d/2\}$4. Because channels are learned in deflation order, truncation yields zero-overhead variable-rate coding. At approximately 0.1 bpp, FRAPPE-Image is reported to provide higher perceptual quality than AVIF with 47 times faster encoding, and the system is stated to be capable of real-time 1080p, 30fps CPU-only encoding (Jacobellis et al., 27 May 2026).

Near-lossless image compression expresses the same principle as a lossy image plus a quantized residual. With raw image $a_j\in\{w/2,h/2,d/2\}$5, lossy reconstruction $a_j\in\{w/2,h/2,d/2\}$6, and residual $a_j\in\{w/2,h/2,d/2\}$7, the reconstruction is

$a_j\in\{w/2,h/2,d/2\}$8

and the $a_j\in\{w/2,h/2,d/2\}$9 bound is enforced by the quantizer

$\mathrm{col}(\Phi(a))$00

which guarantees

$\mathrm{col}(\Phi(a))$01

The residual probability model is derived by bin summation,

$\mathrm{col}(\Phi(a))$02

and then bias-corrected for context mismatch. Reported Kodak rates are 1.84 bpsp at $\mathrm{col}(\Phi(a))$03, 1.38 at $\mathrm{col}(\Phi(a))$04, and 0.92 at $\mathrm{col}(\Phi(a))$05 (Bai et al., 2021).

OrpQuant formulates quantization itself as orthogonal residual projection. For a macro-block, the primary PoT basis $\mathrm{col}(\Phi(a))$06 defines

$\mathrm{col}(\Phi(a))$07

the residual basis $\mathrm{col}(\Phi(a))$08 defines

$\mathrm{col}(\Phi(a))$09

and the joint image is

$\mathrm{col}(\Phi(a))$10

The quantizer is

$\mathrm{col}(\Phi(a))$11

with

$\mathrm{col}(\Phi(a))$12

The paper frames conventional low-bit PoT quantization as a Low Angular Resolution Regime and uses an exactly orthogonal residual basis, built by Strided Dual-Exchange, to enlarge the representable directional image. Reported results include 6.10 PPL for LLaMA-2-7B at W3/A16, approximately 15 minutes full-model calibration time for LLaMA-2-7B, and a 0.35 ns critical path at 28nm (Xiang et al., 25 May 2026). These works show that the residual projection idea extends beyond reconstruction into coding theory, geometric quantization, and transport simulation.

6. Discriminant residual subspaces, assumptions, and scope

In image-set classification, residual projection becomes explicitly discriminative. DRA defines related and unrelated regression residuals

$\mathrm{col}(\Phi(a))$13

and a class score

$\mathrm{col}(\Phi(a))$14

A projection $\mathrm{col}(\Phi(a))$15 is then learned so that, after projection,

$\mathrm{col}(\Phi(a))$16

becomes more discriminative. The PE model minimizes

$\mathrm{col}(\Phi(a))$17

with generalized eigenproblem

$\mathrm{col}(\Phi(a))$18

and the TE model extends the scatter definitions to all positive and negative pairs. The nonfeasance strategy sets the unrelated group to the class complement,

$\mathrm{col}(\Phi(a))$19

to avoid selection mistakes. Reported results include 99.73 ± 0.11% on LFW-a for DRA-PE-eig, 87.55 ± 1.04% on LAG Scheme 2, and 91.49 ± 0.51% on Caltech256 for DRA-TE-exp (Ren et al., 2020).

The cited literature also makes clear that residual projection formulations are assumption-sensitive. In RBP-Pose, the near-zero-mean residual claim depends on unbiased prior reconstruction, unbiased assignment, unbiased pose and size, and zero-mean symmetric observation noise (Zhang et al., 2022). In variable projection, local convergence requires differentiability, full column rank, Lipschitz Jacobians, and positive definite reduced Hessians near the solution (Chen et al., 2024). In GMRES-based CT, the guaranteed monotonic decrease is for the residual norm of the chosen square system, not necessarily for $\mathrm{col}(\Phi(a))$20 (Sidky et al., 2022). In Kaczmarz with remotest-set control, the cited coverage theorem is proved for consistent underdetermined full row rank systems under specific initialization conditions (Popa, 2017).

Architectural versions of the formulation also have explicit scope limits. CPRN is 2D slice-based, scale-specific, and may degrade under strong noise, artifacts, or domain shift (Feng et al., 2019). HBPN assumes test degradations remain compatible with the learned down/up-sampling operators; otherwise LR-space consistency may weaken (Liu et al., 2019). RBP-DIP provides no convergence guarantees, even though the paper reports faster loss reduction and better SNR than DIP alone (Shu et al., 2022). FRAPPE notes that, at moderate-to-high rates, PSNR/SSIM lags symmetric learned codecs and tuned standards, and current variable-rate deployment stores one decoder snapshot per channel count (Jacobellis et al., 27 May 2026). OrpQuant identifies the dual-basis construction as a minimal repair, with higher-order extensions left open (Xiang et al., 25 May 2026).

Taken together, these works indicate that the residual projection-image formulation is best understood as a structural principle: define a residual where discrepancy is naturally measured, project or backproject it into the space where decisions are made, and couple that transformed residual to the governing geometry, operator algebra, or synthesis model. The exact operators differ sharply across fields, but the governing rationale is stable: residuals become more informative when expressed in the image, subspace, or latent domain that matches the task.