Residual Projection Neural Network (RPNN)

• Residual Projection Neural Network (RPNN) is a conceptual framework that integrates projection operators to enforce structured consistency while using residual corrections to compensate for unresolved dynamics.
• It is applied across domains such as reduced-order modeling, CT reconstruction, image enhancement, MRI super-resolution, and machine unlearning, each utilizing tailored projection and residual strategies.
• Key methods like ResNet-ROM, RBP-DIP, BPP, CPRN, and RRNN demonstrate improved accuracy and stability by coupling classical projection techniques with neural residual learning.

Searching arXiv for recent and relevant papers on “Residual Projection Neural Network” and closely related terminology. Residual Projection Neural Network (RPNN) does not denote a single standardized architecture in the arXiv literature surveyed here. Instead, the label is best understood as a family resemblance term for methods that combine some form of projection or back-projection with residual learning or residual correction. In different domains, the “projection” component refers to classical Galerkin/POD reduced-order projection, CT forward/backprojection, learned cross-resolution projection operators, or low-rank projection in parameter space; the “residual” component refers variously to ResNet-style skip dynamics, additive closure correction, iterative residual backprojection, or stagewise residual fitting. A further complication is acronym collision: one 2023 path-planning paper uses RPNN to mean Region Prediction Neural Network, not residual projection neural network (Xie et al., 2019, Shu et al., 2022, Michelini et al., 2021, Feng et al., 2019, Andrecut, 2024, Cao et al., 2022, Huang et al., 2023).

1. Terminological scope and representative usages

The surveyed literature does not present a canonical paper titled “Residual Projection Neural Network.” Instead, closely related ideas appear under domain-specific names. This suggests that RPNN is more useful as an interpretive descriptor than as a fixed historical term.

Domain	Official term	Projection/residual interpretation
Nonlinear ROM	ResNet-ROM	Galerkin/POD projection plus residual neural closure
CT reconstruction	RBP-DIP	Explicit residual backprojection guiding an untrained U-net
Image enhancement	Back-Projection Pipeline	Multi-resolution learned projection/back-projection with residual updates
MRI super-resolution	Coupled-Projection Residual Network	Coupled up/down projection plus residual high-frequency refinement
Random-feature classification	Residual Random Neural Networks	Stagewise residual fitting via least-squares projections
Path planning	Region Prediction Neural Network	Acronym collision; not “Residual Projection”

A central distinction across these works is the meaning of “projection.” In reduced-order modeling, projection is classical ROM projection onto a reduced basis. In CT, it is the physical forward operator and its adjoint. In super-resolution and enhancement, it is cross-scale down/up mapping. In residual random networks and machine unlearning, it is projection in the least-squares or subspace sense. A plausible implication is that any encyclopedia treatment of RPNN must foreground these semantic differences rather than assume a single architectural template.

2. Projection-based reduced-order dynamics with residual neural closure

A particularly direct match to an RPNN-like concept is the 2019 closure-learning framework for nonlinear model reduction, whose official name is ResNet-ROM. The method starts from a high-fidelity nonlinear PDE, introduces a reduced basis $\{\varphi_1,\dots,\varphi_r\}$, and uses Galerkin projection to obtain the standard GP-ROM

$\dot a = A a + a^\top B a.$

The paper’s core claim is that this projection-based ROM becomes inadequate for nonlinear convection-dominated systems when the basis is aggressively truncated, because the retained modes do not capture the interaction between resolved and discarded modes. That missing interaction is encoded as a closure term $\tau$, yielding the closure-augmented system

$\dot a = A a + a^\top B a + \tau.$

The proposed hybrid model then replaces $\tau$ by a learned residual neural correction,

$$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$

with the network input explicitly defined as

$X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$

and the network output approximating the closure term $\tau$ (Xie et al., 2019).

The residual network is a simplified fully connected ResNet with propagation rule

$$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$

Its training objective is

$\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$

The paper emphasizes that data is used only to complement classical physical modeling, not to replace it. In this formulation, the “projection” is entirely classical: Galerkin/POD-ROM projection is used both to derive the reduced dynamics and to define the filtered closure discrepancy. The “residual” is standard residual learning in the ResNet sense. This directly supports the interpretation of the method as a projection-based reduced-order model with residual neural closure rather than a network whose internal layers perform projection (Xie et al., 2019).

The workflow is explicitly offline/online. Offline, high-fidelity simulations generate snapshots, a reduced basis is constructed, reduced coordinates $\dot a = A a + a^\top B a.$0 are obtained, closure labels are computed from ROM-projection filtering, and a ResNet is trained. Online, the closed reduced ODE is evolved using the learned closure. The benchmark is the 1D Burgers equation on $\dot a = A a + a^\top B a.$1, $\dot a = A a + a^\top B a.$2, with finite element spatial discretization, a uniform mesh with $\dot a = A a + a^\top B a.$3 grid points, forward Euler time discretization, $\dot a = A a + a^\top B a.$4, $\dot a = A a + a^\top B a.$5 snapshots, ROM basis size $\dot a = A a + a^\top B a.$6, a 6-block ResNet, and Adam. The paper distinguishes a reconstructive regime—training and testing at the same $\dot a = A a + a^\top B a.$7—from a predictive regime, where training uses $\dot a = A a + a^\top B a.$8 and testing uses $\dot a = A a + a^\top B a.$9. In both regimes, the ResNet-ROM is reported as more accurate than GP-ROM, with much smaller spurious oscillations. Against several closure baselines, the reported $\tau$0 errors are $\tau$1 for ResNet-ROM / CL-ROM, $\tau$2 for POD-AV, $\tau$3 for POD-L, $\tau$4 for EF-ROM, $\tau$5 for AD-ROM, and $\tau$6 for DDF-ROM (Xie et al., 2019).

Within the present topic, this paper is important because it makes the projection-residual decomposition unusually explicit. Projection provides the physically anchored low-dimensional skeleton; the residual network supplies only the unresolved correction. A common misconception would be to read “projection” here as a learned projection layer inside the neural architecture. The paper explicitly does not do that.

3. Residual backprojection as an iterative neural reconstruction mechanism

In CT reconstruction, the 2022 RBP-DIP framework offers another strong RPNN-like instantiation, but in a different sense. The setting is the discrete forward model

$\tau$7

and the reconstruction is constrained to lie in the range of an untrained U-net $\tau$8 by solving

$\tau$9

The key innovation is that the network input $\dot a = A a + a^\top B a + \tau.$0 is not fixed. Instead, it is updated at each iteration through residual back projection, using the back-projected data inconsistency

$\dot a = A a + a^\top B a + \tau.$1

Algorithmically, the update is written as

$\dot a = A a + a^\top B a + \tau.$2

This is a residual-projection-guided neural procedure in which the physics operator $\dot a = A a + a^\top B a + \tau.$3 and its adjoint $\dot a = A a + a^\top B a + \tau.$4 form an explicit correction loop external to the trainable graph (Shu et al., 2022).

The paper is clear that this is not a standard supervised residual architecture. The network is an untrained U-net optimized per instance; the residual correction acts on the input latent/image-like variable $\dot a = A a + a^\top B a + \tau.$5, not as an internal ResNet block; and the method is fundamentally iterative rather than a fixed-depth trained feedforward model. Nonetheless, the mapping to an RPNN-like idea is direct: the residual is computed in projection space, backprojected by $\dot a = A a + a^\top B a + \tau.$6, and injected into the next neural reconstruction step. The step-size schedule is controlled by a sigmoid-shaped $\dot a = A a + a^\top B a + \tau.$7 with $\dot a = A a + a^\top B a + \tau.$8, $\dot a = A a + a^\top B a + \tau.$9, and total iterations $\tau$0. The implementation uses PyTorch, RMSProp, initial learning rate $\tau$1, learning-rate decay by $\tau$2 every $\tau$3 iterations, batch normalization even with batch size $\tau$4, and random initialization of U-net weights (Shu et al., 2022).

The experiments cover FORBILD phantom data, LIDC-IDRI lung CT images, and Finnish Inverse Problems Society real CT data, with both parallel-beam and fan-beam geometries, few-view, limited-angle, and noisy/low-dose-like settings. The gains over plain DIP are especially strong in highly ill-posed regimes. For parallel-beam LIDC-IDRI with $\tau$5 projections over $\tau$6–$\tau$7, the reported results are $\tau$8 dB / $\tau$9 for RBP-DIP versus $$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$0 dB / $$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$1 for DIP. In the limited-angle $$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$2–$$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$3 case with $$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$4 projections, RBP-DIP reaches $$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$5 dB / $$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$6 versus $$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$7 dB / $$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$8 for DIP. In fan-beam limited-angle reconstruction with $$\dot a = A a + a^\top B a + f^{\text{net}}(a,Re,t),$$9 projections over $X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$0–$X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$1, RBP-DIP obtains $X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$2 dB / $X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$3, while one pretrained baseline, MED50, slightly exceeds it in SNR with $X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$4 dB / $X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$5. The paper also states that RBP-DIP lacks comprehensive theoretical proof of convergence, remains relatively sensitive to noise, and is computationally slow because optimization is done per test image (Shu et al., 2022).

For the broader RPNN topic, the main significance of RBP-DIP is conceptual. It shows that “projection” need not mean a learned dimensionality reduction layer; it can mean explicit use of the physical forward model and adjoint in an iterative residual-correction loop around a neural prior.

4. Learned projection/back-projection networks for image enhancement and super-resolution

A more architectural interpretation of residual projection appears in image enhancement and super-resolution. The 2021 Back-Projection Pipeline (BPP) explicitly presents itself as a simple extension of residual networks that works simultaneously in multiple resolutions. It generalizes the single-state ResNet update $X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$6 to a coupled multi-scale system in which resolution-indexed states $X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$7 are updated by learned downscale and upscale operators. The discrete dynamics are written as

$X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$8

with the lowest-scale state fixed. The architecture uses $X_0 = [Re,\, t,\, a_1,\dots,a_r]^T,$9 resolution levels, $\tau$0 back-projection layers or flux-blocks, feature widths $\tau$1 channels from lowest to highest resolution, $\tau$2 convolutions, and scale transfer operators initialized as $\tau$3 bicubic filters and then trained. The paper emphasizes that the restriction operators $\tau$4 and interpolation operators $\tau$5 are nonlinear and time dependent, and that the resulting system is “causal in scale”: lower-resolution states do not depend on higher-resolution states after initialization (Michelini et al., 2021).

BPP is residual in multiple senses. Each scale evolves additively, each flux block uses corrected features $\tau$6, and the final image prediction is global residual learning,

$\tau$7

The model is trained in PyTorch with Adam, initial learning rate $\tau$8, halved every $\tau$9 backpropagation steps, and uses instance normalization before ReLU inside flux units. The reported main configuration has about $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$0 million parameters and throughput about $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$1 million pixels/sec on Titan X with FP16. On super-resolution benchmarks, BPP-SRx4 reports $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$2 PSNR on Set14, $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$3 on BSDS100, $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$4 on Urban100, and $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$5 on Manga109; for $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$6 SR it reports $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$7, $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$8, $$X_{t+1} = X_t + \tanh(W_t X_t + b_t), \qquad t=1,\dots,N-1.$$9, and $\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$0 on those datasets, respectively. On DeRaindrop, BPP reaches PSNR $\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$1 and SSIM $\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$2, compared with $\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$3 for DeRaindrop without GAN and $\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$4 for DeRaindrop with GAN (Michelini et al., 2021).

The 2019 Coupled-Projection Residual Network (CPRN) for MRI super-resolution gives a closely related but more explicitly two-branch design. Its shallow sub-network uses coupled-projection blocks—alternating up-projection and down-projection with reconstruction-error feedback—to preserve content consistency between low-resolution and high-resolution images. Its deep sub-network uses multiple residual blocks to learn the high-frequency difference between $\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$5 and $\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$6. The shallow branch therefore acts as a projection-consistency stream, while the deep branch acts as a residual-detail stream. In the step-wise fusion variant CPRN_S, shallow down-projection outputs are injected progressively into the deep residual branch by

$\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$7

The standard CPRN uses $\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$8 coupled-projection blocks and $\min \|\tilde{\tau} - \tau\|_F^2 + \lambda R(W,b).$9 residual blocks; CPRN_S uses $\dot a = A a + a^\top B a.$00 coupled-projection blocks and $\dot a = A a + a^\top B a.$01 residual blocks. The best reported channel configuration is $\dot a = A a + a^\top B a.$02 channels in the shallow network and $\dot a = A a + a^\top B a.$03 in the deep network, with no batch normalization in either branch. Training uses PyTorch 1.0, Adam, weight decay $\dot a = A a + a^\top B a.$04, initial learning rate $\dot a = A a + a^\top B a.$05, batch size $\dot a = A a + a^\top B a.$06, $\dot a = A a + a^\top B a.$07 epochs, and $\dot a = A a + a^\top B a.$08 patches (Feng et al., 2019).

The MRI experiments use Brats, ATLAS_native, and ATLAS_standardized, with bicubic downsampling for $\dot a = A a + a^\top B a.$09 and $\dot a = A a + a^\top B a.$10 super-resolution and PSNR/SSIM evaluation. CPRN outperforms SRCNN, VDSR, EDSR, and DDBPN on all three datasets at both scales. On Brats at $\dot a = A a + a^\top B a.$11, CPRN reports $\dot a = A a + a^\top B a.$12 versus $\dot a = A a + a^\top B a.$13 for DDBPN. On Brats at $\dot a = A a + a^\top B a.$14, CPRN reports $\dot a = A a + a^\top B a.$15, and CPRN_S improves that to $\dot a = A a + a^\top B a.$16. The ablations further show that projection-only and residual-only variants both underperform the full model, supporting the claim that projection consistency and residual high-frequency refinement are complementary (Feng et al., 2019).

Taken together, BPP and CPRN are the closest matches to an architectural reading of “Residual Projection Neural Network.” In both cases, projection/back-projection is embedded into the network’s state-update mechanism rather than appended as a purely external post-processing step.

5. Projection-residual formulations beyond model reduction and imaging

Projection-residual ideas also appear in settings that are not naturally described as feedforward neural architectures. The 2024 Residual Random Neural Networks (RRNN) paper studies a single-layer feedforward neural network with random hidden weights and replaces standard one-shot fitting by stagewise residual correction. At stage $\dot a = A a + a^\top B a.$17, a random projection matrix $\dot a = A a + a^\top B a.$18 defines hidden features $\dot a = A a + a^\top B a.$19, a ridge-regression head $\dot a = A a + a^\top B a.$20 is fit to the current residual $\dot a = A a + a^\top B a.$21, and the residual is updated by

$\dot a = A a + a^\top B a.$22

The final predictor is the additive sum

$\dot a = A a + a^\top B a.$23

Geometrically, each stage fits the residual by least squares in the span of the current random feature matrix. For $\dot a = A a + a^\top B a.$24, this is the orthogonal projection of the residual onto that span; with ridge regularization, it becomes a shrinked projection operator. The paper’s best reported RRNN results use $\dot a = A a + a^\top B a.$25, $\dot a = A a + a^\top B a.$26, $\dot a = A a + a^\top B a.$27, $\dot a = A a + a^\top B a.$28, yielding $\dot a = A a + a^\top B a.$29 on MNIST with $\dot a = A a + a^\top B a.$30 and $\dot a = A a + a^\top B a.$31 on fashion-MNIST with $\dot a = A a + a^\top B a.$32. This is not a conventional residual network, but it is a residual projection system in a mathematically literal least-squares sense (Andrecut, 2024).

The 2022 machine-unlearning paper titled “Machine Unlearning Method Based On Projection Residual” uses projection residual in yet another sense. Its key update is

$\dot a = A a + a^\top B a.$33

where $\dot a = A a + a^\top B a.$34 is the span of deleted examples. The paper states that this can be computed in $\dot a = A a + a^\top B a.$35, and that for any gradient-based update $\dot a = A a + a^\top B a.$36,

$\dot a = A a + a^\top B a.$37

For neural networks, however, the method is only approximate: all but the last layer are treated as fixed feature maps, and only the final layer is updated. The experiments use FIT evaluation and a bidirectional LSTM text classifier; the reported original training accuracy is $\dot a = A a + a^\top B a.$38, the unlearned model remains close to $\dot a = A a + a^\top B a.$39, and the time-consumption table lists $\dot a = A a + a^\top B a.$40 for the proposed projection residual method versus $\dot a = A a + a^\top B a.$41 for INF and $\dot a = A a + a^\top B a.$42 for retraining (Cao et al., 2022).

A terminological warning is provided by the 2023 path-planning work RPNN-RRT*. Here RPNN explicitly means Region Prediction Neural Network, not residual projection neural network. The model is a U-Net-like encoder-decoder with SE-Res Conv blocks and $\dot a = A a + a^\top B a.$43 convolution projection shortcuts $\dot a = A a + a^\top B a.$44, trained with a hybrid loss

$\dot a = A a + a^\top B a.$45

It reports about $\dot a = A a + a^\top B a.$46 Dice accuracy for region prediction and uses the predicted region to bias RRT* sampling. The planner terminates when the generated path cost is within $\dot a = A a + a^\top B a.$47 of the optimal path length within $\dot a = A a + a^\top B a.$48 samples, and across five scenarios RPNN-RRT* reports $\dot a = A a + a^\top B a.$49 success rate. The paper is relevant to the present topic only because it demonstrates acronym collision and because its internal residual block uses a projection shortcut; the acronym itself should not be expanded as “Residual Projection Neural Network” in this context (Huang et al., 2023).

6. Recurrent design motifs, distinctions, and misconceptions

Across the surveyed literature, “projection” denotes at least four different operations: projection onto a reduced basis in ROM closure learning, physical forward/backprojection in CT reconstruction, learned down/up scale transfer in image enhancement and MRI SR, and algebraic projection onto feature spans or deleted-data spans in residual least squares and unlearning. Likewise, “residual” may refer to ResNet skip dynamics, additive closure terms, back-projected data inconsistency, or stagewise residual targets. This suggests that RPNN is not a single blueprint but a cross-domain pattern in which a projection operator and a residual correction mechanism are composed to improve approximation, stability, or efficiency (Xie et al., 2019, Shu et al., 2022, Michelini et al., 2021, Feng et al., 2019, Andrecut, 2024, Cao et al., 2022).

Several misconceptions follow from collapsing these meanings. First, a method can be “projection-based” without containing any learned projection layer: ResNet-ROM uses classical Galerkin/POD projection, and RBP-DIP uses explicit physics operators $\dot a = A a + a^\top B a.$50 and $\dot a = A a + a^\top B a.$51. Second, a method can be “residual” without being a standard ResNet: RRNN uses residual least-squares fitting, and the machine-unlearning method uses projection of parameter differences. Third, not every paper using the acronym RPNN concerns residual-projection learning at all: in path planning, the acronym means Region Prediction Neural Network.

A plausible synthesis is that the most defensible encyclopedia definition of RPNN is a hybrid model class in which a projection-like operator supplies a structured consistency mechanism and a residual-like operator supplies an additive correction mechanism. Under that reading, ResNet-ROM represents the projection-plus-residual-closure variant for reduced-order PDE dynamics; RBP-DIP represents iterative residual backprojection around a neural prior; BPP and CPRN represent feedforward learned projection/back-projection architectures for image enhancement and super-resolution; RRNN and projection-residual unlearning represent algebraic versions of the same pattern. What the current literature does not yet provide is a universally accepted architecture, notation, or benchmark suite under the exact title “Residual Projection Neural Network.”