Equivariant Plug-and-Play (EPnP)

• EPnP is a computational framework that enforces symmetry equivariance through randomized group transformations to enhance image restoration in inverse problems.
• It integrates a Monte Carlo approximation to average denoiser outputs, mitigating artifacts and improving the stability of plug-and-play methods.
• EPnP offers theoretical convergence guarantees, better MSE performance, and reduced spectral norms, with applications in deblurring, super-resolution, and MRI.

Equivariant Plug-and-Play (EPnP) is a computational framework in inverse problems and image restoration that enhances the stability and reconstruction quality of plug-and-play (PnP) methods by enforcing symmetry-related equivariances on denoisers at test time. EPnP achieves this by randomizing over group transformations (such as rotations, reflections, and translations) within each PnP iteration, systematically aligning the algorithmic behavior with invariances of the image distribution. The resulting method can be viewed both as an efficient Monte Carlo approximation to an equivariant proximal operator and as a descent-based solution to a variational problem with a group-averaged prior.

1. Background: Plug-and-Play Methods and Inverse Problems

In linear inverse problems, the goal is to reconstruct an image $x \in \mathbb{R}^n$ from degraded measurements $y \in \mathbb{R}^m$ generated via a known linear operator $A : \mathbb{R}^n \to \mathbb{R}^m$ with additive noise $\epsilon$:

$y = A x + \epsilon.$

A standard MAP approach seeks

$\hat{x} = \arg\min_x f(x) + \lambda r(x)$

with $f(x) = \frac{1}{2} \|A x - y\|_2^2$ encoding data fidelity, and $r(x)$ serving as a regularization or image prior. Proximal-splitting algorithms address this by alternating a gradient step on $f$ with a proximal (regularization) step.

Plug-and-Play (PnP) methods replace the explicit proximal map of $r$ with a powerful, often pretrained, denoiser $D$:

$$x_{k+1} = D \left( x_k - \gamma \nabla f(x_k) \right) = D \left( x_k - \gamma A^T (A x_k - y) \right),$$

thereby sidestepping explicit prior design and leveraging denoising networks trained for Gaussian denoising. RED (Regularization by Denoising) and Langevin-based samplers follow closely related rationales.

2. Motivation for Equivariance in Plug-and-Play Algorithms

Despite their flexibility, standard PnP methods may be numerically unstable or produce suboptimal reconstructions, particularly when the denoiser does not respect the inherent symmetries (e.g., rotation, reflection, translation) of the image prior. State-of-the-art convolutional denoisers are typically only approximately translation-equivariant and generally break rotation or reflection symmetry, leading to mismatches in the algorithm’s fixed-point iteration and manifesting as divergence or undesirable artifacts.

Many natural image priors are either invariant or symmetric under actions of finite or compact groups $G$, such as the dihedral group (rotations/reflections) or translation subgroups. The proximal operator of a $G$-invariant regularizer is necessarily $G$-equivariant; that is, applying a group transform to the input of the operator commutes with the operator output. The absence of exact equivariance in $D$ breaks this property, resulting in instabilities and the amplification of artifacts in the nullspace of $A$.

3. Methodology: The EPnP Algorithm

EPnP imposes group equivariance on any pretrained denoiser via randomization over group actions at inference. Let $G$ be a finite (or compact) group of unitary transforms $T_g$ (e.g., 90° rotations, flips, shifts). The core components are:

• Group-averaged denoiser:

$$D_G(x) = \frac{1}{|G|} \sum_{g \in G} T_g^{-1} D(T_g x).$$

By design, $D_G$ satisfies $D_G(T_h x) = T_h D_G(x)$ for all $h \in G$.

• Monte Carlo approximation:

The cost of averaging over $|G|$ transformations motivates a stochastic variant:

$$\tilde{D}_G(x) = T_g^{-1} D(T_g x), \quad g \sim \text{Uniform}(G),$$

where a single $g$ is sampled per iteration.

• EPnP iteration:

In place of the standard update, EPnP performs:

$$\begin{align*} &\text{Sample } g_k \sim \text{Uniform}(G) \ &z \leftarrow x_k - \gamma A^T (A x_k - y) \ &x_{k+1} \leftarrow T_{g_k}^{-1} D(T_{g_k} z) \end{align*}$$

yielding an update that is equivariant in expectation. For generic transformation structures, the algorithm can be written more generally as:

$x_{k+1} = J_G^T D_\sigma(G y_k)$

where $J_G^T$ is the Jacobian-transpose of $G$ (equal to $G^{-1}$ for isometries).

• Equivariant variational objective:

EPnP corresponds to an “equivariant” prior,

$$r_\sigma^\pi(x) = -\mathbb{E}_{G \sim \pi} [ \log p_\sigma ( G x ) ],$$

where $p_\sigma$ is the Gaussian-smoothed prior and $\pi$ a measure over $G$, thus solving:

$$\hat{x} = \arg\min_x f(x) + \lambda r_\sigma^\pi(x).$$

Component	Standard PnP	EPnP
Denoiser call	$D(x - \gamma\nabla f(x))$	$T_g^{-1} D(T_g[x - \gamma\nabla f(x)])$
Equivariance enforced	No	In expectation, with respect to $G$
Extra computational cost	Baseline (one pass)	One additional group transform/inverse per iter

This formulation enables a black-box wrapper around any pretrained denoiser, with no retraining or modification to the network architecture required. Group $G$ can be selected to match domain invariances (e.g., rotations for microscopy, full shifts and flips for natural images).

4. Theoretical Properties and Analysis

EPnP is analyzed as a stochastic, perturbed proximal-gradient algorithm. Key theoretical claims include:

• Improved Denoising MSE:

For $G$-invariant noise and signal distributions,

$$\mathbb{E} \| D_G(x + \epsilon) - x \|^2 \leq \mathbb{E} \| D(x + \epsilon) - x \|^2,$$

indicating that group-averaged denoisers do not increase MSE.

• Jacobian symmetrization:

A necessary condition for a denoiser $D$ to correspond to a differentiable prior is that its Jacobian $J_x$ be symmetric. Group averaging asymptotically symmetrizes $J_x$. Empirically, the normalized asymmetry $\|J - J^T\|_F / \|J\|_F$ in popular denoisers drops by $3{-}5\times$ when using EPnP.

• Lipschitz constant reduction:

The averaged denoiser $D_G$ satisfies

$\text{Lip}(D_G) \leq \text{Lip}(D).$

For linear $D$ with a non-equivariant leading singular mode, the inequality is strict. Spectral norms of $J_x$ are empirically reduced, enhancing contractivity.

• Spectral incoherence for stability:

If the forward operator $A^T A$ is not $G$-equivariant (e.g., has random structure as in accelerated MRI sampling), its eigenstructure is misaligned with that of an equivariant $D$. This further reduces the spectral norm of their composition, reinforcing convergence.

• Convergence guarantees:

Assuming data-fidelity $f$ is $L_f$-smooth and bounded below, and the denoiser is prox-structured (i.e., $D_\sigma = \mathrm{Id} - \nabla h_\sigma$, $\nabla h_\sigma$ is $L_h$-Lipschitz, $L_h < 1$), and the group acts in an unbiased way ($\mathbb{E}[J_G^T G] = \mathrm{Id}$), then for $\lambda \geq 3 L_f$, the gradient norm on the equivariant objective

$$\frac{1}{N} \sum_{k=0}^{N-1} \mathbb{E} \| \nabla \tilde{F}(x_k - \zeta_k) \|^2 \leq \frac{C_1(\tilde{F}(x_0) - F^*)}{N} + C_2 \mu^2,$$

vanishing as $N \rightarrow \infty$ when the group is finite (so $\mu = 0$). The proof involves descent-type inequalities and control of stochastic perturbations.

5. Empirical Performance and Evaluation

Experiments on various inverse problems concretely demonstrate the advantages of EPnP. Benchmarks include motion/gaussian deblurring (BSD10/Set3C), $2\times$–$8\times$ super-resolution, and accelerated MRI with standard and advanced denoisers (DnCNN, DRUNet, SwinIR, SCUNet, DiffUNet, GSNet). Key findings:

• Deblurring (BSD10, motion kernel):
  • DnCNN–PnP: $30.4 \pm 0.3$ dB (unstable)
  • EPnP–DnCNN: $30.9 \pm 0.1$ dB
• Deblurring (BSD10, gaussian):
  • DRUNet–PnP: $16.5 \pm 9.8$ dB (unstable)
  • EPnP–DRUNet: $28.4 \pm 2.3$ dB
• MRI ($4\times$):
  • DnCNN–PnP: $28.6 \pm 4.3$ dB
  • EPnP–DnCNN: $30.1 \pm 3.9$ dB
• Super-resolution ($4\times$, Set3C):
  • LipDnCNN–PnP: $31.7 \pm 2.0$ dB
  • EPnP–LipDnCNN: $31.8 \pm 2.0$ dB

Qualitatively, EPnP consistently eliminates geometric artifacts (ghosting, checkerboard patterns) that present in standard PnP after many iterations and matches or surpasses convergent baselines (wavelets, TGV, GSPnP), all without retraining or structural modification.

In the deblurring setting with DRUNet and small noise ($\sigma_y=5/255$), classical PnP achieves $29.98$ dB PSNR, while EPnP with random $90^\circ$ rotations yields $30.22$ dB at identical wall-clock cost, corresponding to improved visual quality and artifact suppression. Similar gains (0.1–0.2 dB) are observed in SAR despeckling and related tasks.

6. Implementation and Practical Considerations

• Computational Overhead:

EPnP adds the cost of one group transform and its inverse per iteration (negligible relative to most denoisers), or up to $|G|$ in the full group-average case. Single random samples per iteration are empirically sufficient.

• Parameter Tuning:

The step size is typically set as $\gamma = 1/\lambda$, with $\lambda$ cross-validated in $[0.1, 1.0]$ to maximize PSNR. The denoising strength $\sigma$ is chosen to match desired prior smoothness, e.g., $\sigma \approx 4$–$8/255$ for images in $[0,1]$.

• Iteration Count and Convergence:

$N \approx 200$–$500$ iterations suffices for most problems; experiments with $10^3$–$10^4$ iterations confirm stability.

• Adaptation to Domain:

The choice of $G$ can be problem-specific. Microscopy may use rotations only; natural images may benefit from fuller groups including translations and reflections.

• Augmentation vs. Exact Equivariance:

EPnP typically applies equivariance only at test time through group action randomization. Exact architectural equivariance (via group convolutions or explicit parameter-tying) can also be used but was found to yield similar improvements to test-time randomization in practice.

• Limitations:

In extremely ill-conditioned problems or with highly irregular denoisers (e.g., non-Lipschitz models at high noise), EPnP can still drift. Remedies include step-size/backtracking or parameter scheduling. For large $|G|$, minibatches or antithetic transform sampling offer scalable alternatives.

7. Significance and Outlook

EPnP introduces a principled, computationally lightweight means to enforce desired equivariances in plug-and-play restoration. It is agnostic to denoiser architecture and leverages only test-time transformations, delivering practical stability and accuracy gains without retraining. The convergence guarantees established under mild assumptions surpass those available for standard PnP. The methodology is compatible with recent innovations in diffusion-based plug-and-play, stochastic proximal schemes, and group-convolutional denoiser architectures.

Future directions include using multi-sample Monte Carlo estimators, extension to continuous symmetry groups, and the design of new network classes with built-in group equivariance, as well as broader integration with probabilistic frameworks for uncertainty quantification and sampling.

EPnP thus represents a robust, theoretically sound extension of the PnP paradigm, addressing the misalignment between learned denoisers and image symmetries to obtain improved and more reliable solutions to challenging inverse imaging problems (Terris et al., 2023, Renaud et al., 13 Nov 2025).