Local Inverse Geometry Can Be Amortized

Abstract: Nonlinear inverse problems often trade inexpensive but fragile first-order updates against curvature-aware methods such as Gauss-Newton and Levenberg-Marquardt, which obtain stronger directions by repeatedly solving Jacobian-based linearized systems. We propose a learned alternative: amortize local inverse geometry into a reusable reverse operator. Our framework learns a bidirectional surrogate, Deceptron, and deploys it through D-IPG (Deceptron Inverse-Preconditioned Gradient), an iterative solver that pulls residual-corrected measurement-space proposals back to latent space. The key mechanism is a Jacobian Composition Penalty (JCP), which trains the reverse Jacobian to act as a local left inverse of the forward Jacobian; its runtime counterpart, RJCP, measures the same inverse-consistency error along optimization trajectories. We prove that D-IPG is first-order equivalent to damped Gauss-Newton under local pseudoinverse consistency, with deviation controlled by composition error and conditioning. Across seven PDE inverse-problem benchmarks, D-IPG outperforms standard baselines, achieves 94.8% mean success across the six-problem reliability suite, and reaches comparable or better recovery quality at up to 77x lower inference-time solve cost on the main benchmarks.

• The paper introduces a learned framework using the Deceptron module to amortize local inverse geometry in nonlinear PDE inverse problems.
• It employs the Jacobian Composition Penalty to enforce local inverse consistency, achieving near-classical curvature-guided updates.
• Empirical results on seven PDE benchmarks demonstrate faster convergence and improved reliability compared to classical solvers.

Amortized Local Inverse Geometry for Nonlinear PDE Inverse Problems

Introduction

"Local Inverse Geometry Can Be Amortized" (2605.13068) introduces a learned framework for nonlinear inverse problems, targeting scenarios where repeated reconstruction of local inverse geometry incurs substantial computational overhead. Traditionally, methods such as Gauss–Newton and Levenberg–Marquardt leverage curvature information for stronger update directions, but their cost is dominated by solving Jacobian-based linearized systems at each optimization step. The paper proposes substituting these repeated computations with a learned amortized operator, instantiated as a bidirectional neural module dubbed Deceptron, whose reverse map serves to efficiently convert measurement-space corrections into latent-space updates.

Methodological Framework

The central methodological innovation is the Deceptron module, comprising a differentiable forward surrogate $f_W$ and a reverse map $g_V$ trained to encode local inverse geometry. Rather than enforcing a global inverse relation, the reverse map is trained to approximate the local left inverse of the forward Jacobian within regions relevant to nonlinear least-squares optimization. The paper introduces the Jacobian Composition Penalty (JCP), a probe-based loss enforcing that the composition $J_g(f_W(x)) J_f(x)$ remains close to the identity, thus promoting differential inverse consistency without requiring explicit Jacobian construction.

During inference, Deceptron induces the Deceptron Inverse-Preconditioned Gradient (D-IPG) solver. At each iteration, D-IPG:

1. Forms a measurement-space proposal $y^{\mathrm{prop}} = f_W(x_t) - \alpha r_t$,
2. Pulls it back with $x^{\mathrm{prop}} = g_V(y^{\mathrm{prop}})$,
3. Applies relaxation and projection onto the feasible latent set,
4. Uses Armijo backtracking for acceptance control.

The analysis establishes that, under local JCP-induced consistency, the D-IPG step recovers the first-order geometry of damped Gauss–Newton, with deviation bounded in terms of the composition defect and conditioning of $J_f(x)$:

$$\|\Delta x_{\mathrm{DIPG}}-\Delta x_{\mathrm{GN},\alpha}\|_2 \le \alpha \frac{\|J_g(f_W(x))J_f(x)-I\|_2}{\sigma_{\min}(J_f(x))} \|r_t\|_2$$

This result formalizes the conditions under which amortized updates faithfully replicate curvature-guided classical steps, while clarifying regimes—particularly ill-conditioned—for which deviation serves as implicit regularization.

Figure 1: Solver performance on PDE inverse benchmarks, showing residual convergence, iteration profiles, and wall-clock efficiency for D-IPG versus classical baselines.

Empirical Evaluation

The empirical section covers seven PDE inverse benchmarks, including diffusive, reaction–diffusion, elliptic, advection–diffusion, and fluid-dynamic settings. D-IPG is benchmarked against Gauss–Newton, Levenberg–Marquardt, L-BFGS, and gradient descent baselines. On representative problems (Heat-3D, Advection–Diffusion-2D, Allen–Cahn-2D):

• D-IPG achieves convergence within 5–7 iterations and $<$0.2s solve time, compared to $>$30 iterations and $>$1s for GN.
• On the full reliability suite (six PDEs), D-IPG attains $g_V$0 mean success under task-specific RMSE thresholds, with $g_V$1 on five of six.
• On Allen–Cahn-2D with basin structure, the benefit of JCP is explicit: 82.5% basin occupancy for D-IPG(+JCP), versus 18.75% for D-IPG(-JCP).

  Figure 2: Allen–Cahn-2D mechanism study; JCP amplifies basin occupancy and suppresses high-error trajectories.

  

  Figure 3: Reliability matrix across six PDEs, reporting success rates and mean wall-clock time; columns are ordered by RJCP diagnostic.

The data and iteration profiles demonstrate that D-IPG consistently outperforms competitors in time-to-tolerance, iteration count, and budgeted solve rate, provided the problem exhibits sufficient local invertibility under the trained surrogate.

Structural and Geometric Diagnostics

The paper elucidates the geometric regime of D-IPG updates. Cosine similarity analyses reveal that successful D-IPG updates span a spectrum from gradient-aligned to highly corrective, directly refuting the notion that the method simply learns rescaled gradient descent. Notably, on Navier–Stokes-2D, D-IPG achieves $g_V$2 reliability despite near-orthogonality with the gradient direction, confirming the capability to learn strong non-gradient geometries.

The role of JCP is further validated via reproducibility metrics: across benchmarks, higher RJCP correlates with increased variance in final RMSE, and in saturated problems, bundle width remains visibly tighter under JCP enforcement.

Figure 4: Full benchmark performance profiles; D-IPG leads on all metrics except problems (Heat-1D) lacking reliable local inverse structure.

Practical Implications and Future Directions

The framework is architecturally agnostic and can be instantiated with MLPs or CNNs, provided the forward model exposes local geometry suitably. Amortized training cost is offset after a few hundred to a few thousand solves versus classical baselines, making D-IPG ideal for amortized regimes (e.g., batch inverse recovery in scientific computing, imaging, or level-set inversion).

Theoretical implications extend to learned operator approaches, highlighting how leveraging differential consistency via JCP enables substantial runtime savings without sacrificing reliability. Future directions entail extension to global convergence safeguards, combining with preconditioned solvers, and adaptation for tasks where local inverse geometry is inherently ill-behaved.

Conclusion

"Local Inverse Geometry Can Be Amortized" demonstrates that amortizing local inverse geometry via learned reverse operators yields substantial gains for nonlinear PDE inverse problems. By leveraging JCP to enforce local pseudoinverse consistency, D-IPG achieves near-classical reliability and quality at orders-of-magnitude lower solve cost in amortized settings. The analytic results precisely delimit conditions for equivalence to Gauss–Newton and characterize the deviation for ill-conditioned regimes. The empirical record corroborates these claims, with robustness across dimensions and problem types, and figures substantiate both quantitative and qualitative aspects of solver behavior. The work establishes a new data-driven protocol for efficient, reliable amortized inverse solving in scientific and engineering contexts.

Figure 5: Qualitative reconstructions for three benchmarks; D-IPG(+JCP) delivers visibly accurate recovered fields compared to classical baselines.