Deceptron: Discrete & Inverse Inference

• Deceptron is a dual framework encompassing discrete perceptron models that use combinatorial and statistical mechanics techniques to analyze capacity and phase transitions.
• It also represents a learned bidirectional module that employs Jacobian Composition Penalty to approximate local inverses for accelerating nonlinear inverse problems.
• Empirical studies on PDE-constrained problems confirm that the D-IPG algorithm derived from Deceptron achieves faster convergence and higher reliability than classical methods.

A Deceptron refers to one of two distinct but highly technical frameworks within the literature: (1) the discrete perceptron, i.e., a perceptron model with weights constrained to a finite set, which has been rigorously analyzed regarding storage capacity and phase transitions, and (2) the learned bidirectional module designed for accelerated nonlinear inverse problems in scientific machine learning. Both are historically and technically grounded in the mathematical analysis of inference in high-dimensional spaces, but they arise in different contexts and employ different methodologies.

1. Discrete Perceptrons (“Deceptron”): Foundations and Rigorous Results

The term Deceptron, established in the neural network theory literature, describes perceptron models where the weight vector $w$ has discrete (as opposed to continuous) entries. Canonical examples include $\pm1$ (binary), $0/1$, and bounded “box” perceptrons, where $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$, $w_j\in\{0,1/\sqrt{n}\}$, or $w_j\in[-1/\sqrt{n},1/\sqrt{n}]$, respectively. The normalization ensures that $\|w\|_2 = O(1)$ in the high-dimensional limit.

The central inquiry is the storage capacity $\alpha_c(\kappa)$: the maximal load $\alpha = m/n$ for which there exists a weight vector $w$ such that

$\pm1$0

for random patterns $\pm1$1 and labels $\pm1$2. The discrete setting renders $\pm1$3 (the admissible weight set) non--convex and combinatorial, distinguishing it sharply from the classical "spherical" perceptron. Analytical machinery must shift from continuous measure integrals to combinatorial and order-statistics techniques (Stojnic, 2013).

Gardner’s replica-based statistical mechanics analysis yields predictions for $\pm1$4 under the assumption of replica symmetry (RS). For instance, for the $\pm1$5 perceptron with threshold $\pm1$6,

$\pm1$7

where

$\pm1$8

At zero threshold, this gives $\pm1$9. For 0/1 perceptrons, capacity is a function of the sparsity $0/1$0 and is explicitly maximized over $0/1$1. Recent works rigorously justify these predictions via Gordon’s Gaussian comparison inequalities, concentration of measure, and combinatorial Hamming-ball counting, establishing precise feasibility/infeasibility phase transitions with overwhelmingly high probability in the linear regime $0/1$2, $0/1$3 (Stojnic, 2013).

A key result is that for the $0/1$4 model at zero threshold, combinatorial arguments show the true capacity is strictly less than the RS value: $0/1$5. This demonstrates explicit replica symmetry breaking in certain regimes, with proven bounds diverging from the RS predictions near critical thresholds.

2. Deceptron as a Learned Local Inverse Framework for Scientific Machine Learning

A distinct and more recent usage of Deceptron denotes a learned bidirectional module for nonlinear inverse problems arising in scientific domains such as PDE parameter inference, medical imaging, and geophysical inversion. In this context, Deceptron refers to a construct composed of a forward surrogate $0/1$6 and a learned local inverse $0/1$7. Together, these maps aim to amortize the cost of reconstructing local inverse geometry, thus circumventing repeated and expensive Jacobian-based linear solves required in Gauss–Newton or Levenberg–Marquardt methods (Kachhadiya, 13 May 2026, Kachhadiya, 26 Nov 2025).

Formally, consider the nonlinear least-squares inverse problem:

$0/1$8

where $0/1$9 is a differentiable surrogate for the true forward model. The Deceptron learns $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$0 such that locally $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$1, and, crucially, enforces that the Jacobian $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$2 acts as a local left-inverse to $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$3.

Jacobian Composition Penalty (JCP)

The core technical innovation enabling effective learning of $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$4 is the Jacobian Composition Penalty (JCP). By minimizing

$w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$5

where $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$6 is a random probe with $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$7, the procedure encourages $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$8 in a Frobenius norm sense, i.e., local pseudoinverse consistency. This local geometry is critical for stable and fast convergence, and allows $w_j\in\{-1/\sqrt{n},+1/\sqrt{n}\}$9 to act as an effective preconditioner (Kachhadiya, 13 May 2026, Kachhadiya, 26 Nov 2025).

3. D-IPG Algorithm and Theoretical Properties

The operational deployment of the learned Deceptron module is via the Deceptron Inverse-Preconditioned Gradient (D-IPG) algorithm. At each iteration $w_j\in\{0,1/\sqrt{n}\}$0,

1. Form the measurement-space proposal $w_j\in\{0,1/\sqrt{n}\}$1.
2. Pull back $w_j\in\{0,1/\sqrt{n}\}$2 using $w_j\in\{0,1/\sqrt{n}\}$3 to get $w_j\in\{0,1/\sqrt{n}\}$4.
3. Take a relaxation and feasible-set projection to produce $w_j\in\{0,1/\sqrt{n}\}$5.

The Armijo backtracking condition on the surrogate least-squares objective provides descent safeguards (Kachhadiya, 13 May 2026).

First-order Taylor analysis proves D-IPG is equivalent to damped Gauss–Newton, up to a deviation controlled by the composition error $w_j\in\{0,1/\sqrt{n}\}$6 and the local conditioning $w_j\in\{0,1/\sqrt{n}\}$7. Explicitly,

$w_j\in\{0,1/\sqrt{n}\}$8

where $w_j\in\{0,1/\sqrt{n}\}$9 is the local pseudoinverse error, and

$w_j\in[-1/\sqrt{n},1/\sqrt{n}]$0

When $w_j\in[-1/\sqrt{n},1/\sqrt{n}]$1, D-IPG and Gauss–Newton coincide on the recoverable subspace.

4. Empirical Performance Across PDE Inverse Problems

Deceptron and D-IPG have been validated on a broad suite of classical PDE-constrained inverse problems, including:

• Heat equation initial-condition recovery (1D, 2D, 3D)
• Darcy 2D permeability inversion
• Advection–Diffusion 2D and Allen–Cahn–2D reaction-diffusion
• Navier–Stokes 2D vorticity inversion

Empirical benchmarks demonstrate that D-IPG with JCP achieves a mean success rate of 94.8% across a six-problem reliability suite, significantly outperforming Gauss–Newton (17.3%) and Levenberg–Marquardt (65.5%). On representative problems, D-IPG attains comparable or better final residuals and RMSE at up to 77$w_j\in[-1/\sqrt{n},1/\sqrt{n}]$2 lower per-instance inference times compared to GN/LM. Dolan–Moré performance profiles show the approach is often fastest and uniquely able to solve all pooled instances (excluding known failure modes such as Heat-1D under poor geometry) within competitive time budgets (Kachhadiya, 13 May 2026).

5. Practical Implementation and Limitations

The Deceptron framework imposes several practical requirements:

• The surrogate $w_j\in[-1/\sqrt{n},1/\sqrt{n}]$3 must be sufficiently accurate and differentiable; both $w_j\in[-1/\sqrt{n},1/\sqrt{n}]$4 and $w_j\in[-1/\sqrt{n},1/\sqrt{n}]$5 must be trained and amortized across multiple inverse instances.
• The efficacy of D-IPG hinges on full-rank local Jacobians for invertibility.
• The method’s global convergence properties are unproven; performance degrades when the surrogate fails to expose reliable local geometry, as evidenced in the Heat-1D case.
• Manual tuning for the JCP weight and step-size schedule, as well as proper constraint projections, is necessary for best results.
• The methodology’s cost benefits accrue primarily in settings where many repeated solves share the same underlying forward structure (Kachhadiya, 13 May 2026, Kachhadiya, 26 Nov 2025).

Applications include large-scale PDE-inverse problems, and scientific–machine–learning pipelines in domains where forward evaluations are expensive but physics are consistent—e.g., geophysics, medical imaging, and fluids.

6. Connections, Extensions, and Outlook

The term Deceptron, as applied to discrete perceptrons, reveals fundamental phenomena about capacity, convexity, and algorithmic tractability; as a learned local inverse, it establishes a paradigm for amortizing local inverse geometry in nonlinear inference.

The DeceptronNet (“v0”) extends the single-step D-IPG preconditioning approach to multi-step, unrolled architectures, as demonstrated in 2D point-spread-function inversion. This approach leverages small, fixed-depth U-Nets to predict direct corrections, yielding convergence in a fixed small number of learned steps. This suggests scalability to more structured or multi-scale inverse pipelines, with ongoing work extending the architecture to coarse-to-fine unrolling, inclusion of richer priors, and automated diagnostic scheduling via the runtime RJCP metric.

In summary, Deceptron refers both to a foundational class of discrete perceptron models and, independently, to a learned bidirectional construct for accelerated scientific inverse problems. Both variants emphasize the interplay of local geometry, feasibility, and computational efficiency, providing a mathematical and practical basis for future advances in high-dimensional inference (Stojnic, 2013, Kachhadiya, 13 May 2026, Kachhadiya, 26 Nov 2025).