Diode ReLU and Shockley Rectification

• Diode ReLU is a neural activation function derived from replacing an ideal diode with a non-ideal Shockley diode, yielding a smooth rectifier expressed via the Lambert W function.
• It leverages circuit-theoretic and monotone operator principles to bridge hardware implementations with computational efficiency while maintaining a one-sided, rectifying behavior.
• The smooth transition of diode ReLU mitigates harsh discontinuities, enabling nuanced gradient approximations and improved analog hardware linearization in neural network training.

Searching arXiv for papers on diode ReLU and related ReLU/diode analogs to ground the article. Diode ReLU denotes, in the most specific sense represented in recent work, the activation function induced by replacing an ideal diode with a non-ideal Shockley diode in resistor–diode analog neural hardware. In that setting, the ideal diode yields the ordinary rectified linear unit, while the non-ideal diode yields a smooth ReLU-like map given in closed form through the Lambert $W$ function (Chaffey, 17 Sep 2025). In a broader and older sense, the phrase also aligns with the long-standing interpretation of ReLU as a rectifying, one-way, diode-like nonlinearity: positive inputs pass, whereas negative inputs are blocked or strongly suppressed, with consequences for spectral content, demodulation, and hardware realization (Kechris et al., 2024).

1. Ideal rectification and the ReLU limit

The circuit-theoretic starting point is the ideal forward-biased diode, modeled as a maximal monotone set-valued operator. In the formulation used for resistor–transformer–gyrator–diode circuits, the ideal diode impedance and its electrical dual admittance are

$$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$

When these operators are inserted into the forward/backward splitting step for the circuit kernel equation, the resolvent becomes exactly

$$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$

Accordingly, the idealized diode is not merely analogous to ReLU; in this framework it is the circuit element whose nonlinear step induces ReLU itself (Chaffey, 17 Sep 2025).

This identification clarifies why rectification is the central motif of the subject. ReLU is the canonical half-wave rectifier in neural-network form: it preserves one sign and suppresses the other. That asymmetry is the mathematical core of the “diode” description, independent of whether the implementation is electronic, analog circuit-theoretic, or optical.

The same ideal-diode limit also aligns with a complexity-theoretic distinction between piecewise-linear and polynomial activations. In the bit model, piecewise-linear activations such as ReLU and leaky-ReLU do not increase the bit-length of values multiplicatively with depth; deep $\mathsf{ERM}_{\text{bit}}$ is NP-complete, and exact one-step backpropagation is polynomial-time. By contrast, for polynomial activations with rational coefficients and degree at least $2$, $\mathsf{ERM}_{\text{bit}}$ is $\#P$-hard (Doron-Arad et al., 22 Feb 2026). This suggests that the ideal-diode/ReLU correspondence is not only physically parsimonious but also computationally congenial under finite precision.

2. Non-ideal Shockley diodes and the diode ReLU proper

The specific activation named diode ReLU arises when the ideal diode is replaced by a physically more accurate non-ideal diode governed by the Shockley equation

$$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$

where $n$, $v_T$, and $$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$0 are the ideality factor, thermal voltage, and saturation current. The activation is the corresponding resolvent $$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$1, which the cited work writes as

$$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$2

This Lambert-$$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$3 expression is what is explicitly called the diode ReLU (Chaffey, 17 Sep 2025).

Its qualitative properties are inherited from the diode physics and from monotone-operator theory. It is monotone because it is the resolvent of a monotone operator. It is asymmetric because it comes from a forward-biased diode model, with a dual reverse-biased version obtained by current/voltage interchange. It is threshold-like because conduction is negligible below the turn-on region and increases above it. Most importantly, it is a smoothed ReLU: the hard kink of the ideal ReLU is replaced by a rounded transition controlled by the non-ideal diode law (Chaffey, 17 Sep 2025).

This sharp distinction between ideal and non-ideal models is essential. In the ideal case, the activation is exactly piecewise linear. In the non-ideal case, the activation is device-specific, smooth, and physically grounded. The term “diode ReLU” is therefore most precise when it refers to this non-ideal, Shockley-induced activation rather than to ReLU’s generic rectifier analogy.

3. Monotone operator equilibrium networks and hardware differentiation

The diode ReLU is introduced within a broader analog-hardware correspondence. The relevant circuits are RTGD networks built from resistors, ideal transformers, ideal gyrators, and diodes, and their port behavior is shown to correspond to the solution of a monotone operator equilibrium network. In the ideal-diode case, this produces a ReLU MonDEQ; in the non-ideal case, the same equilibrium-network machinery yields the diode ReLU (Chaffey, 17 Sep 2025).

The circuit kernel is written as an inclusion involving a positive semidefinite matrix $$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$4, diode operators $$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$5 and $$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$6 applied entrywise, and the external input and output maps. Solving that kernel by forward/backward splitting gives an iteration of the form

$$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$7

with $$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$8. In this update, the nonlinear step is precisely the diode resolvent: ideal-diode resolvent for ReLU, Shockley-diode resolvent for diode ReLU (Chaffey, 17 Sep 2025).

The same work further introduces hardware linearization, a procedure for computing gradients directly in hardware by replacing each nonlinear diode with its local linear approximation. The resulting linearized circuit solves the linearized implicit-differentiation equation associated with the equilibrium network. This point matters for the encyclopedia definition of diode ReLU because it situates the activation not merely as a transfer curve but as part of a trainable physical computing substrate. The activation function is determined by the device model, and differentiation is likewise realized at the device level.

4. Rectification, DC generation, and demodulation

Outside the specific MonDEQ setting, the “diode” aspect of ReLU has long been understood through signal processing. ReLU can be written as

$$v \in Z_{\text{diode}}(i) := \begin{cases} 0 & i>0,\ (-\infty,0] & i=0, \end{cases} \qquad i \in Y_{\text{diode}}(v) := \begin{cases} 0 & v>0,\ (-\infty,0] & v=0. \end{cases}$$9

which makes its rectifying role explicit: one half of the signed waveform is preserved, and the nonlinear absolute-value term folds the other half. In the frequency domain, this rectification introduces higher-frequency oscillations together with a constant DC component (Kechris et al., 2024).

For an input represented as $$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$0, the cited frequency-domain analysis shows that rectification creates harmonics at $$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$1, sum frequencies $$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$2, difference frequencies $$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$3, and a constant term. The output is decomposed into three parts: a constant DC term, a scaled copy of the original signal, and higher-order nonlinear spectral terms. The DC component has amplitude

$$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$4

and after convolution it becomes

$$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$5

so that the DC level depends jointly on the input’s frequency amplitudes and the filter response. In the reported interpretation, this DC term is not merely an artifact; it is an informative representation of frequency content that can be extracted by global average pooling and can help CNNs converge to a weight configuration close to the initial random weights (Kechris et al., 2024).

A related but distinct signal-processing view treats ReLU as a demodulator. On amplitude-modulated inputs, half-wave rectification can recover a low-frequency envelope, and ReLU is therefore described as a nearly ideal demodulator. The same abrupt nonlinearity also produces nonlinear distortion products, termed decoy features, which can mislead learning. The proposed remedy in that setting is Parallel Dither, in which noisy replicas are passed through ReLU and gradients are averaged; the claim is that this suppresses distortion products while preserving the true demodulated component (Simpson, 2015).

Taken together, these analyses explain why the diode analogy persists. Even when no literal diode is present, ReLU behaves as a rectifier in the classical engineering sense: it generates a unipolar response, creates a DC offset, and performs demodulation-like extraction of low-frequency structure.

5. Optical and photonic realizations of diode-like ReLU behavior

Several optical systems realize ReLU-like or diode-like activation without implementing the specific Shockley-based diode ReLU. In a periodically-poled thin-film lithium niobate nanophotonic waveguide, an all-optical ReLU is produced by phase-sensitive $$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$6 frequency conversion. Positive input is encoded by $$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$7, negative input by $$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$8, and a bias pulse at $$(I+\alpha\psi)^{-1}(\cdot)=\mathrm{relu}(\cdot),\qquad \mathrm{relu}(u):= \begin{cases} u & u\ge 0,\ 0 & \text{otherwise}. \end{cases}$$9 with $\mathsf{ERM}_{\text{bit}}$0 drives either second-harmonic generation or degenerate optical parametric amplification depending on the phase relation $\mathsf{ERM}_{\text{bit}}$1. The resulting transfer function closely approximates $\mathsf{ERM}_{\text{bit}}$2, with reported $\mathsf{ERM}_{\text{bit}}$3, energy per activation of about $\mathsf{ERM}_{\text{bit}}$4, rise time $\mathsf{ERM}_{\text{bit}}$5 fs, fall time $\mathsf{ERM}_{\text{bit}}$6 fs, characteristic response time $\mathsf{ERM}_{\text{bit}}$7 fs, and practical time per activation taken as about $\mathsf{ERM}_{\text{bit}}$8 fs. In an MNIST CNN, replacing ideal ReLU layers with the measured optical response reduced accuracy from $\mathsf{ERM}_{\text{bit}}$9 to $2$0, and after only $2$1 epochs of fine-tuning accuracy returned to $2$2 (Li et al., 2022).

A second photonic route uses compact doubly resonant cavities exploiting phase-sensitive second-harmonic generation in a $2$3 medium. Here the sign of the input is encoded in the optical phase, with $2$4 for positive input and $2$5 for negative input, and the second-harmonic output power behaves approximately as

$2$6

The design targets $2$7 and $2$8, uses a multilayer AlGaAs/air cavity of about $2$9, and reports $\mathsf{ERM}_{\text{bit}}$0, $\mathsf{ERM}_{\text{bit}}$1, $\mathsf{ERM}_{\text{bit}}$2, $\mathsf{ERM}_{\text{bit}}$3, $\mathsf{ERM}_{\text{bit}}$4, response times of about $\mathsf{ERM}_{\text{bit}}$5 fs and $\mathsf{ERM}_{\text{bit}}$6 fs for the two modes, suppression for negative-phase inputs by more than $\mathsf{ERM}_{\text{bit}}$7 dB, agreement with coupled-mode predictions within about $\mathsf{ERM}_{\text{bit}}$8, and $\mathsf{ERM}_{\text{bit}}$9 for inputs up to about $\#P$0 fJ. In a CNN for MNIST, ideal ReLU accuracy is reported as $\#P$1 and optical ReLU accuracy as $\#P$2, within about $\#P$3 (Ahmadnejad et al., 28 Apr 2025).

A third optical realization uses a semiconductor slave laser subject to optical injection from a master laser. In a broad regime above the Hopf bifurcation of the injection-locking diagram, the output power versus input power develops a kinked, piecewise-linear curve that resembles a ReLU or leaky ReLU. For $\#P$4, the threshold power at the kink is about $\#P$5. The paper models the activation used in network simulations as

$\#P$6

for ReLU-I and

$\#P$7

for ReLU-II, with $\#P$8 decreasing nonlinearly from $\#P$9 at $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$0 to $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$1 at $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$2, while the threshold rises nonlinearly from about $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$3 to $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$4. In an MNIST classifier with two fully connected hidden layers of $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$5 and $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$6 nodes, ReLU-I achieved $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$7 to $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$8, ReLU-II $$v = Z_S(i) := n v_T \log\!\left(\frac{i}{i_s}+1\right),$$9 to $n$0, and standard leaky ReLU $n$1 (Liu et al., 2023).

These optical demonstrations are not the same object as the Shockley-based diode ReLU. Their relevance lies in showing that diode-like rectification can be instantiated through phase-sensitive frequency conversion, doubly resonant nonlinear cavities, or injection-locked laser dynamics while remaining useful as a neural activation.

6. Distinctions from other ReLU modifications

Diode ReLU should not be conflated with other methods that alter ReLU’s inactive branch or its backward dynamics. N-ReLU (Noise-ReLU) replaces the negative branch with Gaussian noise,

$n$2

and is explicitly described as a zero-mean, expectation-aligned stochastic extension of ReLU, with

$n$3

Its purpose is to mitigate dying ReLUs by preserving gradient flow in inactive regions and acting as an annealing-style regularizer. The cited source is directly relevant to a diode-ReLU query only in a conceptual sense and explicitly notes that it is not a true diode ReLU in the sense of a deterministic asymmetric gate with a fixed negative-side leak (Manik et al., 10 Nov 2025).

A different line of work leaves the forward function exactly equal to ReLU and modifies only the backward pass. SUGAR preserves $n$4 during the forward pass but replaces the derivative with a smooth surrogate by Forward Gradient Injection,

$n$5

This strategy is motivated by the claim that the forward rectification itself is not the main problem; rather, the zero gradient for negative inputs causes dead units. In that sense, SUGAR is conceptually close to the general idea of keeping a diode-like forward path while relaxing the backward dynamics, but it is not a diode-derived activation in the hardware or Shockley sense (Horuz et al., 28 May 2025).

Other ReLU alternatives depart even further from the diode picture. PLU, for example, is defined by

$n$6

and is presented as a hybrid of tanh and ReLU with three linear pieces, gradients $n$7 or $n$8, and a well-defined inverse on the entire domain. Its goal is greater expressivity and avoidance of dead units, not diode-based rectification (Nicolae, 2018).

The main misconception, therefore, is terminological. In the strict circuit-theoretic sense, diode ReLU is the Shockley-induced smoothed ReLU from analog equilibrium hardware. In looser usage, “diode ReLU” may refer to any ReLU-like asymmetry or rectifier-inspired activation. The distinction is substantive: only the former is defined by an explicit non-ideal diode model and a corresponding Lambert-$n$9 resolvent.