Lee Oscillator Activation Function

• The Lee Oscillator Activation Function is a hardware-based analog firing rate mechanism that produces a sigmoid-like response using an S-shaped switching element and a tunable resistor.
• It employs coupled RC circuits and analytical exponential-sigmoid fitting to precisely control the oscillation frequency, aligning with continuous, differentiable transfer functions.
• The design enables direct analog rate coding for spike neural networks, offering compact implementation with adjustable parameters for interfacing with variable resistance sensors.

The Lee Oscillator Activation Function refers to the firing rate response of an analog relaxation oscillator neuron based on a circuit topology introduced for neuromorphic and spike neural network (SNN) applications. This circuit uses a switching element with an S-shaped current–voltage (I–V) characteristic and a tunable resistor to produce a functional dependence between the control resistor value and the output spike frequency that closely approximates a sigmoid function. This hardware-based activation enables direct analog rate coding in SNNs, providing a minimal and tunable physical realization of a continuous, differentiable, sigmoid-like transfer function.

1. Circuit Architecture and Physical Principles

The oscillator circuit consists of a constant current source ($I_0$) applied in series to three elements: an S-shaped switching element ("S-switch"), two capacitors ($C_1$ and $C_2$) in series, and a return path to ground. A variable control resistor ($R_{\mathrm{ctl}}$) is connected in parallel with $C_1$. The S-switch displays an S-shaped static I–V curve with a region of negative differential resistance bounded by two threshold voltages ($U_{\mathrm{th}}$ and $U_n$).

The core operation is governed by relaxation oscillations: as the node voltage $U_{\mathrm{sw}}(t)$ rises under constant current, the S-switch abruptly transitions ("flips") between OFF and ON states once $U_{\mathrm{sw}}$ crosses specific thresholds. Capacitors $C_1$ and $C_2$, together with $R_{\mathrm{ctl}}$, control the charge/discharge dynamics and hence the oscillation period. The frequency of these oscillations, $f(R_{\mathrm{ctl}})$, constitutes the neuron's activation output.

2. Analytical Formulation for Firing Rate Response

The switching cycle of the Lee-oscillator neuron is described as two alternating phases (ON and OFF states of the S-switch). The voltage evolution on $C_1$ and $C_2$ is determined by coupled first-order linear ordinary differential equations (ODEs) in each switching phase, with coefficients set by $R_{\mathrm{ctl}}$ and the S-switch branch resistance ($R_{\mathrm{on}}$ or $R_{\mathrm{off}}$).

General analytic solutions for $U_1(t)$ (across $C_1$) and $U_2(t)$ (across $C_2$) take the form: $$U_1(t) = V_0 + M_1 e^{-\alpha_1 t} + M_2 e^{-\alpha_2 t}$$

$$U_2(t) = V_0 + M_1 B_1 e^{-\alpha_1 t} + M_2 B_2 e^{-\alpha_2 t}$$

with system parameters $a_1$, $a_2$, $b$ set by the circuit configuration and S-switch state, and $\alpha_{1,2}$ determined by the system matrix eigenvalues. Switching times are found by solving transcendental equations for the evolution of $U_{\mathrm{sw}} = U_1 + U_2$ to reach $U_{\mathrm{th}}$ (ON transition) and $U_n$ (OFF transition). The firing frequency is determined as $f(R) = 1 / [T_1(R) + T_2(R)]$, where $T_1$ and $T_2$ are the respective phase durations.

3. Sigmoid-Like Activation Function Emergence

Over an extended range of $R_{\mathrm{ctl}}$, the frequency–resistor characteristic $f(R)$ assumes a classic sigmoid shape, with low-frequency and high-frequency plateaus separated by a sharply increasing region. This relationship is tightly fitted by a five-parameter exponential-sigmoid function: $$f_{\rm app}(R) = A_1 \frac{1 - e^{-A_3 R}}{1 + e^{-A_2 (R - A_4)}} + A_5$$ For example, with $C_1 = 10$ nF, $C_2 = 1$ μF, $I_0 = 150$ μA, typical fit parameters are $A_1=2398.8$ Hz, $A_2=0.0848$ Ω⁻¹, $A_3=0.00415$ Ω⁻¹, $A_4=196$ Ω, and $A_5=54$ Hz. The lower plateau $A_5$ reflects slow oscillation for low $R_{\mathrm{ctl}}$, while $A_1+A_5$ sets the high-frequency limit for large $R_{\mathrm{ctl}}$. The steepness and central $R$ value for the transition are governed by $A_2$ and $A_4$.

The emergence of the sigmoid is a direct consequence of the transition from "C$_2$-dominated" to "C$_1$-dominated" time constants as $R_{\mathrm{ctl}}$ is swept, reflecting the circuit physics of series-parallel RC charging in each phase.

4. Implementation in Spike Neural Network Architectures

The Lee oscillator activation function is designed for use as a physical analog neuron in SNNs, wherein the frequency of output spikes directly encodes the activation level. This approach eliminates the need for digital-to-analog conversion or tabulated activation functions in hardware neurons. The neuron’s "input" is set by the value of $R_{\mathrm{ctl}}$, which can be directly interfaced with resistive sensors or upstream neuron outputs via configurable resistance-control blocks (e.g., MOSFETs, memristors, or thermal couplings).

Spike trains with frequency $f(R_{\mathrm{ctl}})$ serve as output, enabling direct firing rate coding—a prevalent biological and abstract coding scheme. The sigmoid-like dependence aligns with activation nonlinearity requirements for effective computation in layered SNNs or neuromorphic systems.

5. Device Modeling Assumptions, Range of Validity, and Tuning

Analytical treatment assumes idealized, piecewise-linear S-switch behavior with instantaneous ON/OFF transitions at fixed thresholds ($U_{\mathrm{th}}$, $U_n$), and lossless capacitors. Accurate operation requires $R_{\mathrm{ctl}} \gg R_{\mathrm{on}}$ and $R_{\mathrm{ctl}} \ll R_{\mathrm{off}}$, constraining the range of resistor values for which the sigmoid model holds. In practical circuits, switching elements may exhibit finite switching time, non-instantaneous I–V branches, and device-to-device variability in threshold and hysteresis properties.

The steepness and dynamic range of the sigmoid activation can be adjusted by the $C_2/C_1$ capacitance ratio; larger $C_2/C_1$ yields a more pronounced sigmoid and a larger central (steep) region. The fit between the analytical/fitted activation and realized frequency over $R_{\mathrm{ctl}}$ is excellent—from ∼50 Ω to ∼300 Ω with the cited component values—beyond which the model saturates or diverges from hardware response.

6. Benefits, Limitations, and Interface Considerations

Benefits:

• Implementation of analog sigmoid-like activation natively in hardware, obviating the need for digital approximation.
• Mechanism for direct firing rate (frequency) coding, compatible with spike-based neuromorphic architectures.
• Compactness: only one S-type nonlinear element, two capacitors, and one resistor per neuron.
• Straightforward interfacing with variable resistance elements, including sensors and adjustable transistor channels.

Limitations:

• Susceptibility to device variability, drift in $U_{\mathrm{th}}$, $U_n$, and hysteresis widths across elements.
• Requirement for precise and stable current sources, challenging for large-scale arrays.
• Power consumption and switching time limitations for S-elements (e.g., VO$_2$ memristors).
• Circuit noise and fabrication mismatch may broaden transition region or distort sigmoid, necessitating calibration.

Input interfacing can be realized via multiple schemes: digital-to-analog-controlled MOSFETs, integrating capacitor charge injection, or thermal coupling for resistively programmable control.

7. Relation to Other Oscillatory and Non-Monotonic Activation Functions

No explicit Lee Oscillator Activation Function is described in literature surveying oscillating activation functions for artificial neurons, such as the Sine Unit, NCU, GCU, or SQU, nor is the term attributed to a named author Lee in the corresponding mathematical or algorithmic activation function context (Noel et al., 2021). The Lee oscillator activation is a hardware-firing-frequency transfer function approximating a sigmoid, not a non-monotonic or oscillatory activation of the type used for multi-hyperplane single-neuron classification or XOR tasks in standard artificial neural networks.

A plausible implication is that the Lee oscillator function is distinct from multi-zero oscillating activation functions explored for enhanced single-neuron expressiveness but specifically tied to neuromorphic physical implementations for rate-coded analog computation. The circuit may inform future developments in mixed-signal or analog VLSI SNNs, particularly where compact, tunable, and physically native sigmoid-like nonlinearities are desired.