--- title: Quantized Neurons url: https://www.emergentmind.com/topics/quantized-neurons type: topic --- # Quantized Neurons Quantized neurons are neural units whose state, parameters, responses, or governing operators are constrained to discrete sets rather than unconstrained continuous values. In contemporary research, the term spans several non-equivalent traditions: low-bit electronic neurons in quantized neural networks, spiking and neuromorphic neurons implemented with fixed-point state updates or multi-level spikes, optical neurons with discretized phase or amplitude settings, and quantum-mechanical constructions in which neurons are represented by qubits, roots of unity, Hamiltonians, or quantum particles [1912.12656] [2310.03049] [2607.05000]. The shared theme is discretization, but the substrate, training procedure, and meaning of “quantization” differ substantially across these literatures. ## 1. Terminology and conceptual scope The literature uses “quantized neurons” in at least four distinct senses. In mainstream deep learning, quantization usually means that weights, biases, and sometimes activations are represented with finite bitwidth and processed with fixed-point or bitwise arithmetic. In optical computing, quantization refers to bounded and discrete physical settings of optical neurons, such as phase values in diffractive layers. In spiking and neuromorphic systems, quantization may refer to fixed-point membrane states, discretized spike phases, or multi-level burst outputs. In quantum machine learning and quantum neuromorphics, it may refer either to qubit-based neuron constructions or to canonical quantization of a neuron’s energy function into a Hamiltonian observable [1912.12656] [2310.03049] [2404.02248] [2305.02018]. | Usage | Quantized object | Representative papers | |---|---|---| | Electronic QNN | Weights, biases, activations | [1912.12656], [2306.13793] | | Spiking / neuromorphic | Membrane states, spike phase, burst levels, fixed-point synapses | [2404.18066], [2404.02248], [2604.25688], [2409.18396], [2410.04992] | | Optical / photonic | Phase or amplitude settings of optical neurons | [2310.03049] | | Quantum / quantum-inspired | Qubit states, roots of unity, Hamiltonians, quantum particles, memristive quantum circuits | [1711.11240], [2503.15482], [2605.24386], [2506.21363] | A recurrent source of ambiguity is that “quantized” can denote low numerical precision or genuine quantum-mechanical quantization. One paper explicitly distinguishes standard machine-learning quantization from neurons that are “genuinely quantum-mechanical—quantized degrees of freedom with path-integral dynamics, tunneling, and thermal fluctuations” [1806.09664]. Another body of work instead treats quantization as canonical quantization: a classical neuron is written as an activation applied to an energy function, then the energy is replaced by a quantum Hamiltonian and the activation is applied by matrix functional calculus [2607.05000]. ## 2. Low-precision electronic quantized neural networks In the low-precision deep-learning literature, a quantized neuron is typically a neuron whose weights and/or activations are drawn from a finite set of levels, enabling reduced memory footprint and cheaper arithmetic. A canonical formulation represents binary weights through a learned scale, $$ x_b = B(x) = \frac{\lVert x \rVert_1}{n}\,\mathrm{sign}(x), $$ and handles the non-differentiability of binarization with a straight-through estimator, $$ \frac{\partial L}{\partial x} \approx \frac{\partial L}{\partial x_b}\cdot \mathbb{I}\{|x|<1\}. $$ For $k>1$, a uniform quantizer on $[0,1]$ is used, $$ Q_k(x)=\frac{1}{2^k-1}\left\lfloor (2^k-1)x \right\rceil, $$ again trained with straight-through backpropagation [1912.12656]. A central result is that quantization need not be homogeneous across depth. In "Mixed-Precision Quantized Neural Network with Progressively Decreasing Bitwidth For Image Classification and Object Detection" [1912.12656], the assigned weight precision decreases stage-by-stage with depth, motivated by the observation that posterior layers operate on progressively more separable features. Representative schedules include VGG-7 with weights $8\rightarrow4\rightarrow2\rightarrow1\rightarrow1\rightarrow1$, ResNet-20 with $4\rightarrow2\rightarrow1$ or $8\rightarrow4\rightarrow2$, and AlexNet or ResNet-18 with $8\rightarrow4\rightarrow2\rightarrow1$. The paper reports that such mixed-precision schedules can reduce parameter memory by more than 30\% relative to homogeneous counterparts while maintaining or improving accuracy, and further reports that higher-precision bottom layers appreciably improve 1-bit networks whereas lower-precision posterior layers can act as a regularizer in $k$-bit networks [1912.12656]. This depth-aware view yields a more specific notion of a quantized neuron: quantization level becomes part of the neuron’s allocated representational capacity. The paper’s weight-quantization pipeline also applies $\tanh$ and per-output-channel normalization before uniform quantization, and notes that per-channel scaling can be folded into batch normalization so inference remains integer-only. On CIFAR-10, mixed-precision VGG-7 with average 1.06-bit weights and 2-bit activations achieved 93.22\%, versus 92.33\% for homogeneous 1/2-bit DoReFa and 92.48\% full precision; on ImageNet, mixed ResNet-18 with average 1.42-bit weights and 2-bit activations reached 65.03\% Top-1, outperforming DoReFa 2/2 at 62.60\% and PACT 2/2 at 64.40\% [1912.12656]. A distinct but related problem is that quantization can create a small set of pathological neurons whose activation status differs from that of the full-precision model. "QNNRepair: Quantized Neural Network Repair" [2306.13793] formalizes this failure mode. It models affine quantization as $$ r = S(q-Z),\qquad q=\operatorname{round}\!\left(\frac{r}{S}+Z\right), $$ and identifies neurons whose ReLU activation state flips after quantization. Statistical fault localization metrics such as Tarantula, Ochiai, Jaccard, Euclid, and Wong3 are used to rank suspicious neurons, after which an LP/MILP adjusts incoming quantized weights so that failing tests recover the full-precision activation sign while perturbations remain bounded. The paper reports that repaired models achieve 24\% higher accuracy than SQuant’s on the independent validation set, especially for ImageNet, and gives MobileNetV2 results improving from 65.86\% to 70.77\% on ImageNet after repair [2306.13793]. This suggests that, in practical deployment, quantized neurons are not merely lower-precision surrogates of floating-point neurons; they can become distinct error loci that require post-quantization diagnostics and correction. ## 3. Quantized spiking and neuromorphic neurons In spiking systems, quantization is often imposed on membrane states, synaptic weights, spike timing, or spike amplitude. A particularly hardware-oriented example is the quantized Context-Dependent LIF neuron, qCLIF, which retains a two-compartment structure—apical context and somatic stimulus—but replaces analog multiplications in the input path with digital spike-weighting through bitwise AND and integer addition [2404.18066]. Its discrete-time update is $$ V^{ap}(t+dt)=\max\{0,\,V^{ap}(t)-\alpha_{leak}+V^{ap}_{input}(t+dt)\}, $$ $$ V^{som}(t+dt)=V^{som}(t)-\beta_{leak}+\mathrm{ReLU}(V^{ap}(t+dt))\cdot V^{som}_{input}(t+dt), $$ with a hard reset when threshold $\theta$ is exceeded. Implemented in 45 nm, a single 8-bit qCLIF occupies approximately $900\,\mu m^2$ and the paper reports 90\% accuracy on DVS Gesture despite 8 bit quantization, with scalability to 82k synapses in a $1.86\,\mathrm{mm}^2$ footprint [2404.18066]. A broader digital framework is QUANTISENC, which exposes neuron precision directly as a configurable hardware parameter. Here, LIF neurons with current-based synapses operate in signed fixed-point $Qn.q$ format, and the membrane update is implemented as $$ V[t+1] = V[t] + \text{grow\_rate}\cdot \text{act}[t] - \text{decay\_rate}\cdot V[t]. $$ The architecture supports multiple reset modes, configurable refractory periods, all-to-all, one-to-one, and Gaussian connectivity, and uses distributed synaptic memory to sustain pipelined execution [2404.02248]. In the reported configurations, Q5.3 offered the best area–power–accuracy trade-off on MNIST, reaching 96.5\% hardware accuracy with RMSE 0.43 mV relative to software, while Q9.7 achieved 97.1\% and Q3.1 dropped to 88.3\% [2404.02248]. In this setting, a quantized neuron is literally a digital arithmetic datapath whose numerical format determines spike timing fidelity, saturation behavior, and power. More recent work shifts from binary spikes to multi-level spike outputs. QB-LIF interprets burst spiking as saturated uniform quantization of membrane potential with a learnable per-layer scale, $$ Q(u;s,N_{\max})=\operatorname{clip}\!\left(\left\lfloor \frac{u}{s}\right\rfloor,0,N_{\max}\right),\qquad b^{(\ell)}[t]=Q(U^{(\ell)}[t];s^{(\ell)},N_{\max}), $$ with emitted amplitude $S^{(\ell)}[t]=s^{(\ell)}b^{(\ell)}[t]$ [2604.25688]. The learned scale is absorbed into downstream weights during inference so execution remains accumulate-only. The paper reports consistent gains over binary and fixed-burst SNNs under short horizons, including 95.40\%, 96.21\%, and 96.43\% on CIFAR-10 with ResNet-20 at $T=1,2,4$, and 73.14\% on ImageNet with ResNet-34 at $T=4$ [2604.25688]. Here the quantized neuron is no longer defined by low precision alone, but by its controlled discrete information throughput per timestep. Another line of work uses quantization to regularize activity rather than merely compress arithmetic. In a bio-inspired olfactory model, analog ET activations are first $l_1$-normalized so that $$ \sum_{i=0}^{n-1} a_i = C, $$ then mapped to duplicated spiking principal neurons with heterogeneous ET$\rightarrow$MC gains [2409.18396]. Each duplicated MC neuron spikes iff $$ a_i\,w_j \ge t, $$ and spike phase is quantized into 50 bins per oscillation cycle. The paper shows that, with heterogeneous gains chosen to emulate uniformly spaced effective thresholds, the fraction of active MCs is approximately yoked to network constants, $$ \frac{Q}{nm}\approx \frac{C}{nu}, $$ which regularizes layer utilization under open-set concentration and saturation variability [2409.18396]. This is a distinct meaning of a quantized neuron: the neuron becomes a discretized phase encoder with deliberately heterogeneous transfer characteristics. A related application-specific architecture is MC-QDSNN, where the MCLeaky neuron introduces multi-compartment dendritic recurrence as an intrinsic memory mechanism. Its quantized form, QMCLeaky, fixes decay constants to 0.875 for hardware simplicity and quantizes thresholds, states, and synapses to 8 bits [2410.04992]. The recurrent compartmental dynamics are $$ V_{d2}^{t} = \alpha_{d2}V_{d2}^{t-1} + \alpha_{d1}V_{d1}^{t-1} + I^{t}, $$ $$ V_{d1}^{t} = \alpha_{d1}V_{d1}^{t-1} + \beta_{d2}V_{d2}^{t} + \alpha_{s}V_{s}^{t-1}, $$ $$ V^{t} = \alpha_{s}V_{s}^{t-1} + \beta_{d1}V_{d1}^{t} - (reset\times \vartheta), $$ $$ S^{t} = \mathrm{SpikeGen}(V^{t}-\vartheta). $$ On stress detection from physiological signals, the full-precision MCLeaky model reached 98.8\% on EDA Combo, while the quantized QMCLeaky+QFC model reached 91.84\% and delivered reported energy savings of 25.12x to 39.20x and EDP gains of 52.37x to 81.9x over ANN baselines [2410.04992]. ## 4. Optical neurons with discrete physical parameters In optical processors, a quantized neuron is a physically realizable optical modulation element whose phase and/or amplitude can take only a finite number of values. QuATON studies diffractive deep neural networks in which each optical neuron applies a phase-only modulation, $$ E^{n}_{out}[x,y] = E^{n}_{in}[x,y]\exp\!\left(j\,Q_s(\varphi_n[x,y],\tau_n)\right), $$ while free-space propagation between layers is modeled by Rayleigh–Sommerfeld diffraction [2310.03049]. Hard quantization of a bounded parameter $x\in[l,u]$ with $N$ levels uses $$ Q_h(x)= \begin{cases} l,& x