Threshold-Based Ternary Function in ML and Hardware

• Threshold-based ternary functions map real inputs to one of three discrete values (+1, 0, -1) using thresholding, enabling a balance between fidelity and resource efficiency.
• They are integral to neural network quantization and hardware logic, where optimal threshold selection minimizes approximation error and maximizes sparsity.
• Adaptive and data-driven thresholding methods further improve performance in deep learning and neuromorphic systems by optimizing trade-offs in energy, memory, and accuracy.

A threshold-based ternary function is a mathematical mapping—widespread in neural network quantization, hardware logic, compressed sensing, neuromorphic systems, and the combinatorics of words—that maps a real (or integer) variable to one of three discrete values according to one or more threshold parameters. In prototypical form, given a threshold Δ > 0 and an input value x, the function assigns +1 if x exceeds Δ, −1 if x is below −Δ, and 0 otherwise. This structure underpins much of contemporary ternary quantization and multi-valued logic, offering a principled trade-off between representation fidelity and resource efficiency.

1. Mathematical Formulation of Threshold-Based Ternary Quantizers

Threshold-based ternary mapping typically takes the following form. For a real input $x$ (or vector $W$), and threshold parameter(s) $\Delta$: $$f(x \mid \Delta) = \begin{cases} +1, & x > \Delta \ 0, & |x| \leq \Delta \ -1, & x < -\Delta \end{cases}$$ This function's use in neural network quantization is canonical in Ternary Weight Networks (TWNs), where full-precision weights $W$ are approximated by ternary values $\tilde{W} \in \{-1, 0, +1\}^n$, with a scaling factor $\alpha \geq 0$, such that the quantization minimizes the $\ell_2$ error: $$(\alpha^*, \tilde{W}^*) = \arg\min_{\alpha \ge 0,\,\, \tilde{W}_i \in \{\pm 1, 0\}} \| W - \alpha \tilde{W} \|_2^2$$ This reduces to the explicit threshold rule above, followed by an optimal setting of $$\alpha = \frac{1}{|I_\Delta|} \sum_{i \in I_\Delta} |W_i|$$, where $I_\Delta = \{i : |W_i| > \Delta\}$ (Li et al., 2016). The selection of $\Delta$ can itself be heuristic or adaptive, with expressions such as $\Delta \approx 0.75\,\text{mean}(|W|)$ commonplace.

In other domains, such as hardware logic, a multi-threshold variant partitions the domain into three or more output regions, e.g., for ternary logic gates with two thresholds $T_1<T_2$: $$f(x) = \begin{cases} 0, & x < T_1 \ 1, & T_1 \leq x < T_2 \ 2, & x \geq T_2 \end{cases}$$ Such mappings generalize the principle to integer-valued trits and manifest in both CMOS and CNFET logic circuits (Unutulmaz et al., 2022, Tabrizchi et al., 2018).

2. Principle and Optimization in Ternary Neural Network Quantization

In quantized deep neural architectures, ternary mapping with thresholding is central to achieving aggressive compression and computational efficiency without prohibitive loss of accuracy. The process is characterized as a two-parameter optimization:

• For fixed $\Delta$, minimize the $\ell_2$ error in the quantized representation by finding the optimal $\alpha$.
• For fixed $\alpha$, select $\Delta$ to balance sparsity (fraction of zeros) and representation accuracy.

The typical workflow in a TWN consists of:

1. Compute $\Delta = 0.75\,\frac{1}{n} \sum_{i=1}^n |W_i|$ for each filter.
2. Apply $f(W_i|\Delta)$ to produce ternarized weights.
3. Set $\alpha$ by closed-form.
4. Use $\alpha\,\tilde{W}$ in the forward and backward passes.

Refinements such as percentile-based thresholds (controlling sparsity directly) (Liu et al., 2022), and gradient-aware regularization terms, enhance convergence and task accuracy. The loss function is strictly the standard task loss (e.g. cross-entropy), with quantization applied outside the backward pass (Li et al., 2016, Liu et al., 2022).

3. Extensions: Hardware Implementations and Multi-Valued Logic

Threshold-based ternary functions are foundational in hardware multiple-valued logic (MVL). A common template is the ternary threshold logic gate (TTLG), which implements: $$f(x_1,\dots,x_n) = \begin{cases} 0, & S < T_1 \ 1, & T_1 \leq S < T_2 \ 2, & S \geq T_2 \end{cases}, \quad S = \sum_{i=1}^n w_i x_i$$ Thresholds $T_1$, $T_2$ are realized via circuit parameters (e.g., device sizing or CNFET channel diameter), directly partitioning signal ranges into three logic states. CNFET ternary gates exploit the relationship $V_\mathrm{th} \propto 1/D_\mathrm{CNT}$, allowing precise engineering of thresholds through material properties (Tabrizchi et al., 2018). TTLGs can serve as drop-in replacements for standard CMOS designs, yielding energy, area, and delay reductions while preserving arithmetic and control functionality (Unutulmaz et al., 2022).

4. Adaptive and Data-Driven Thresholding Methodologies

Adaptive or jointly-learned thresholds are employed to maximize representational efficiency and task performance. In loss-aware ternary quantization, thresholds can be set as per-layer percentiles of $|W|$ and made learnable via gradient descent, even with associated scaling factors subject to analytical updates (Liu et al., 2022).

Soft thresholding, as in Soft Threshold Ternary Networks (STTN), sidesteps explicit threshold parameters by constructing ternary outputs as the sum of two correlated binary masks, leading to improved approximation error and state-of-the-art accuracy on large-scale classification tasks (Xu et al., 2022). This demonstrates that threshold-based ternary quantization is not limited to rigid hard-threshold functions, but can encompass adaptive, data-dependent mechanisms that emerge during stochastic optimization.

5. Beyond Neural Computation: Applications in Neuromorphic Systems and Combinatorics

Threshold-based ternary encoding is also integral to energy-efficient spiking neuromorphic architectures. The threshold-adaptive encoding (TAE) approach produces a ternary spike train via dynamic thresholds, with the rule: $$s(t) = \begin{cases} +1, & x(t)-u(t) \geq \Delta(t) \ -1, & x(t)-u(t) \leq -\Delta(t) \ 0, & \text{otherwise} \end{cases}$$ followed by immediate update of the baseline and threshold, ensuring the encoding adapts to local input statistics. This event-driven, online adaptation leads to orders-of-magnitude gains in inference energy and memory utilization in quantized ternary spiking neural networks (QT-SNN) (Wang et al., 7 Jul 2024).

In combinatorics and the study of infinite words, threshold-based ternary functions arise via the finite-repetition threshold (FRt) in the avoidance of word exponents above certain values. The mapping from substrings to trits is governed by the minimal periods of repetition, with sharp thresholds (e.g., $r(3)=7/4$ for ternary alphabets) delineating combinatorial regimes (Badkobeh et al., 2011).

6. Theoretical and Biological Perspectives

Threshold-based ternary computation extends to theoretical neuroscience and quantum computation models. Johnson and Winlow introduce a threshold-based phase ternary scheme for the brain, where logic states are tied to sub-threshold, at-threshold, and refractory states of the neuronal membrane. This tripartite partition, determined by the membrane potential crossing discrete thresholds, enables phase-encoded computation beyond the classical Turing framework (Johnson et al., 2020). The resulting operational units are qutrits, with threshold crossings forming computational fixed points.

7. Tabular Synthesis of Ternary Threshold Functions Across Domains

Domain	Input/Threshold Structure	Output Mapping
Neural Network Quantization	$W_i$, single $\Delta$	$\{-1,0,+1\}$ via $f(W_i|\Delta)$
MVL Hardware (TTLG)	$\sum w_i x_i$, $T_1,T_2$	$\{0,1,2\}$
Neuromorphic TAE	$x(t)$, dynamic $\Delta(t),u(t)$	$\{-1,0,+1\}$
Combinatorics on Words	substring period/exponent, $r(a)$	$\{-1,0,+1\}$ patterns
Theoretical Neuroscience	$V(t)$, activation thresholds	$\{-1,0,+1\}$ (rest/threshold/refractory)

The threshold-based ternary function is thus a unifying, cross-domain abstraction for discretizing real-valued signals, weights, and states into trivalent representations, with thresholds playing a central role in controlling the sparsity, expressivity, and operational invariants across domains ranging from deep learning and digital logic to theoretical biology and combinatorics.