Primitive-specific MLP Design

• Primitive-specific MLP is an artificial neural network where every arithmetic operation and activation function is custom-designed to meet the constraints of specific hardware substrates.
• Its design leverages quantization of weights to powers-of-two, pruning of accumulator trees via genetic algorithms, and low-bit approximations of activations to substantially reduce area and power consumption.
• The methodology enables modular assembly of complex architectures from domain-tailored primitives, facilitating practical neural inference in ultra-resource-constrained environments.

A primitive-specific multilayer perceptron (MLP) is an artificial neural network in which the constituent computational primitives—multipliers, accumulators (adders), and activation circuits—are co-designed and custom-generated for the constraints, statistics, and hardware characteristics of a particular deployment substrate. This approach departs from the conventional MLP construction utilizing generic, library-based arithmetic and activation primitives, instead leveraging targeted hardware-aware approximations and optimizations to realize highly area- and power-efficient networks suited for ultra-constrained domains, such as printed electronics or resource-limited CMOS (Afentaki et al., 2023). Conceptual and algorithmic frameworks for primitive-specific and algebraically structured MLP composition also support modular assembly of complex architectures from domain-relevant primitive elements (Peng, 2017).

1. Definition and Scope

In the context of MLPs, "primitive-specific" denotes a hardware/software co-design methodology whereby every arithmetic operation (multiplication, accumulation) and each activation function is not generically implemented but is individually tailored for the requirements of the target platform and the statistical properties of the trained model. In printed electronics, for example, off-the-shelf digital arithmetic primitives are infeasible because of large feature sizes (~5 μm), high transistor capacitances, and stringent power/area budgets. A primitive-specific MLP may replace floating-point multipliers with wire shifts if weights are quantized to signed powers of two; collapse redundant accumulator bit-slices identified via analysis of network behavior; and approximate activation functions—such as ReLU or Argmax—using only those logic bits necessary to preserve accuracy within explicit thresholds (Afentaki et al., 2023). The term also admits a broader algebraic interpretation, where modular MLPs are assembled from characteristic "primitive" nets corresponding to set-theoretic or logical decompositions of the target problem domain (Peng, 2017).

2. Architectural Principles

Primitive-specific MLPs are characterized by bespoke, often bit-level optimized hardware logic that replaces or radically simplifies generic operations:

• Multipliers: When network weights are quantized to the nearest signed power-of-two, the multiplication $x_i w_i$ reduces to a wire shift—allowing the elimination of conventional multiplier cells (Afentaki et al., 2023).
• Accumulators: Custom adder trees are generated where unnecessary partial sum bits—those that do not influence final classification outcomes—are pruned. Genetic optimization algorithms (such as NSGA-II) are employed to identify minimal sets of adder bits that suffice for target accuracy constraints, further reducing area and switching activity.
• Activation Functions: Activations such as ReLU are implemented as low-bitwidth quantized functions (e.g., QReLU, clipped to 8 bits using a few gates). Argmax operations are approximated by pairwise comparisons of only the most discriminative bits, determined via accuracy-preserving bit-masking strategies (Afentaki et al., 2023).

This architectural discipline generalizes beyond printed FETs to domains such as near-threshold CMOS, MRAM-in-logic, and memristor crossbar arrays, where similar constraints demand co-optimization at the primitive level.

3. Automated Framework for Primitive-Specific MLP Construction

An end-to-end pipeline for primitive-specific MLP design consists of four major stages (Afentaki et al., 2023):

• Neural Architecture Specification: Starting from a trained floating-point MLP or a training script (e.g., in PyTorch/Keras), the input and training set are normalized, and a canonical architecture with (for example) a single hidden layer and ReLU activations is adopted.
• Bespoke Circuit Generation:
  • Multiplier Approximation: Each weight $w_i$ is quantized to $\tilde w_i = \mathrm{sign}(w_i) 2^{k_i}$, with $k_i=\arg\min_{k} |w_i - 2^k|$. Products $x_i \tilde w_i$ are implemented as logical shifts.
  • Activation Approximation: QReLU clamps activations to 8 bits ($$\mathrm{QReLU}(u) = \min\{\max(0, u), 2^{B_\text{act}}-1\}$$, $B_\text{act}=8$); Argmax comparators are masked for bit efficiency.
  • Accumulator Approximation: Genetic algorithms search over adder-mask chromosomes minimizing the number of full-adders (area estimate) and constraining accuracy loss (typically $<$5% on training set).
• Integration and Hardware Synthesis: For each candidate design on the Pareto front, bespoke multipliers, pruned accumulators, and approximated activation logic are synthesized and mapped to the target hardware library, such as printed EGFET cells.
• Evaluation: Physical area (from synthesis), power consumption (from static analysis), and end-to-end test-set accuracy are recorded for all candidate designs.

The following table summarizes a subset of empirical results for MLP implementations under this framework (Afentaki et al., 2023):

Dataset	Baseline Area [cm²]	Full-Approx Area [cm²]	Baseline Power [mW]	Full-Approx Power [mW]	Test Acc. Loss
Arrhythmia	266	13.5	998	12.8	–3.2%
Breast Cancer	12.0	0.08	40	0.08	–1.9%

These results indicate area decreases of up to $151\times$ and power reductions of $808\times$ for accuracy losses within $w_i$0.

4. MLP Algebra and Primitive Decomposition

An alternative perspective on primitive-specific MLPs is provided by the MLP algebra framework (Peng, 2017). Here, "primitive" refers to characteristic MLPs (nets approximating the indicator function of a simple domain), and complex MLPs are synthesized by applying algebraic operations (SumNet, IProductNet, DifferenceNet, OProductNet) to such primitives, mirroring set-theoretic or logical composition:

• Complementary Net: The complement of an MLP flips the sign of the last layer's weights and biases, effecting $w_i$1 for a binary classifier.
• SumNet: Constructs a $w_i$2-layer net implementing the union (logical OR) of the domains learned by two $w_i$3-layer nets.
• IProductNet: Implements the Cartesian product (AND) of networks, facilitating separate learning for subdomains or features.
• Characteristic MLP: Given domain $w_i$4, one can train an MLP such that $w_i$5 on $w_i$6, $w_i$7 elsewhere.

These operators support a structured, deterministic assembly of MLPs reflecting the compositional logic of the underlying data, and pseudocode for each is explicitly provided in (Peng, 2017). Design guidelines specify decomposing a target domain into elementary regions (primitives) and reassembling the full classifier MLP through such algebraic construction.

5. Methodological Trade-Offs and Hardware Generalization

Primitive-specific MLPs are not limited to printed electronics; their co-design philosophy and circuit-level optimization are pertinent to any domain dominated by resource constraints. The key methodological elements (Afentaki et al., 2023):

1. Quantize weights to hardware-friendly representations (e.g., power-of-two, small LUT, stochastic).
2. Prune accumulator trees without exceeding a prescribed accuracy loss.
3. Simplify activation logic as dictated by statistics of network activations.
4. Search the design space (via genetic algorithms or others) for optimal area/power/accuracy trade-offs.

This methodology is extensible to near-threshold CMOS, MRAM-in-logic, crossbar accelerators, and related substrates, enabling practical MLP inference in ultra-resource-constrained scenarios.

6. Significance, Applications, and Implications

Primitive-specific MLP construction enables the realization of machine learning inference on previously inaccessible hardware platforms by performing holistic, bit-level co-optimization of neural network primitives. In printed electronics (PE), this approach brings battery-powered, area- and energy-feasible neural classifiers within reach—demonstrated, for example, by reducing a (274,5,16) MLP for arrhythmia classification from 266 cm²/998 mW to 13.5 cm²/12.8 mW with less than 5% accuracy loss (Afentaki et al., 2023). Furthermore, the MLP algebraic assembly provides a deterministic pipeline to engineer complex architectures from functional primitives, eliminating ad hoc trial-and-error in architecture search (Peng, 2017).

A plausible implication is that as new hardware substrates emerge—each with distinct constraints—a primitive-specific approach, blending circuit-level co-design and compositional algebraic structure, will be critical in exploiting the full expressive capacity of deep learning within strict physical and operational boundaries.