smallNet: Compact Edge Neural Networks

• smallNet is a class of compact neural network designs that optimize parameters, memory traffic, and energy for edge and embedded systems.
• Key design principles include minimal 3x3 filters, squeeze bottlenecks, and hardware-aware evaluation to meet strict storage and latency constraints.
• Approaches such as structured pruning, quantization, and adaptive synthesis yield effective trade-offs between size, speed, and accuracy.

smallNet denotes a class of deliberately compact neural-network designs optimized for constrained deployment, and in the cited literature the term also appears as the proper name of specific models for closed-loop EEG-BCI and tiny-FPGA inference. Across these uses, the central objective is joint control of parameter storage, memory traffic, inference latency, and energy while retaining task-level accuracy under embedded constraints such as few-megabyte memory budgets, batch-size-1 inference, and limited arithmetic throughput (Iandola et al., 2017, Ortega et al., 2018, Bascuñán et al., 29 Sep 2025).

1. Scope, constraints, and design objectives

In embedded and edge settings, “small” refers primarily to few parameters and a small model storage footprint. The keynote on small neural nets frames the target regime as models with on the order of $\leq 2$M parameters and total storage around a few MB, with representative constraints including an 8192 KB SRAM budget for parameters and intermediate activations, a minimum of 15 FPS and preferably 30 FPS, and less than 2 Joules per frame (Iandola et al., 2017). The same source emphasizes that off-chip DRAM access costs approximately $100\times$ the energy of an arithmetic operation, so parameter size and memory traffic are first-order design variables rather than secondary implementation details.

This constraint profile explains why smallNet research repeatedly couples architectural minimization with hardware-aware evaluation. In edge-device image recognition, specialized deployments may require only a small number of target classes, often fewer than 10; under those conditions, the classifier head and final convolution can be reduced substantially without altering the upstream feature extractor (Shafiee et al., 2017). In EEG-BCI, compactness is tied to rapid retraining and online use under closed-loop timing constraints rather than only storage minimization (Ortega et al., 2018). In tiny FPGAs, compactness additionally means fitting logic, BRAM, and DSP budgets while avoiding complex software stacks and vendor IP dependencies (Bascuñán et al., 29 Sep 2025).

A recurring misconception is that smallNet design is reducible to parameter count alone. The cited work instead treats storage, compute, energy, and deployment friction as coupled quantities. SqueezeNet, for example, drastically reduces parameters, but later work explicitly notes that this does not automatically imply a commensurate reduction in total computation (Iandola et al., 2017).

2. Architectural principles for compact networks

The most explicit playbook for smallNet design in the cited material is based on SqueezeNet and its descendants. SqueezeNet uses “Fire” modules consisting of a squeeze layer of $1\times1$ convolutions and an expand layer combining $1\times1$ and $3\times3$ convolutions. Three design rules are emphasized: decrease the number of $3\times3$ filters, decrease the number of input channels to $3\times3$ filters via the squeeze bottleneck, and downsample late in the network (Iandola et al., 2017). The parameter count of a Fire module is given as

$$\text{params}_{\text{Fire}} = C_{in} \cdot s + s \cdot e_{1} + 9 \cdot s \cdot e_{3}.$$

This makes the cost of the $3\times3$ branch explicit and formalizes why replacing large fractions of $3\times3$ filters with $100\times$0 filters is effective.

These principles yielded SqueezeNet with 1.2M parameters and 4.8 MB in FP32, approximately $100\times$1 smaller than AlexNet, and after Deep Compression as little as 480 KB (Iandola et al., 2017). The same playbook recommends replacing fully connected layers with convolutional layers, spacing downsampling evenly, considering depthwise separable or group convolutions, using shuffle operations when group structure is pervasive, and applying pruning, quantization, Huffman coding, or distillation as later stages.

SquishedNets extend the same logic to few-class edge scenarios. Starting from SqueezeNet v1.1, the final convolutional layer conv10 is modified to produce exactly 10 output channels using 10 $100\times$2 filters for ImageNet-10, instead of 1000 filters. Because conv10 contains approximately 40% of all parameters in SqueezeNet, this single architectural change yields a significant reduction in model size while leaving the macroarchitecture unchanged up to conv10 (Shafiee et al., 2017). This is a particularly clear example of a smallNet principle that is easy to overlook: when the number of classes is small, the classifier head can dominate the parameter budget.

The broader design-space methodology imported from embedded microprocessor design is equally important. It prescribes application-specific benchmarks and metrics, explicit enumeration of the design space, reproducible evaluation procedures, and systematic exploration of trade-offs such as accuracy versus size, energy, and latency (Iandola et al., 2017). This suggests that smallNet is best understood not as a single architecture family, but as a disciplined design regime.

3. Synthesis, pruning, and compression mechanisms

Several cited works treat smallNet construction as an optimization problem rather than a fixed manual architecture search. SquishedNets use deep evolutionary synthesis to generate progressively more efficient descendants of a modified SqueezeNet v1.1 ancestor. The paper formulates the synthesis process with a genetic encoding $100\times$3 and synthesis probability

$100\times$4

where the environmental factor model $100\times$5 is set to a value less than 1 so that synaptogenesis occurs in a resource-starved environment (Shafiee et al., 2017). Fifteen generations were performed, and the resulting models show a clear size–speed–accuracy trade-off.

Variant	Size and throughput	Top-1 accuracy
SquishedNet-1	2.4 MB, 156.09 images/sec	81.2%
SquishedNet-2	2.0 MB, 174.86 images/sec	79.6%
SquishedNet-3	1.3 MB, 225.35 images/sec	78.6%
SquishedNet-4	0.95 MB, 256.00 images/sec	77.0%

All four variants were evaluated on an Nvidia Jetson TX1 with batch size 32. The smallest model is $100\times$6 smaller than SqueezeNet v1.1 and $100\times$7 smaller than AlexNet, and these results were obtained without compression or quantization (Shafiee et al., 2017).

Smallify approaches the same problem from within training itself. It introduces a switch layer with per-neuron or per-channel gates $100\times$8, defined by

$100\times$9

and optimizes

$1\times1$0

The key point is structured pruning: entire neurons or channels are deactivated and removed on-the-fly, leaving dense but smaller tensors rather than sparse unstructured matrices. The paper reports that Smallify can reduce network size by $1\times1$1 with a $1\times1$2 improvement in inference time with similar accuracy, and on CIFAR-10 with VGG16 it obtained 92.07% accuracy, identical to the static network, while being $1\times1$3 smaller (Leclerc et al., 2018).

LilNetX frames smallNet training as a joint optimization of accuracy, storage rate, and computation. Its loss combines cross-entropy, a self-information penalty in a reparameterized latent space, and structured sparsity terms:

$1\times1$4

The method penalizes

$1\times1$5

and uses both an $1\times1$6 term and a group-lasso penalty over $1\times1$7 slices to induce unstructured and structured sparsity (Girish et al., 2022). The abstract reports up to 50% smaller model size and 98% model sparsity on ResNet-20 while retaining the same CIFAR-10 accuracy, and 35% smaller model size with 42% structured sparsity on ResNet-50 trained on ImageNet compared with existing state-of-the-art compression methods.

Taken together, these methods show three distinct routes to a smallNet: architecture synthesis under explicit resource scarcity, structured pruning during training, and latent-space compression with structured sparsification. A plausible implication is that “smallNet” research has shifted from manual architecture minimization toward jointly optimizing representation, topology, and hardware efficiency.

4. Augmented and dynamically reconfigurable smallNets

A separate line of work addresses the fact that very small networks often underfit. NetAug argues that tiny models should be augmented at the model level rather than regularized by noise injection. Instead of dropout or strong data augmentation, the tiny network $1\times1$8 is embedded as a deterministic sub-network inside a larger augmented model $1\times1$9, and training uses both the tiny-path loss and an auxiliary loss from a sampled larger subnet. At inference, only the tiny model is retained, so there is zero inference overhead (Cai et al., 2021).

The empirical results are consistent with the underfitting claim. On ImageNet, MobileNetV3 w0.35 rises from 58.1% to 60.3% top-1, MobileNetV2-Tiny rises from 51.7% to 53.3%, and larger models may not benefit, as ResNet-50 changes from 76.8% to 76.5% (Cai et al., 2021). Standard regularizers that help large models are reported to hurt tiny ones: dropout, Mixup, AutoAugment, RandAugment, and DropBlock all reduce MobileNetV2-Tiny accuracy. NetAug also improves transfer to object detection; for the same level of performance, it requires 41% fewer MACs on Pascal VOC and 38% fewer MACs on COCO than the baseline. This makes NetAug a training-time smallNet method: it does not alter the deployed architecture, but it changes how a tiny model is supervised.

NaviSlim pushes adaptivity to runtime. It is a gated slimmable network for micro-UAV navigation that dynamically selects both a slimming factor and sensor power levels based on context. In its formulation, the active parameter count in an MLP backbone is dominated by an $1\times1$0 term, so shrinking width yields quadratic savings in active parameters and FLOPs. The system uses an auxiliary gating network trained with TD3 to choose either compute width or sensing level, while a distilled navigation backbone remains robust across widths and sensor settings (Johnsen et al., 2024).

The reported reductions are substantial relative to static worst-case designs. On test scenarios, NaviSlim shows dynamic reduced model complexity on average between 57% and 92%, and sensor utilization between 61% and 80% (Johnsen et al., 2024). The paper also notes an important caveat: for small backbones, gating overhead can dominate, so dynamic slimming does not automatically outperform a static network. This is another reminder that smallNet design is deployment-specific; control overhead can erase theoretical savings if the base model is already very small.

5. SmallNet for closed-loop EEG-BCI

In the EEG-BCI literature, SmallNet is a specific compact convolutional architecture for four-class, self-paced, closed-loop control in the Cybathlon BrainRunners game. The network consists of a single convolutional layer, one fully connected layer, and a final logistic regression classification layer that outputs a 4-way softmax. The input is a topographical spectral tensor $1\times1$1 built from 64 EEG electrodes recorded at 500 Hz, using 1.2 s windows, 300 ms segments with 75% overlap, a Hamming window, Welch’s periodogram, and 2D topographic projection of electrode power values (Ortega et al., 2018).

The system is explicitly personalized. Eight candidate mental tasks were considered, and 70 models covering all $1\times1$2-choose-$1\times1$3 combinations were trained. For the volunteer discussed in the paper, the selected set was right hand, feet, relax, and mental humming, with mean offline test accuracy 46.85%, while the top-ranked combination right hand, feet, lips, and numbers reached 55.01% (Ortega et al., 2018). Kruskal–Wallis tests with Bonferroni correction showed no absolute winner across all subjects, which justified user-specific selection.

The principal operational result is online rather than offline. In architecture selection, SmallNet achieved 41.8 $1\times1$4 1.3% test accuracy offline, SmallNet+1CL achieved 46.5 $1\times1$5 8.6%, and SmallNet+1FC achieved 41.1 $1\times1$6 1.9%. During online play, SmallNet reached 47.6 $1\times1$7 6.6% for four classes, above the 25% chance level, with a decision every 300 ms (Ortega et al., 2018). Perfect decoding would yield approximately 60 s completion time, whereas the volunteer averaged approximately 147 s, similar to the 140–180 s reported for the Cybathlon BCI race.

The paper’s strongest methodological point is the offline-to-online discrepancy. Cross-validated test accuracies obtained from video-recorded offline data systematically overestimated online decoding accuracy, with $1\times1$8, whereas models trained adaptively on game-recorded data better matched real-time behavior (Ortega et al., 2018). Non-adaptive training showed a $1\times1$9 drop from offline to online, while adaptive training reduced the drop to $3\times3$0. This suggests that, in BCI contexts, “smallNet” is not only an architectural descriptor but also part of a rapid-update protocol designed to remain usable in closed-loop conditions.

6. smallNet as a tiny-FPGA convolution engine

The 2025 paper titled “smallNet: Implementation of a convolutional layer in tiny FPGAs” uses the term for a hand-coded, Verilog-only implementation of the first stage or stages of a CNN targeted at very small Zynq devices and related SoMs, SoCs, and ASIC-oriented flows (Bascuñán et al., 29 Sep 2025). The design avoids vendor IP cores inside the convolution engine and is organized around custom modules for window generation, multiply–accumulate, bias addition, sigmoid activation, max pooling, a dense layer, and a simple control FSM. Its convolution is realized as a filter-polynomial-like structure, with streaming windows and line buffers rather than a software framework or high-level synthesis pipeline.

The implemented convolution follows the standard discrete 2D form

$3\times3$1

and the paper also gives the corresponding $3\times3$2-transform view

$3\times3$3

All parameters and activations use 32-bit fixed-point two’s-complement, with weights and biases extracted from a trained Keras model and converted to binary.

The MNIST topology is deliberately minimal: Conv1 with $3\times3$4, stride 1, same padding, one input channel and one output channel; MaxPool1; Conv2 with the same kernel configuration; MaxPool2; Flatten to 49; Dense to 10 outputs; and Max Finder. The full model has approximately 510–550 trainable parameters, corresponding to approximately 1.99 KB of weights (Bascuñán et al., 29 Sep 2025).

Metric	Reported value
LUTs	2052 of 14,400
Flip-flops	1587
BRAM	~25 KB
DSP slices	48
Hardware inference time	109 ms
Speedup vs PS-side software	~5.1×
Total on-chip power	1.505 W
Hardware accuracy	≈81% on 23 test images

The implementation was validated on a single-core Cora Z7 using AXI-DMA between the processing system and programmable logic. The paper reports average PS-side Python/TensorFlow/Keras inference time of 560 ms on the ARM Cortex-A9 at 667 MHz, versus 109 ms in hardware, yielding about 9.17 frames/s end-to-end instead of 1.79 frames/s (Bascuñán et al., 29 Sep 2025). Accuracy declined from 93.47% in CPU software on the evaluated subset to 88.03% in post-synthesis simulation and approximately 81% in hardware on 23 DMA-fed test images, which the paper attributes to fixed-point quantization, single-filter layers, and the very small model capacity.

This hardware smallNet illustrates a different but complementary meaning of compactness. Here the goal is not just a compact learned model, but a compact implementation substrate: no high-level Python co-development, no IP-core dependence inside the convolution engine, modest LUT and DSP use, and a path toward SoM, SoC, and ASIC deployment (Bascuñán et al., 29 Sep 2025). A plausible implication is that smallNet, in this sense, is as much a digital design methodology as a neural-network architecture.

Across these literatures, smallNet emerges as a convergent research theme rather than a single canonical model. It encompasses architecture rules such as kernel and channel reduction, training methods that compensate for underfitting or learn width and sparsity directly, domain-specific compact models such as the EEG-BCI SmallNet, and hand-coded hardware realizations for tiny accelerators. What unifies these otherwise diverse systems is the decision to treat model size, runtime behavior, and deployment mechanics as primary design variables rather than downstream optimizations.