AutoSlim: Neural & Accelerator Optimization

• AutoSlim is a dual-optimization framework that addresses neural channel allocation via greedy slimming and FPGA automata reduction via transition pruning.
• In neural architecture search, it employs a slimmable network with a one-shot, non-uniform channel search strategy to enhance accuracy and efficiency.
• For symbolic accelerators, AutoSlim uses a Random Forest classifier to prune low-impact transitions, reducing resource usage and improving throughput.

Searching arXiv for AutoSlim and closely related papers to ground the article. {"query":"AutoSlim arXiv AutoSlim Towards One-Shot Architecture Search for Channel Numbers Slimmable ConvNeXt Machine Learning-Based Graph Simplification for Symbolic Accelerators", "max_results": 10} AutoSlim is a term used for two distinct optimization frameworks in the arXiv literature. In neural architecture search, AutoSlim denotes a one-shot method for selecting per-layer channel numbers under resource constraints by training a single slimmable network and then greedily slimming layers with the smallest estimated accuracy drop (Yu et al., 2019). In FPGA-based symbolic acceleration, AutoSlim denotes a machine learning–guided graph simplification framework that prunes low-impact transitions in weighted NFAs for overlays such as NAPOLY+, using execution-derived features and a Random Forest classifier to reduce graph density while preserving practical functional behavior (Yu et al., 9 May 2026). Later work on width-adaptive CNNs treats the first AutoSlim as a canonical BN-based non-uniform channel-allocation baseline and contrasts it with LayerNorm-based alternatives such as Slimmable ConvNeXt (Haberer et al., 21 May 2026).

1. Terminological scope and research lineage

The neural-network AutoSlim emerged from the slimmable-network literature, where a single shared-weights model exposes multiple width configurations that can be selected at inference time. Within that setting, AutoSlim addresses the specific problem of how to allocate channels non-uniformly across layers so that a model attains higher accuracy than uniform width multipliers at the same FLOPs, latency, memory footprint, or model size (Yu et al., 2019). In subsequent discussions of width-adaptive inference, AutoSlim is grouped with classical CNN-based slimmable methods such as Slimmable Networks and US-Nets, which relied on switchable batch normalization, sandwich rule training, inplace distillation, and per-width BN calibration (Haberer et al., 21 May 2026).

The accelerator AutoSlim belongs to a different research lineage. It targets symbolic accelerators that execute automata graphs on FPGAs, especially score-aware NFAs mapped to NAPOLY+, and seeks to reduce excessive memory usage, routing congestion, and redundant computation by pruning low-utility graph structure (Yu et al., 9 May 2026). Its optimization variable is not channel count in a DNN but the transition structure of a weighted automaton.

Usage of “AutoSlim”	Optimization target	Core mechanism
Neural architecture search	Per-layer channel numbers under resource budgets	Slimmable supernet plus greedy channel slimming
Symbolic accelerators	Weighted NFA graph size, density, and fanout	Random Forest–guided transition pruning

A plausible implication is that the term now functions as a homonym across two otherwise unrelated optimization problems. In technical writing, the surrounding context—channel search in CNNs versus graph simplification for NFAs—is therefore essential.

2. Neural AutoSlim: constrained channel allocation in slimmable networks

In its original form, AutoSlim formulates channel selection as a constrained optimization problem over per-layer channel counts $C = \{C_1, C_2, \dots, C_L\}$:

$$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$

Here $R(C)$ may be FLOPs, latency, parameters, or memory, provided that the resource is deterministic for a given architecture and runtime environment (Yu et al., 2019).

The method relies on a slimmable performance estimator. Rather than training many sampled architectures, AutoSlim trains a single slimmable network that approximates the accuracy of many channel configurations. For ImageNet, the slimmable estimator is trained for 50 epochs with a squeezed learning-rate schedule, using a held-out 50K subset of the training set as the validation set for greedy slimming; after search, the selected fixed architectures are retrained from scratch for full schedules—300 epochs for mobile networks with linearly decaying learning rate, and 100 epochs for ResNet-50 variants with a step learning-rate schedule (Yu et al., 2019).

The resource model is explicit. FLOPs are counted as MACs and exclude BN FLOPs and parameters. For a standard or group convolution,

$$\mathrm{FLOPs} = H_{out} W_{out} \, C_{out} \, K_h K_w \, \frac{C_{in}}{g},$$

and for a depthwise convolution,

$$\mathrm{FLOPs}_{dw} = H_{out} W_{out} \, K_h K_w \, C_{in}.$$

Parameter counts, excluding BN and bias, follow the corresponding convolutional formulas (Yu et al., 2019). For mobile models, the slimmable network spans width multipliers from $0.15\times$ to $1.5\times$, but the search itself is non-uniform: layers may receive different channel counts rather than a single global width multiplier.

Training incorporates the standard slimmable-network machinery of the period. The method uses switchable or universal BN together with BN re-calibration, sandwich rule training, and inplace distillation so that the slimmable model can serve as a useful relative accuracy estimator across many widths (Yu et al., 2019). Later summaries explicitly note that AutoSlim inherits the S-Net training recipe: shared weights across widths, switchable BN, multi-width training, and a search stage that selects layer-wise channel numbers by validation accuracy under FLOPs or latency constraints (Kuratsu et al., 2022).

3. Greedy slimming algorithm and empirical behavior

After slimmable pretraining, AutoSlim performs a single greedy pass. Starting from the largest feasible network, it considers slimming each layer by one channel group and estimates the validation accuracy drop:

$$\Delta_l = \mathrm{Acc}(C) - \mathrm{Acc}\!\big(C_{(l \leftarrow C_l - \Delta c_l)}\big).$$

At each step, the method picks the layer with minimal accuracy drop, updates the current channel configuration, and repeats until the target budget is met (Yu et al., 2019). The criterion is purely accuracy-drop based per one-group reduction; it does not normalize by resource saved per layer. To reduce search complexity, channels are divided into evenly sized groups, for example 10 groups per layer.

This one-shot greedy procedure produces non-uniform channel allocations that differ systematically from default width-multiplier heuristics. On MobileNet v2, AutoSlim consistently allocates fewer channels to shallow layers and more to deeper layers than the default architecture. Compared with AMC on MobileNet v2, AutoSlim uses dramatically more channels in deep layers and fewer channels in shallow layers, reflecting the high FLOPs cost of early layers with large spatial maps (Yu et al., 2019).

ImageNet results establish the method’s resource–accuracy trade-off. At roughly 300M FLOPs, AutoSlim-MobileNet v2 reaches 74.2% top-1 accuracy at 305M FLOPs, compared with 71.8% for default MobileNet v2 at 300M FLOPs and 74.0% for MNasNet 1.0× at 317M FLOPs. At roughly 200M FLOPs, AutoSlim-MobileNet v2 reaches 73.0% at 207M FLOPs versus 69.8% for MobileNet v2 0.75× at 209M FLOPs. AutoSlim-ResNet-50 at 570M FLOPs reaches 72.2% top-1 accuracy, which is 1.3% higher than MobileNet v1 1.0× at 569M FLOPs (Yu et al., 2019).

The method also reports latency improvements under the single-core CPU measurements used in the paper. For example, AutoSlim-MobileNet v2 at 305M FLOPs is reported at 77 ms versus 81 ms for MobileNet v2 1.0×, and AutoSlim-MNasNet at 315M FLOPs is reported at 71 ms versus 76 ms for MNasNet 1.0× (Yu et al., 2019). This underscores that the search is not restricted to FLOPs optimization in principle, even though FLOPs are the default reported constraint.

4. Relationship to S-Net, SP-Net, and Slimmable ConvNeXt

Later work reframed AutoSlim as one solution to the broader slimmable-subnetwork selection problem. In the SP-Net account, S-Net allows runtime selection among predefined widths but suffers because each sub-network uses the same proportion of channels in every layer. AutoSlim addresses that core problem by first training a slimmable supernet and then selecting high-accuracy sub-networks through one-shot, resource-constrained per-layer channel search (Kuratsu et al., 2022). SP-Net replaces uniform-width sub-networks with pruning-learned non-uniform structures embedded into one shared model, using multi-base pruning, slimmable channel sorting, and zero padding match pruning.

The comparison is informative because it isolates AutoSlim’s architectural-search cost. For ResNet-50 at approximately 2.0G FLOPs, SP-Net reports 24.3% error versus 24.4% for AutoSlim, with 0 search cost versus 900 GPU-hours. At approximately 1.0G FLOPs, both report 26.0% error, again with 0 versus 900 GPU-hours. For MobileNet variants, AutoSlim remains competitive or slightly better at some budgets, but SP-Net’s central claim is parity or near parity without an explicit architecture search stage (Kuratsu et al., 2022).

Slimmable ConvNeXt revisits the deployment problem from the opposite direction. It argues that classical slimmable CNN pipelines—including those surrounding AutoSlim—carry normalization overhead because BN-based backbones require switchable BN, per-width statistics, and calibration. By using ConvNeXt blocks with LayerNorm and inverted bottlenecks, Slimmable ConvNeXt eliminates switchable BN and BN calibration, trains all widths jointly without distillation or sandwich rules, and still permits AutoSlim-style non-uniform width search on top of LN (Haberer et al., 21 May 2026). On Tiny models, Slimmable ConvNeXt-T with 3 subnetworks reaches 80.8% top-1 at 4.5 GMACs and 77.4% at 1.2 GMACs, outperforming HydraViT, MatFormer-S, and SortedNet-S at comparable budgets (Haberer et al., 21 May 2026).

The relation between AutoSlim and Slimmable ConvNeXt is especially precise in the ablations. On ConvNeXt-T with 3 or 4 subnetworks, AutoSlim-style non-uniform allocation modestly improves quarter-capacity accuracy from 66.9% to 67.9%, but is marginal or slightly negative at half and full capacity, for example from 80.8% to 80.4% at full width (Haberer et al., 21 May 2026). This suggests that AutoSlim’s channel-search logic remains usable in modern width-adaptive models, but its gains depend strongly on the backbone and width regime.

5. AutoSlim for symbolic accelerators: weighted-NFA simplification

In the symbolic-accelerator literature, AutoSlim operates on score-aware NFAs mapped to FPGA overlays such as NAPOLY+. A standard NFA is represented as

$A = (Q, \Sigma, \delta, q_0, F),$

with accepted language

$$L(A) = \{\, w \in \Sigma^* \mid \delta^*(q_0, w) \cap F \neq \emptyset \,\}.$$

NAPOLY+ enriches this model with per-edge scores $$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$0. For a run

$$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$1

the path cost is

$$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$2

and the minimum accepting cost for input $$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$3 is

$$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$4

Ranking semantics in NAPOLY+ are determined by $$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$5 (Yu et al., 9 May 2026).

The pruning objective is hardware-centric:

$$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$6

A practical surrogate is

$$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$7

with $$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$8, $$\max_{C} \; \mathrm{Acc}(C) \quad \text{s.t.} \quad R(C) \le B .$$9, and $R(C)$0 capturing LUT, FF, BRAM, URAM, and routing effects (Yu et al., 9 May 2026). The paper states a stronger ideal condition in the score-aware setting—preserving both acceptance and minimum-cost ranking—but also states that the implementation emphasizes acceptance preservation and critical-path conservation in practice rather than formal equivalence proof.

The end-to-end pipeline begins with ANML-compatible XML automata whose transitions are converted into CSV records of $R(C)$1 and per-edge score. The current implementation uses edge score as the primary feature. Node-level aggregates such as degree and total node score are computed during parsing, and the paper motivates additional features such as edge visitation frequency, node centrality, path coverage, temporal locality, and entropy of labels as possible extensions, but only the score is used in the reported Random Forest configuration (Yu et al., 9 May 2026).

Supervised labels are generated from a score threshold $R(C)$2:

$R(C)$3

where $R(C)$4 is the edge-score feature. A Random Forest classifier is then trained on the labeled dataset $R(C)$5 using standard cross-validation. Transitions predicted as keep are retained; transitions predicted as prune are removed. Unreachable states are then eliminated, and duplicated destinations are described conceptually as mergeable even though the released pipeline does not elaborate a detailed state-merging algorithm (Yu et al., 9 May 2026).

6. Evaluation, deployment impact, and open issues

For neural AutoSlim, the main strengths are one-shot search efficiency relative to RL-based NAS and strong ImageNet accuracy at matched budgets. The principal limitations are also explicit. Greedy slimming is potentially suboptimal because it makes one-group reductions based only on immediate accuracy drop. The quality of the result depends on the slimmable model being a reliable relative accuracy estimator across configurations. Cross-dataset transfer is limited: channel configurations optimized on ImageNet do not generalize well to CIFAR-10, as shown by the gap between an ImageNet-searched AutoSlim-MobileNet v2 transferred to CIFAR-10 and a CIFAR-10-specific search (Yu et al., 2019).

For accelerator AutoSlim, the reported hardware impact is substantial. Across datasets, total transitions are reduced by more than 30–40%. At 1K nodes, average transitions per node drop from approximately 180 to just above 100; at 64K nodes, they drop below 5. For the 64K dataset, the unpruned design uses more than 47,000 LUTs and 30,000 registers, whereas the pruned design uses fewer than 7,000 LUTs and approximately 2,000 registers; URAM usage is reduced by approximately 50% across all sizes. The study also reports up to 18% throughput increase and notes associated improvements in power efficiency (Yu et al., 9 May 2026).

The deployment logic is closely tied to fanout. In NAPOLY+, higher fanout can reduce minimum latency because more transitions are explored in parallel, but it also drives LUT and register usage, routing complexity, and worst-case latency. AutoSlim’s pruning reduces average fanout and graph density, thereby mitigating the routing blowup that limits automata scaling on FPGAs (Yu et al., 9 May 2026). In this sense, its primary contribution is front-end simplification rather than back-end mapping optimization.

A recurrent misconception is that both AutoSlim systems provide formal guarantees. The neural version does not seek optimality guarantees beyond the greedy constrained search procedure, and the accelerator version does not present a formal proof of language or cost equivalence after pruning. The accelerator paper explicitly characterizes its verification as pragmatic, calls out accept-state preservation and edge-coverage metrics as planned, and leaves stronger formal methods as future work (Yu et al., 9 May 2026). Another misconception is that AutoSlim always refers to a single method family; the literature now uses the name for both non-uniform neural channel search and ML-guided automata simplification, which are technically unrelated despite their shared emphasis on resource-aware slimming.