MambaLite-Micro: MCU Inference for Mamba Models

• MambaLite-Micro is a deployment system that runs trained PyTorch Mamba models directly on resource-constrained microcontrollers.
• The system reimplements the selective state-space model in plain C, combining operator fusion with lifetime-aware memory allocation to reduce peak memory by up to 83%.
• It achieves 100% prediction consistency with PyTorch baselines while eliminating runtime dependencies, enabling efficient inference for tasks like keyword spotting and human activity recognition.

MambaLite-Micro is a deployment system for executing a trained PyTorch Mamba model directly on resource-constrained microcontrollers (MCUs). It is presented as a fully C-based, runtime-free inference engine and, to the authors’ knowledge, the first deployment of a Mamba-based neural architecture on an actual MCU rather than simulation or desktop execution. Its central contribution is a memory-optimized implementation of Mamba inference that exports FP32 PyTorch weights as static C arrays, reimplements the Mamba layer and supporting operators in plain C, and combines operator fusion with lifetime-aware memory layout optimization to eliminate large intermediate tensors. On the reported keyword spotting (KWS) and human activity recognition (HAR) tasks, it reduces peak memory by 83.0%, preserves 100% prediction consistency with the PyTorch baselines, and reports an average numerical error of only $1.7\times 10^{-5}$ relative to PyTorch, while running on both ESP32S3 and STM32H7 microcontrollers (Xu et al., 5 Sep 2025).

1. Problem setting and claimed contribution

MambaLite-Micro addresses a specific systems mismatch between the theoretical efficiency of Mamba sequence models and the practical constraints of embedded deployment. Mamba is attractive for edge settings because it uses a selective state-space model (SSM) rather than full attention, but the reference implementation is not directly deployable on MCUs. The paper identifies three barriers: limited MCU memory, lack of native operator support for Mamba in embedded inference stacks, and the absence of an embedded-friendly export path because the reference implementation depends on custom Triton GPU kernels and is not ONNX-exportable (Xu et al., 5 Sep 2025).

The work’s stated novelty is an end-to-end workflow that ports a trained PyTorch Mamba model onto actual MCU hardware. The system does not rely on TensorFlow Lite Micro, ONNX runtime, vendor-specific AI runtimes, Triton, or desktop-side operator emulation. Instead, the trained weights are converted into plain C arrays and compiled directly into firmware, yielding a self-contained executable binary. The inference path, including the handcrafted Mamba layer, selective state-space recurrence, and supporting operators, is manually reproduced in C (Xu et al., 5 Sep 2025).

The paper frames memory as the dominant deployment bottleneck. In standard Mamba inference, large 4D intermediates of shape $(B,D,L,N)$ are materialized, where $B$ is batch size, $D$ is hidden dimension, $L$ is sequence length, and $N$ is state dimension. On the ESP32S3, the naive KWS implementation requires 1,384,472 bytes (1352 KB) of peak RAM, exceeding available memory. MambaLite-Micro therefore centers on removing these intermediates and reorganizing execution around streaming and buffer reuse (Xu et al., 5 Sep 2025).

2. End-to-end deployment pipeline

The deployment path begins with model training in PyTorch 2.6.0 using FP32 weights, Adam, learning rate $1\mathrm{e}{-3}$, and batch size 32. For both evaluated tasks, the architecture is

$$\text{linear projection} \rightarrow \text{Mamba}(d\_model=64) \rightarrow \text{global temporal pooling} \rightarrow \text{linear classifier}.$$

This is the model that is subsequently exported to MCU firmware (Xu et al., 5 Sep 2025).

Weight export is deliberately minimal. Rather than serializing to ONNX or a TensorFlow Lite Micro flatbuffer, the trained parameters are converted to plain C arrays and embedded as static data in the firmware binary. This design removes runtime-managed model loading and aligns the deployment artifact with conventional embedded compilation workflows (Xu et al., 5 Sep 2025).

Inference logic is then reimplemented manually in embedded C. The paper emphasizes that the implementation reproduces only what is required for inference on MCU targets, not the full training-time machinery of Mamba. At the model level, on-device execution consists of input preprocessing, linear projection, a single Mamba layer, global temporal pooling, and a final linear classifier. The Mamba component includes learned parameter $A:(D,N)$, input-dependent projections $s_B(x)$, $(B,D,L,N)$0, $(B,D,L,N)$1, the $(B,D,L,N)$2 gating term, discretization, selective state-space recurrence, and the output projection from hidden state to sequence output (Xu et al., 5 Sep 2025).

The final stage is firmware compilation. Because the codebase is plain C and runtime-free, portability is demonstrated by deployment on two heterogeneous MCU environments: ESP32S3 and STM32H7 / Portenta H7. This suggests that the portability claim rests less on any vendor-specific kernel library than on the narrowness of the implemented operator set and the static embedding of weights into the application binary (Xu et al., 5 Sep 2025).

3. Mathematical formulation and memory-optimized selective SSM execution

The paper’s mathematical core is the reformulation of selective SSM inference so that discretized state-transition tensors are never materialized in full. In the original implementation, with input $(B,D,L,N)$3 and output $(B,D,L,N)$4, the parameter and input-dependent terms are defined as

$(B,D,L,N)$5

$(B,D,L,N)$6

$(B,D,L,N)$7

$(B,D,L,N)$8

The reference structure then constructs discretized 4D tensors: $(B,D,L,N)$9 followed by the state-space recurrence

$B$0

These $B$1 and $B$2 tensors have shape $B$3, which the paper identifies as prohibitive for MCU deployment (Xu et al., 5 Sep 2025).

MambaLite-Micro fuses discretization into the recurrence. Instead of storing full sequence-length tensors, it computes the required slice for each time step $B$4 on demand: $B$5 so that each recurrent update becomes

$B$6

This change reduces memory complexity from

$B$7

to

$B$8

The paper explicitly characterizes this as enabling streaming execution along sequence length $B$9: the live state is roughly $D$0, together with small working buffers, rather than all per-time-step transition tensors (Xu et al., 5 Sep 2025).

This fused implementation is paired with lifetime-aware allocation and reuse. Temporary buffers are allocated only for the duration of active use and are reused across operators whose lifetimes do not overlap. The paper does not provide allocator pseudocode, but it clearly describes the scheduling principle: allocate only when needed, release or reuse immediately after last use, and combine this with fused recurrence so that fewer temporaries exist at all. A plausible implication is that MambaLite-Micro’s gains arise from the interaction of algorithmic reformulation and systems-level memory scheduling rather than from operator fusion alone (Xu et al., 5 Sep 2025).

4. Runtime-free C systems design

A defining systems feature of MambaLite-Micro is that it is runtime-free. The implementation is entirely in plain C, with no TensorFlow Lite Micro, no ONNX runtime, no vendor inference stack, and no GPU-specific dependency. The supporting operator set includes linear projection, the Mamba selective SSM recurrence, elementwise operations, exponential, einsum-equivalent multiply-accumulate computations, pooling, and final linear classification. The paper does not enumerate every kernel separately, but it is explicit that all required operators are implemented manually in C to reproduce the PyTorch inference graph faithfully (Xu et al., 5 Sep 2025).

Precision is FP32 throughout. Both the PyTorch reference and the MCU implementation operate in float32, and the main reported results do not use quantization. This is consequential for interpreting the reported numerical error: the $D$1 average error is measured between two FP32 implementations rather than between quantized and full-precision models (Xu et al., 5 Sep 2025).

The paper also reports concrete toolchains and hardware targets. ESP32S3 is specified as 240 MHz, 320 KB RAM, and 8 MB Flash, built with PlatformIO, framework-espidf 5.3.1, CMake 3.16, esptoolpy 4.5.1, ninja 1.9, and toolchain-xtensa-esp-elf 13.2.0. STM32H7, using Arduino Portenta H7 with STM32H747XIH6 @ 480 MHz, is specified as 511.35 KB RAM and 768 KB Flash, built with PlatformIO, platform-ststm32 19.3.0, framework-arduino-mbed 4.3.1, tool-dfuutil-arduino 1.11.0, and toolchain-gccarmnoneeabi 7.2.1. The host export environment uses Python 3.11.13 and PyTorch 2.6.0 (Xu et al., 5 Sep 2025).

These implementation choices define the system’s portability claim. Because weights are static C arrays and the operator path is handwritten C, deployment is largely decoupled from model-exchange runtimes or accelerator-specific graph compilers. This suggests a design philosophy closer to embedded systems programming than to standard neural-network inference stacks (Xu et al., 5 Sep 2025).

5. Experimental evaluation, correctness, and resource profile

The evaluation covers two tasks—Keyword Spotting and Human Activity Recognition—on both ESP32S3 and STM32H7. For KWS, the dataset is Speech Commands v2. Two label settings are used: a 3-class setup with yes, no, _unknown_, and a 10-class setup with left, no, off, on, one, right, three, two, yes, _unknown_. Inputs are 0.1 second audio segments resampled to 16 kHz and converted to 40-dimensional log-Mel filterbank features; the main text also describes each input as a 4,000-point mel-spectrogram preprocessed from an audio segment sampled at 16 kHz, then linearly projected into a $D$2 representation. For HAR, the dataset is UCI-HAR, with raw 561-dimensional features zero-padded to 570 and reshaped to input dimension 57 and sequence length 10 (Xu et al., 5 Sep 2025).

Numerical fidelity is evaluated at two levels. First, the paper reports layer-level comparisons using average $D$3 error, average mean error, and worst-case $D$4 error across samples. Across 3-class KWS, 10-class KWS, and 6-class HAR, the overall average error is $D$5, with sample-level errors in the range of $D$6 to $D$7 and worst-case deviations below $D$8. Second, at the end-to-end level, “100% consistency” means identical final predictions between the MCU implementation and the PyTorch reference on the evaluation sets. The paper explicitly clarifies that this is prediction agreement with PyTorch, not 100% task accuracy against ground truth (Xu et al., 5 Sep 2025).

Ground-truth accuracies are also reported. For 3-class KWS, the MCU implementation reports 92.0% accuracy; for 10-class KWS, 92.5%; for HAR, 92.7%. In all three cases, PyTorch-vs-C confusion matrices are perfectly diagonal, indicating no behavioral degradation relative to the trained PyTorch baseline (Xu et al., 5 Sep 2025).

Ablation on ESP32S3 isolates the effect of the two memory optimizations. The unfused KWS baseline uses 1352 KB peak RAM. Adding fusion without lifetime management yields 611 KB peak RAM and 1256.2 ms latency. Full MambaLite-Micro reduces peak RAM to 230 KB and latency to 1133.2 ms. The KWS peak RAM reduction from 1,384,472 B to 235,620 B corresponds to 83.0% (Xu et al., 5 Sep 2025).

Deployment	Peak RAM	Latency
ESP32S3 KWS, 3 classes	235,620 B (230 KB)	1133.2 ms
ESP32S3 KWS, 10 classes	235,724 B (230 KB)	1133.6 ms
STM32H7 KWS, 3 classes	282,932 B (276 KB)	934.9 ms
STM32H7 KWS, 10 classes	282,932 B (276 KB)	964.1 ms
ESP32S3 HAR, 6 classes	44,244 B (43.2 KB)	123.42 ms
STM32H7 HAR, 6 classes	29,492 B (28.8 KB)	94.79 ms

Flash sizes are also reported: 369,444 B and 372,336 B for ESP32S3 KWS with 3 and 10 classes, 305,424 B and 307,248 B for STM32H7 KWS, 360,740 B for ESP32S3 HAR, and 308,720 B for STM32H7 HAR. The paper notes that increasing KWS output classes from 3 to 10 has negligible effect on memory or latency because the final classifier is small relative to the sequence-processing portion (Xu et al., 5 Sep 2025).

The paper further includes a practical comparison to a TFLM-based attention baseline for KWS. That attention model uses int8 quantization, 4 heads, head dimension 16, total hidden dimension 64, and sequence length 100, with reported inference time 3,991 ms. MambaLite-Micro in FP32 reports 1133.2 ms on the corresponding task. The paper interprets this as evidence that Mamba is already practical on MCU and may improve further with post-training quantization, fixed-point arithmetic, and SIMD acceleration (Xu et al., 5 Sep 2025).

6. Interpretation, limitations, and relation to the broader “MambaLite” nomenclature

Several interpretive points are necessary for precise reading. First, the paper’s “first deployment” claim is explicitly qualified as “to the authors’ knowledge.” Second, its 100% consistency result concerns agreement with the PyTorch baseline, not perfect classification against ground truth. Third, the reported memory-complexity improvement from $D$9 to $L$0 applies to the large discretization intermediates and the recurrence computation; its whole-model significance arises because this operator dominates peak RAM (Xu et al., 5 Sep 2025).

The paper also leaves clear boundaries around what is and is not implemented. It is FP32-only; no quantized Mamba deployment is provided. KWS latency remains around one second per inference on MCU. No energy measurements are reported. The evaluation studies relatively compact, single-Mamba-layer task models rather than a broad architecture sweep, and the operator descriptions remain at a higher level than full low-level kernel listings or allocator pseudocode. These are practical rather than conceptual limitations, but they matter for assessing transferability to more demanding embedded workloads (Xu et al., 5 Sep 2025).

Within the broader literature, the name “MambaLite” has been used in distinct contexts that should not be conflated. “MedMambaLite: Hardware-Aware Mamba for Medical Image Classification” describes a vision-oriented, hardware-aware Mamba model for medical image classification, centered on architectural redundancy removal, parameter sharing, and knowledge distillation, with deployment on NVIDIA Jetson Orin Nano rather than MCU firmware (Aalishah et al., 7 Aug 2025). “MambaLiteUNet: Cross-Gated Adaptive Feature Fusion for Robust Skin Lesion Segmentation” describes a compact Mamba-U-Net segmentation framework with AMF, LGFM, and CGA modules, again not an MCU inference engine and not a model explicitly named “MambaLite-Micro” (Rahman et al., 22 Apr 2026). In that sense, MambaLite-Micro is best understood not as a lighter vision backbone in the style of those works, but as an embedded deployment stack for 1D sequence inference on actual microcontrollers (Xu et al., 5 Sep 2025).

A common misconception would therefore be to treat all “MambaLite” papers as members of a single architecture family. The available evidence suggests a looser naming pattern: MambaLite-Micro is a systems paper on runtime-free MCU inference; MedMambaLite is a hardware-aware compact vision classifier; and MambaLiteUNet is a compact segmentation design recipe. Their shared theme is efficiency, but their technical objectives, target hardware, and optimization levers differ substantially (Aalishah et al., 7 Aug 2025); (Rahman et al., 22 Apr 2026).