On-Device Anomaly Detection

• On-device anomaly detection is a technique that uses machine learning on resource-constrained embedded systems to identify abnormal data patterns in real time.
• Techniques include ensemble methods, one-class SVMs, recurrent architectures, and statistical rules, each optimized for minimal memory, latency, and energy use.
• Hardware-aware designs balance detection accuracy, energy efficiency, and latency, making these methods essential for IoT, smartphones, and industrial controllers.

On-device anomaly detection refers to the class of machine learning and statistical techniques that enable detection of abnormal data patterns directly on resource-constrained embedded platforms—such as microcontrollers, IoT nodes, smartphones, and industrial controllers—without reliance on continuous cloud or server connectivity. It is motivated by the strict requirements of real-time response, privacy preservation, limited bandwidth, and minimal memory/power footprint typical of edge deployments. The paradigm encompasses a spectrum of unsupervised, semi-supervised, and supervised models, with implementation and algorithmic choices fundamentally dictated by the interplay between detection accuracy, latency, memory, and energy constraints (Benmachiche et al., 22 Dec 2025).

1. Algorithmic Foundations

Four families of anomaly detection algorithms dominate current edge deployments, each offering distinct accuracy-efficiency tradeoffs (Benmachiche et al., 22 Dec 2025):

1. Isolation Forest (IF):

IF is an unsupervised ensemble of binary trees that recursively partitions the feature space, exploiting the principle that anomalies are more susceptible to isolation via random splits. The score

$s(x,\psi) = 2^{-E[h(x)]/c(\psi)}$

defines anomaly likelihood, where $h(x)$ is mean path length and $c(\psi)$ is the expected path length of a random binary tree. Key hyperparameters (tree count $T$, subsample size $\psi$, max tree depth) scale linearly in RAM and latency. IF is robust under light quantization/pruning but is more accurate for larger $T$ and moderate $\psi$ (Benmachiche et al., 22 Dec 2025).

2. One-Class SVM (OC-SVM):

OC-SVMs fit a hypersurface via a kernel function to enclose "normal" data in feature space. The primal optimization minimizes $$\frac{1}{2}\|w\|^2 + \frac{1}{\nu n}\sum \xi_i - \rho$$, with thresholding determined by the parameter $\nu$ and kernel inverse width $\gamma$. State-of-the-art hardware-adaptive techniques (Nyström sketching, support vector pruning) reduce memory and improve speed with minor boundary fidelity tradeoff (Benmachiche et al., 22 Dec 2025).

3. Recurrent Architectures (LSTM-Autoencoder, 1D-CNN, Tiny Transformers):

Encoder-decoder or sequence-prediction models learn the underlying dynamics of normal time series, flagging examples where reconstruction or forecasting error exceeds a chosen threshold. 8-bit quantization and aggressive pruning maintain $< 1\%$ F1-score loss versus float baselines on Arm Cortex-M MCUs, supporting sub-50 ms inference (Benmachiche et al., 22 Dec 2025).

4. Statistical/Threshold-Based Methods:

Control charts, moving averages, and z-score or EWMA rules deliver deterministic $O(1)$ per-sample computation, ultra-low RAM (<10 kB), and fixed latency. For simple Gaussian data, z-thresholds ($k \approx 3$) yield low false alarms, but statistical methods miss subtle or complex context anomalies (Benmachiche et al., 22 Dec 2025).

Additional techniques for specific contexts include:

• Local Outlier Factor (LOF) with online reservoir sampling to realize uniform memory-constrained learning and scoring pipelines on MCUs (Szydlo, 2022).
• Kernel PCA, Variational Autoencoders and compact deep architectures (e.g., OutlierNets) for structured data or sensor signals on mobile or industrial platforms (Abbasi et al., 2021, Vella et al., 2021).
• Rule-based methods for binary thresholding, extensively used in hard-real-time scientific or safety-critical embedded controllers (Toledo et al., 2024).

2. Resource Profiling and Performance Benchmarks

A core concern in on-device anomaly detection is quantifying and managing memory, computational latency, and energy to ensure compliance with the platform envelope (Benmachiche et al., 22 Dec 2025). Typical metrics include:

Method	RAM (KB)	Latency	Power	Example HW	F1
Isolation Forest	120–160	<50 ms	~5 mW	Cortex-M4 @ 80 MHz	0.95
OC-SVM (RBF)	500–1000	150–300ms	~15 mW	ARM Cortex-A7/PiZero	0.95
LSTM-Autoencoder (8-bit)	180–250	30–40 ms	~8 mW	Cortex-M7 + TFLM	0.97
1D-CNN	100–120	20–30 ms	~6 mW	Cortex-M4 + CMSIS-NN	0.95
Statistical threshold	<10	<5 ms	~1 mW	Any MCU (fixed pt.)	0.81

Models employing quantization and pruning (e.g., 1D-CNN or LSTM-AE at 8 bits) show only minimal loss in F1 (typically <1%) with sharp reductions in both memory and latency, enabling systems such as Cortex-M4/M7 (128–512 KB RAM) to deploy temporal models for moderately complex tasks (Benmachiche et al., 22 Dec 2025). Statistical and threshold methods remain dominant for Cortex-M0/M3-class MCUs (≤64 KB RAM) where soft real-time (RT) deadlines (<10 ms) and zero training overhead are essential.

3. Hardware-Aware Design and Selection

Algorithm choice is fundamentally co-determined by platform class and the stringency of real-time and memory requirements (Benmachiche et al., 22 Dec 2025):

Platform	Constraints	Recommended Method	Justification
Cortex-M0	≤64 KB RAM, <10 ms latency	Statistical/Threshold	Deterministic, <10 KB RAM, no training
Cortex-M4	128 KB RAM, <50 ms latency	Isolation Forest	Linear time, high F1, fits in RAM
Cortex-M7	256 KB RAM, <50 ms latency	1D-CNN or LSTM-AE	Quantization/pruning; temporal patterns, higher complexity
Pi Zero	512 MB RAM, soft RT	OC-SVM, Hybrid IF+AE	Support vector sketching or cascades feasible
Jetson Nano	>4 GB RAM, GPU accel.	Tiny Transformer/CNN	Transformer attention running in sub-20 ms on GPU

Cascaded/hybrid pipelines, where statistical or tree-based models serve as ultra-light “screeners” before invoking heavier deep models, can optimize average runtime and ensure hard-confidence filtering under severe RAM and compute limits (Benmachiche et al., 22 Dec 2025).

4. Tradeoffs in Accuracy, Latency, and Energy

Empirical tradeoff studies show that increasing the number of estimators (e.g., $T$ in IF, or tree depth in decision forests) generally yields diminishing accuracy returns with a proportional rise in memory and latency (Benmachiche et al., 22 Dec 2025, Martinez-Rau et al., 2024). For instance, doubling $T$ in IF incurs a 2$\times$ increase in RAM/latency for a gain of merely 1–2% F1. Similarly, for LSTM/1D-CNN, 8-bit quantization reduces RAM by $\approx 4\times$ with virtually no F1 loss, while 50% weight pruning delivers a 30% speed-up, provided retraining is allowed (Benmachiche et al., 22 Dec 2025, Abbasi et al., 2021).

Statistical methods (e.g., z-threshold, EWMA) provide deterministic ultra-fast inference, but are sensitive to model misspecification and fail to capture complex or adaptive drift. Advanced models (e.g., OC-SVM with Nyström approximation(Yang et al., 2021)) realize order-of-magnitude improvements in speed and storage with only minor AUC degradation, facilitating their deployment on edge CPUs and MCUs.

5. Emerging Trends: Continual Learning, Adaptation, and TinyML

Recent advances highlight the need for on-device adaptation, continual learning, and explainability in deployment scenarios typified by concept drift, domain shift, or small-batch manufacturing (Benmachiche et al., 22 Dec 2025, Ren et al., 15 Dec 2025). Key trends include:

• Online/Continual Learning: Streaming PCA, rank-one sequential autoencoders (e.g., ONLAD leveraging OS-ELM and forgetting) are vital for tracking evolving normalcy (Tsukada et al., 2019).
• Hardware-aware Neural Architecture Search (NAS): Automated search co-optimizes accuracy, quantization, and memory footprint, producing sub-10 kB models with near-baseline accuracy (e.g., OutlierNets for acoustic anomaly detection) (Abbasi et al., 2021).
• Federated and Split Learning: Model updates are exchanged and aggregated to enable global adaptation without raw data egress, as in federated IF or autoencoder pipelines for IoT (Zhang et al., 2021, Ochiai et al., 2024).
• Attention/Tiny-Transformer Models: Lightweight attention or sparse transformer blocks (e.g., MemATr) enable richer temporal context in <200 KB RAM for ARM MCUs (Benmachiche et al., 22 Dec 2025).
• Explainability: Built-in feature-attribution for anomaly decisions is emerging as an auxiliary to automated thresholding (Benmachiche et al., 22 Dec 2025).

6. Deployment Guidelines and Benchmarking

Deployment of on-device anomaly detection benefits from well-calibrated benchmarks combining detection metrics (precision, recall, F1) with hardware metrics (latency, RAM, power). The following practices are recommended (Benmachiche et al., 22 Dec 2025):

• Start by profiling available RAM and allowable inference latency.
• For RAM <64 KB or strict latency, use statistical detectors.
• For 64–200 KB, deploy IF with $T \sim 50–100$, $\psi \sim 256$, adapting $T$ as accuracy/latency allows.
• For time-series and RAM >200 KB, use quantized LSTM-AE or pruned 1D-CNN.
• With >1 MB RAM, consider OC-SVM (with Nyström/QuickShift++), hybrid cascades, or attention models.
• Integrate anomaly detectors with on-device profiling to auto-select hyperparameters.
• Design cascaded pipelines: statistical filter $\longrightarrow$ tree model $\longrightarrow$ deep net for flagged events.
• Harden models via secure bootstrapping and adversarial sanitization.
• Embrace MLPerf Tiny or similar benchmarks for cross-system and cross-method comparability.

Standardized reporting and integration with profiling utilities streamline hyperparameter tuning and enforce resource constraint adherence.

7. Limitations and Future Directions

While the state-of-the-art achieves high F1 ($\sim$0.95–0.97) with milliwatt-level power and sub-50 ms latency on midrange MCUs, open challenges include: dynamic adaptation under abrupt concept drift (requiring continual learning or federated updates), explainability on microcontroller-class hardware, and graceful scaling to high-dimensional or multimodal sensor streams (Benmachiche et al., 22 Dec 2025). TinyML trends point toward neural architecture search, hybrid statistical-neural cascades, and plug-and-play explainability as critical next steps.

The field continues to evolve toward more automated, adaptive, and resource-aware methods that close the gap between edge-resident detection capability and complex, safety-critical application domains—establishing robust real-time anomaly detection as a cornerstone of decentralized, privacy-preserving, and resilient IoT and embedded deployments.