Qute: TinyML Uncertainty Ensemble

• Qute is a TinyML ensemble that leverages early-exit techniques and knowledge distillation to quantify uncertainty under stringent memory and compute constraints.
• It integrates lightweight ensemble heads via training-time parameter copying and progressive loss-weighting, improving calibration on OOD and CID inputs.
• Qute achieves superior uncertainty quality with a 31% reduction in latency and a 40% smaller model size, making it ideal for safety-critical microcontroller deployments.

Searching arXiv for the primary QUTE paper and nearby name-colliding works for disambiguation. QUTE is a resource-efficient early-exit-assisted ensemble architecture for uncertainty quantification in TinyML model monitoring. It targets deployments on milliatt-scale, KB-sized microcontrollers that operate without access to true labels, where uncertainty must be estimated under stringent memory and compute budgets. The method attaches additional output blocks at the final exit of a base network, distills early-exit knowledge into these blocks, and forms a lightweight ensemble whose predictions are averaged in a single forward pass. Reported results show superior uncertainty quality on tiny models, comparable performance on larger models with 59% smaller model sizes than the closest prior work, an average 31% reduction in latency on a microcontroller, and improved detection of accuracy-drop events (Ghanathe et al., 2024).

1. Problem setting and motivation

QUTE is motivated by the deployment regime of TinyML devices: safety-critical or remote systems such as cameras on autonomous vehicles and sensors in industrial systems, operating with only a few tens of kilobytes of SRAM/flash and very low compute budgets, on the order of a few $10^5$–$10^7$ FLOPS per inference (Ghanathe et al., 2024). In this setting, the central monitoring problem is not merely classification accuracy but uncertainty quality under field conditions where labels are unavailable.

The paper distinguishes two kinds of distributional shifts encountered in deployment. Out-of-distribution (OOD) inputs correspond to entirely unseen semantic classes. Corrupted-in-distribution (CID) inputs preserve the nominal class set but degrade the observation process, for example through fogged or frosted lenses, motion blur, or electronic noise. In both cases, well-calibrated uncertainty is operationally important: an overconfident model may propagate incorrect downstream decisions, whereas a model that recognizes its own uncertainty can trigger fail-safe or human-in-the-loop interventions (Ghanathe et al., 2024).

Conventional uncertainty quantification methods are poorly matched to this regime. Bayesian neural nets and Monte Carlo dropout require multiple forward passes or substantial parameterization, while deep ensembles scale model size with the number of ensemble members. Early-exit-based ensembles reduce repeated computation by collapsing multiple exits into a single forward pass, but still incur overhead from buffering and extra layers that remains prohibitive for a few-kilobyte budget. QUTE is positioned as a response to that specific TinyML constraint profile (Ghanathe et al., 2024).

2. Architectural organization

QUTE begins with a base network of depth $D$, composed of feature blocks

$a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$

Into this network it inserts $K$ lightweight early-exit classifiers

$$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$

at depths $d_1<d_2<\cdots<d_K$. Each $g_{\theta_k}$ is described as a small conv-+-dense softmax. At the final feature map $a_D$, QUTE does not retain one large terminal output block. Instead, it attaches $K$ additional ensemble heads

$10^7$0

each of comparable cost to its corresponding early-exit; the original final head is discarded (Ghanathe et al., 2024).

The key training mechanism is early-view knowledge distillation, termed EV-assistance. During training, each final ensemble head $10^7$1 is encouraged to imitate its corresponding early-exit $10^7$2. After every mini-batch, the parameters are copied according to

$10^7$3

The description states that this makes the filters in $10^7$4 track those in $10^7$5. In the last 10% of epochs, the base network $10^7$6 is frozen, so that each exit pair $10^7$7 continues to co-train in isolation, which is intended to encourage diversity among the $10^7$8 ensemble heads (Ghanathe et al., 2024).

At inference time the architecture is strictly single-pass. The input traverses the backbone once to produce $10^7$9. Each head $D$0 then applies a small depth-wise convolution followed by dense+softmax to produce $D$1. The predictive distribution is the arithmetic mean

$D$2

This organization preserves an ensemble interpretation while keeping the inference graph compact enough for TinyML deployment (Ghanathe et al., 2024).

3. Uncertainty formulation and learning objective

QUTE uses the ensemble outputs $D$3 to compute scalar uncertainty scores after a single forward pass. The predictive entropy is

$D$4

The mutual information, described as a measure of epistemic uncertainty in an ensemble, is

$D$5

The ensemble variance is

$D$6

The paper explicitly treats entropy, MI, and variance as alternative uncertainty scores computed from the same single-pass ensemble outputs (Ghanathe et al., 2024).

The training loss combines standard classification and KL-based distillation:

$D$7

with

$D$8

The weights $D$9 are increasing weights, for example $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$0, chosen to promote diversity, while $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$1 down-weights the original base output if retained (Ghanathe et al., 2024).

The paper’s explanation of why QUTE works better is correspondingly mechanistic. Early-exit assistance injects “diverse” intermediate features into each final head, progressive loss-weighting prevents collapse to a single homogeneous head, and small depth-wise heads keep per-member cost low enough that an ensemble of $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$2 remains feasible on a few-kilobyte device. This suggests that QUTE’s uncertainty quality depends on coupling representational diversity to a deployment-constrained head design rather than on enlarging the backbone itself (Ghanathe et al., 2024).

4. Training and inference workflow

The training procedure is specified as a staged pipeline. First, the base network $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$3 and all exit parameters $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$4 are initialized. For each mini-batch $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$5, the model performs a forward pass through $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$6, computes each early-exit $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$7 and each ensemble head $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$8, evaluates the total loss, and back-propagates updates to all parameters. At the end of the batch it copies $a_i = f_i(a_{i-1}),\quad i=1,\dots,D,\quad a_0=x.$9 for all $K$0. After 90% of the epochs, the backbone is frozen and only the exit pairs continue to co-adapt. The final saved model consists of the base network and the $K$1 ensemble heads $K$2; all early-exit blocks $K$3 are discarded (Ghanathe et al., 2024).

Inference is simpler. A single forward pass computes $K$4, each ensemble head produces $K$5, and these are averaged to $K$6. The final predicted label is

$K$7

after which entropy, MI, or variance can be computed as the uncertainty score. This separation is important: early exits are a training-time assistance mechanism, whereas the deployed uncertainty monitor uses only the compact final-head ensemble (Ghanathe et al., 2024).

The paper also emphasizes the deployment footprint. The final model weights and heads fit in a few $K$8 KB of flash/SRAM, and a single small-head forward pass costs only a few $K$9 FLOPS. These statements place QUTE within the resource envelope that motivated the work in the first place (Ghanathe et al., 2024).

5. Experimental regime and reported results

The experimental setup spans image and speech tasks, together with OOD and CID benchmarks designed to stress uncertainty estimation rather than nominal accuracy alone (Ghanathe et al., 2024).

Benchmark	Base model	Approx. parameters
MNIST	4-layer CNN	$$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$0 K
CIFAR-10	ResNet-8	$$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$1 K
Tiny-ImageNet	MobileNetV2	$$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$2 M
Speech Commands	4-layer DS-CNN	$$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$3 K

The CID benchmarks are MNIST-C, CIFAR10-C with 19 corruptions $$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$4 5 severities, and Tiny-ImageNet-C with 15 $$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$5 5. The OOD test sets are Fashion-MNIST for MNIST, SVHN for CIFAR-10, and unused words in Speech Commands for keyword spotting. Training uses Adam, learning-rate decay, 200 epochs on an RTX 2080 for larger networks, and 20 epochs for MNIST (Ghanathe et al., 2024).

On MNIST, averaged over 3 splits, the in-distribution comparison reports that QUTE with 4.4 K parameters achieves $$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$6, Brier $$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$7, and NLL $$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$8. EE-ensemble with 10.9 K parameters has $$\text{EE}_k:\;g_{\theta_k}(a_{d_k})\to p(y\mid x;\theta_k)$$9, Brier $d_1<d_2<\cdots<d_K$0, and NLL $d_1<d_2<\cdots<d_K$1, while a deep ensemble with $d_1<d_2<\cdots<d_K$2 K $d_1<d_2<\cdots<d_K$3 attains $d_1<d_2<\cdots<d_K$4, Brier $d_1<d_2<\cdots<d_K$5, and NLL $d_1<d_2<\cdots<d_K$6. On corrupted in-distribution data, QUTE reports $d_1<d_2<\cdots<d_K$7 and NLL $d_1<d_2<\cdots<d_K$8, compared with EE-ensemble at $d_1<d_2<\cdots<d_K$9 and NLL $g_{\theta_k}$0, and deep ensemble at $g_{\theta_k}$1 and NLL $g_{\theta_k}$2. The paper summarizes this as an average NLL improvement of approximately 6% versus EE-ensemble while using approximately 40% of its model size (Ghanathe et al., 2024).

For CIFAR-10 and Tiny-Imagenet, the reported trend is similar: QUTE matches or slightly trails the very largest ensembles on in-distribution data but outperforms them on corrupted data, while being 3–6$g_{\theta_k}$3 smaller. Aggregated across all benchmarks, QUTE is on average 3.1$g_{\theta_k}$4 smaller in parameter count than the leading early-exit ensemble and requires approximately 3.8$g_{\theta_k}$5 fewer FLOPS per inference. The paper further states that in a microcontroller context these savings translate to approximately 31% lower wall-clock latency and approximately one-third the energy consumption. The limitations section also states that there are “No real-hardware microcontroller timing results in this paper (future work).” This suggests that the latency and energy discussion should be read cautiously as a deployment-oriented claim rather than as a full real-hardware study (Ghanathe et al., 2024).

A separate evaluation concerns accuracy-drop event detection under CID inputs, using sliding-window monitoring of confidence versus sliding accuracy and treating drop events as positives. Averaged over severities, MNIST-C yields QUTE AUPRC $g_{\theta_k}$6 and best $g_{\theta_k}$7, compared with EE-ensemble at $g_{\theta_k}$8 and deep ensemble at $g_{\theta_k}$9. On CIFAR10-C at severity $a_D$0, QUTE’s $a_D$1 rises to approximately $a_D$2 at high severity, outperforming all baselines at the hardest levels. On Tiny-Imagenet-C, QUTE outperforms both MC-dropout and EE-ensemble on event detection at severity 4–5. The paper therefore treats model monitoring, not only predictive scoring, as a primary use case (Ghanathe et al., 2024).

6. Interpretation and limitations

QUTE is presented as the first early-exit ensemble specifically optimized for TinyML. Its defining claim is that high-quality uncertainty estimates on both OOD and CID data can be obtained in a single forward pass, with 3.1$a_D$3 smaller models and 3.8$a_D$4 fewer FLOPS compared to the best prior early-exit ensemble (Ghanathe et al., 2024).

The paper’s own limitations are explicit. It notes a slight drop in in-distribution calibration versus large deep ensembles on very large datasets such as Tiny-Imagenet. It also notes that training is more complex, because of the parameter copies and loss-weight scheduling. Finally, it identifies the absence of real-hardware microcontroller timing results as future work. These caveats matter because they delimit the scope of the reported efficiency claims: QUTE is optimized for deployment viability under TinyML constraints, but the training recipe is not simpler than baseline uncertainty methods, and the largest-scale calibration trade-offs remain visible (Ghanathe et al., 2024).

A plausible implication is that QUTE is most attractive where on-device monitoring is mandatory, labels are absent, and the marginal resource cost of conventional ensembles is unacceptable. In that regime, the method’s central compromise is not between accuracy and uncertainty alone, but between uncertainty quality and deployable systems overhead.

7. Nomenclature and related name collisions

The name “Qute” is ambiguous in arXiv-indexed literature and should be distinguished from several unrelated works. “Qute: Towards Quantum-Native Database” describes a quantum database vision in which quantum computation is treated as a first-class execution option throughout the database stack (Chen et al., 16 Feb 2026). “Qutes: A High-Level Quantum Programming Language for Simplified Quantum Computing” introduces a high-level language built upon Qiskit for quantum algorithm development (Faro et al., 17 Mar 2025). “QuTE: decentralized multiple testing on sensor networks with false discovery rate control” concerns decentralized multiple hypothesis testing on graphs with FDR guarantees (Ramdas et al., 2022). “QuEPT: Quantized Elastic Precision Transformers with One-Shot Calibration for Multi-Bit Switching” addresses elastic post-training quantization for Transformers and LLMs (Xu et al., 13 Feb 2026).

Within this literature, QUTE in the TinyML sense refers specifically to uncertainty quantification with early-exit-assisted ensembles for model monitoring on resource-constrained microcontrollers (Ghanathe et al., 2024).