Speedup Illusion in Systems

• Speedup illusion is a phenomenon where reported local acceleration fails to enhance overall task performance due to misaligned metrics.
• It occurs in domains such as robotics, embodied AI, and LLM decoding, where faster micro-level operations can result in degraded throughput or lower success rates.
• Key challenges include hidden overheads, synchronization delays, and quality degradation that obscure genuine speed improvements.

Speedup illusion denotes a family of phenomena in which reported, nominal, or expected acceleration fails to yield the relevant end-to-end gain. Across current research, the term is used for mismatches between local rate improvements and task-level outcomes in robotic control, embodied inference, speculative decoding, human-AI interaction, and parallel computing; it also appears, in a different constructive sense, in systems that deliberately replace exact dynamics or exact signals with functionally equivalent surrogates (Arachchige et al., 13 Jun 2025, Liu et al., 31 Dec 2025, Wang et al., 26 Jun 2026, Yu et al., 22 May 2026).

1. Conceptual scope

In its dominant diagnostic sense, speedup illusion refers to a discrepancy between a visible or reported acceleration and the objective that actually matters. In robotic imitation learning, nominally faster execution can reduce effective throughput because success rates drop, references become untrackable, motion becomes jerky, or latency causes pauses; the result may be higher nominal speed but lower task throughput (Arachchige et al., 13 Jun 2025). In speculative decoding for LLMs, apparent acceleration can be bounded away by the draft–target speed ratio and further eroded by low acceptance, rollback, synchronization overhead, and verification cost (Shen et al., 9 Jan 2026, Liu et al., 31 Dec 2025). In embodied AI more broadly, per-step inference acceleration can lengthen end-to-end completion time or degrade task success because rollout length and observation-to-action delay change the closed-loop system itself (Wang et al., 26 Jun 2026). In human-AI interaction, the illusion is cognitive rather than algorithmic: people accurately forecast independent completion time but underestimate AI-assisted completion time, so expected time savings exceed realized ones (Yu et al., 22 May 2026).

The concept therefore differs from simple underperformance. A system may genuinely execute some inner loop faster while still exhibiting a speedup illusion if the benchmarked proxy is not aligned with the operational objective. This suggests that the core issue is not speed in isolation, but the choice of objective, baseline, and measurement granularity.

A second, constructive usage appears in several engineering literatures. "Mirage" transmits semantics rather than pixels and relies on generative reconstruction at the receiver, so the "illusion" is a perceptual substitute for raw video transmission (Wu et al., 7 Feb 2026). "Photonic Temporal Illusion" uses space-time modulation of permittivity to reproduce the response of a target time-invariant structure while modifying transient behavior, including faster energy accumulation in high-$Q$ settings (Ptitcyn et al., 8 Jan 2026). In these cases, illusion names an engineered representational substitution rather than a benchmarking failure.

2. Mechanisms that produce illusory acceleration

A recurring mechanism is closed-loop degradation under faster operation. In faster-than-demonstration robotic control, smaller execution intervals alter robot dynamics, increase tracking error, and push the observation–action pair into out-of-distribution regions relative to training. SAIL formalizes this through the shift $$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$, and uses tracking error $e(t)$ as a proxy for this mismatch; the same study identifies controller-dependent targets, latency-induced action depletion, and high-gain overshoot as direct sources of illusory speedup (Arachchige et al., 13 Jun 2025). TISED makes a parallel point for embodied inference: reducing inference latency changes not only per-chunk time but also rollout length $N$ and action delay $D_{\mathrm{act}}$, so end-to-end gains can reverse sign when degraded action quality forces retries or longer trajectories (Wang et al., 26 Jun 2026).

A second mechanism is pipeline waste. In speculative decoding, the nominal appeal of proposing many tokens is limited by acceptance dynamics and the architecture of verification. For PSD, the formal ceiling $S_{\mathrm{PSD}} \le C$ implies that the realized speedup cannot exceed the draft-to-target speed ratio, even before accounting for rejections (Shen et al., 9 Jan 2026). Production-scale measurements on vLLM show an additional source of erosion: verification accounts for $42\%$–$95\%$ of end-to-end time, while acceptance length varies substantially across positions, requests, and datasets, so fixed-$k$ proposals waste verification compute on rejected positions (Liu et al., 31 Dec 2025).

A third mechanism is hidden overhead. In empirical parallel performance analysis, observed speedup decomposes as

$S_p = \frac{p\,T_{\mathrm{seq}}}{T_1 + F_p + I_p},$

where $$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$0 is total idle time and $$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$1 is work inflation induced by communication, synchronization, and memory effects (Acar et al., 2017). Related work on the memory wall shows that naïve Amdahl-style models become misleading when average data-access delay depends on core count and on the processor–memory frequency ratio $$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$2; under those conditions, adding cores or raising CPU frequency can lower realized speedup rather than improve it (Furtunato et al., 2019). Parallel-in-time methods add another layer of vulnerability: over-resolution, benign parameter regimes, optimistic baselines, and ignored communication costs can make performance look substantially better than it is (Goetschel et al., 2021).

A fourth mechanism is human-time decomposition rather than machine-time decomposition. In AI-assisted task completion, the total assisted time is

$$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$3

with prompt construction time, model generation time, and post-response reading or processing time. The study on cognitive offloading reports that $$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$4 was minimal, mean $$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$5 seconds, while most of the time cost lay in prompting and post-response processing; the logic-problem condition was dominated by post-response processing because verbose outputs increased interpretation cost (Yu et al., 22 May 2026). The resulting illusion is not that the model generates slowly, but that users ignore the surrounding coordination time.

3. Measurement, bounds, and diagnostics

Because speedup illusion is a measurement problem as much as a systems problem, the literature repeatedly replaces single proxy metrics with structured diagnostic sets.

In robotic imitation learning, SAIL evaluates Success Rate (SR), ATR, Speedup-over-demo (SOD), Throughput-with-regret (TPR), and motion-smoothness or consistency metrics including SPARC, LDLJ, CON, and WED. The explicit criterion is that a method with higher SOD but low SR or negative TPR is a speedup illusion (Arachchige et al., 13 Jun 2025). In embodied inference, TISED decomposes end-to-end speedup into a per-chunk factor and a rollout-length factor,

$$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$6

which makes visible the possibility that faster chunks can still produce slower tasks if $$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$7 rises sufficiently (Wang et al., 26 Jun 2026).

In LLM decoding, the literature distinguishes theoretical ceilings, oracle ceilings, and production measurements. PSD is bounded by $$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$8, whereas production-grade SD studies report throughput ratios, Mean Accepted Tokens, and oracle upper bounds such as

$$p_{\mathrm{exec}}(s,a \mid \Delta_t \ll \Delta^*) \neq p_{\mathrm{data}}(s,a)$$9

under memory-bound assumptions (Shen et al., 9 Jan 2026, Liu et al., 31 Dec 2025). The point is not merely that a method is slower than theory, but that end-to-end throughput can diverge sharply from microbenchmark intuition once verification and batching dominate.

In human-AI interaction, the relevant observables are not only completion time but also miscalibration and effort. The study defines a speedup ratio $e(t)$0, an illusion gap $e(t)$1, and assisted-time miscalibration $e(t)$2. It also shows that completion time and NASA-TLX effort are only weakly correlated, $e(t)$3, so subjective ease is not a reliable proxy for elapsed time (Yu et al., 22 May 2026).

In empirical parallel computing, factored speedup analysis replaces a single $e(t)$4 curve with the components $e(t)$5, $e(t)$6, and $e(t)$7, or equivalently with maximal, idle-time-specific, inflation-specific, and actual speedup curves (Acar et al., 2017). In domain decomposition, standard efficiency $e(t)$8 is explicitly rejected as the wrong yardstick for divide-and-conquer designs; the recommended metric is DC efficiency,

$e(t)$9

because the relevant ideal is $N$0, not $N$1 (Herrera-Revilla et al., 2019).

Setting	Nominal speed indicator	Counter-diagnostic
Faster robotic execution	SOD, shorter wall-clock time	SR, TPR, SPARC, LDLJ, CON, WED
Speculative decoding	Draft rate, fixed-$N$2 speedup	$N$3, MAT/AMT, wall-time throughput
Embodied inference	Lower $N$4	$N$5, $N$6, $N$7, $N$8
Human-AI assistance	Expected time savings	$N$9, $D_{\mathrm{act}}$0, hidden timer, NASA-TLX dissociation
Parallel computing	Raw $D_{\mathrm{act}}$1 curve	$D_{\mathrm{act}}$2, $D_{\mathrm{act}}$3, memory-wall or DC-efficiency analysis

Taken together, these frameworks suggest that speedup illusion is best understood as a failure of measurement design: an acceleration claim is fragile if it is not accompanied by explicit accounting for failure modes, overhead terms, or objective misalignment.

4. Domain-specific manifestations

In robotics, the phenomenon is directly observable in throughput metrics. SAIL reports up to $D_{\mathrm{act}}$4 speedup in simulation and up to $D_{\mathrm{act}}$5 in the real world, but it also gives examples where nominal acceleration and effective throughput diverge. On Wiping Board, SAIL has SOD $D_{\mathrm{act}}$6 versus DP-Fast $D_{\mathrm{act}}$7, yet SR is $D_{\mathrm{act}}$8 versus $D_{\mathrm{act}}$9 and TPR is $S_{\mathrm{PSD}} \le C$0 versus $S_{\mathrm{PSD}} \le C$1, which the paper treats as a mixed outcome rather than an unqualified gain. It further notes cases where DP-Fast shows higher SOD but negative TPR, such as Stacking Cups and Folding Cloth, thereby exemplifying the intended meaning of speedup illusion (Arachchige et al., 13 Jun 2025).

In production LLM inference, the phenomenon appears as a gap between laboratory-style optimism and deployed throughput. The systematic vLLM study reports consistent $S_{\mathrm{PSD}} \le C$2–$S_{\mathrm{PSD}} \le C$3 end-to-end speedups at realistic settings, while oracle upper bounds range from $S_{\mathrm{PSD}} \le C$4 to $S_{\mathrm{PSD}} \le C$5 depending on workload; verification dominates runtime, and acceptance length is position-dependent (Liu et al., 31 Dec 2025). The "Double" framework responds to a more formal version of the same problem: PSD remains limited by $S_{\mathrm{PSD}} \le C$6, and mid-sequence rejections produce rollback waste and pipeline stalls. By introducing draft-side iterative retrieval and target-side authoritative retrieval, Double raises the effective ceiling to $S_{\mathrm{PSD}} \le C$7 and reports $S_{\mathrm{PSD}} \le C$8 on LLaMA3.3-70B and $S_{\mathrm{PSD}} \le C$9 on Qwen3-32B while remaining training-free and lossless (Shen et al., 9 Jan 2026).

In embodied AI, TISED identifies two paradoxes. On static tasks, stronger lightweighting can reduce per-chunk latency monotonically while producing a U-shaped $42\%$0 because $42\%$1 eventually inflates. On dynamic tasks, moderate lossy optimization can improve success above baseline because reduced $42\%$2 lowers staleness exposure $42\%$3, even if action quality declines slightly (Wang et al., 26 Jun 2026). In human-AI interaction, an analogous discrepancy appears at the user level: participants predicted AI assistance to be faster than independent completion by $42\%$4 seconds on average, but in the completion sample actual AI-assisted time exceeded predicted time by $42\%$5 seconds, while overall actual completion times did not differ between independent and AI-assisted completion on easy tasks (Yu et al., 22 May 2026).

Parallel computing provides older but structurally similar examples. The PinT literature catalogs experimental practices that inflate apparent performance: selecting parameters that drive the solution to a steady state, adding diffusion, over-resolving in time or space, using over-stringent solver tolerances, reporting theoretical instead of measured speedups, or benchmarking against weak serial baselines (Goetschel et al., 2021). In shared-memory performance analysis, the corresponding hazards are idle time, work inflation, and memory effects that are invisible in a bare speedup curve (Acar et al., 2017). Memory-wall modeling sharpens the same point by showing that naïve expectations such as "higher CPU frequency always helps" can fail once $42\%$6 drives memory delay high enough to cap the attainable speedup (Furtunato et al., 2019).

5. Genuine speedup, supralinearity, and engineered illusion

A central misconception is that every surprising speedup should be treated as an illusion. Several literatures explicitly reject that conclusion.

In divide-and-conquer domain decomposition, supralinear speedups are argued to be expected rather than suspicious. The proposed benchmark is not $42\%$7 but

$42\%$8

For problems with $42\%$9 and $95\%$0, the divide-and-conquer target becomes $95\%$1. The reported DVS-BDDC experiments reach $95\%$2 at $95\%$3 and $95\%$4 at $95\%$5, with DC-efficiency values from $95\%$6 down to $95\%$7; within that framework, the speedups are "perfectly normal" and not artifacts (Herrera-Revilla et al., 2019).

A related distinction appears in formal parallel-algorithm analysis. For a fixed DAG with serial work $95\%$8 and span $95\%$9, the bound

$k$0

formally excludes true superlinear speedup for a fixed algorithm and problem instance. Apparent superlinearity must then come from changed effective work, changed node costs, memory hierarchy effects, or inconsistent baselines rather than from violation of the bound itself (Gunther, 2011). This complements, rather than contradicts, the divide-and-conquer argument: the DDM paper changes the performance goal because decomposition changes problem size and effective work per processor (Herrera-Revilla et al., 2019).

Several papers also use "illusion" positively, as a deliberate surrogate that preserves a target property while changing implementation. SLPIC slows fast particles so that explicit timesteps can be much larger, and in an argon plasma sheath benchmark it achieved a wall-clock speedup of $k$1 without loss of accuracy for the target slow phenomenon (Werner et al., 2015). Mirage replaces raw video by captions and compact semantic anchors, then synthesizes the video at the receiver; in the reported UCF101 setting, Mirage (VQ-VAE-256) reduced transmitted payload from $k$2 KB to approximately $k$3 KB and achieved a data-speedup of about $k$4 (Wu et al., 7 Feb 2026). Photonic Temporal Illusion similarly uses time-varying permittivity to reproduce the response of a target structure while altering the transient trajectory, including faster accumulation in high-$k$5 systems (Ptitcyn et al., 8 Jan 2026). In these cases, the "illusion" is engineered equivalence at a chosen abstraction level.

The same distinction appears in quantum information. The paper on the mechanism of quantum speed-up argues that the speed-up is genuine and that the illusion lies instead in the apparent determinism of the unitary map when initial and final measurements are ignored (Castagnoli, 2011). The proposed quantum-simulation architectures for 2D Ising-type quenches likewise frame their claim as a genuine quantum speedup, supported by complexity-theoretic hardness and explicit certification rather than by informal benchmark inflation (Bermejo-Vega et al., 2017).

6. Boundary conditions and open problems

The persistence of speedup illusion across domains reflects unresolved modeling and benchmarking difficulties. In robotic imitation learning, open issues include changing robot–object interaction dynamics at higher speed, sensing bandwidth and multi-modal synchronization limits, torque limits and safety constraints, and finite offline coverage of the state–action space (Arachchige et al., 13 Jun 2025). In embodied efficient inference, TISED abstracts scheduling jitter, OS-level latencies, and queueing, and broader physical validation as well as parametric identification of $k$6 remain open (Wang et al., 26 Jun 2026).

In LLM decoding, both theoretical and systems gaps remain prominent. The production vLLM study identifies verification cost and fragmented, position-dependent acceptance as the dominant sources of gap between practical speedups and oracle ceilings, and points to adaptive proposers, acceptance predictors, and cheaper verification as the main routes forward (Liu et al., 31 Dec 2025). Double reduces rollback and ceiling effects, but its gains diminish when retrieval coverage is poor, when environments force serialization, or when AMT remains low (Shen et al., 9 Jan 2026).

In human-AI interaction, the present evidence is limited to short tasks and a crowdworking setting. The study explicitly calls for mapping the complexity boundary at which AI reliably saves both time and effort, modeling heterogeneous prompting strategies, and using careful within-subject designs to study calibration (Yu et al., 22 May 2026). In parallel-in-time and parallel performance more generally, the open problem is methodological: many misleading practices are still possible unless baselines, tolerances, communication costs, and cost-versus-accuracy curves are all reported explicitly (Goetschel et al., 2021, Acar et al., 2017).

A broad implication is that speedup illusion is not a single failure mode but a recurrent structural pattern. It arises whenever a local proxy—tokens per draft round, control frequency, per-step latency, subjective effort reduction, or raw speedup curve—is substituted for the end-to-end objective without accounting for instability, overhead, quality loss, or baseline mismatch. The mature response in the literature is therefore not merely to demand faster systems, but to require objective-aligned metrics, explicit ceilings, failure accounting, and, where possible, certification of what has actually been sped up.