Latency Analysis of State-of-the-Art CUAs

Updated 30 July 2025

The paper highlights an integrated analysis of CUAs by merging transmission, computational, and behavioral latency metrics to quantify trade-offs in decoding complexity and power efficiency.
Latency evaluation reveals that optimizing DNN inference, through hardware-aware scheduling and mitigation of GPU tail effects, can reduce processing delays by up to 27%.
Advanced caching mechanisms and adaptive networked designs show significant latency reductions in bursty traffic scenarios and enhance benchmarking of multi-traffic systems.

Latency analysis for state-of-the-art Communication and Utility Architectures (CUAs) encompasses diverse systems including wireless transmission, edge inference, caching, and computer-use agents. The current research trajectory extends latency analysis from classical network transmission metrics to computational, architectural, and behavioral bottlenecks, with methods spanning rigorous analytical models, empirical evaluation, and optimization-aware design. The following sections address the principal advances and findings in latency analysis, drawing on recent literature.

1. Computational Complexity and Decoding Latency in Communication Systems

Latency in ultra-reliable low-latency communication (URLLC) systems is determined not only by the channel transmission time but also by the decoding duration required at the receiver. The total latency for a single codeword transmission is expressed as: $d_t = n T_s + k c T_b$ where:

$n$ is the codeword blocklength,
$T_s$ is the symbol time,
$k$ is the number of information bits,
$c$ is the number of binary operations per bit (decoding complexity),
$T_b$ is the hardware-specific time per binary operation.

For extended BCH codes with Ordered Statistics (OS) decoding, $c$ is given by: $c = \frac{k^2}{8} + \frac{n}{2} \sum_{i=0}^s \binom{k}{i}$ where $s$ is the OS decoding order. The careful selection of $s$ directly trades off performance (in terms of block error rate, BLER) and decoding time. Higher $s$ improves performance but amplifies latency through increased computational load.

An empirical trade-off between decoding complexity and additional required transmit power for fixed BLER is modeled as: $\log_2(c) = \frac{1}{a (\Delta \rho)^{\gamma} + b}$ with $a, b, \gamma$ fitted per code blocklength. This quantifies how stringent computational constraints shift optimal operating points, often reducing the achievable transmission rate under tight latency deadlines by over 80% relative to classical information-theoretic predictions without decoding delay (Celebi et al., 2019).

2. Hardware-Dependent DNN and Edge Inference Latency

End-to-end latency for DNN-based CUAs on edge devices depends critically on the match between workload granularity and hardware capabilities. On GPUs, the "GPU tail effect" describes the phenomenon where the last wave of thread blocks cannot fully utilize all streaming multiprocessors (SMs), resulting in substantial idle time and a non-linear "latency staircase." The latency for a layer under this scheduling is: $L = A_l \cdot \left\lceil \frac{B}{S} \right\rceil$ with $A_l$ as processing cycle duration, $B$ thread blocks, and $S$ SMs (Yu et al., 2020).

Optimizing DNNs for latency thus requires joint consideration of layer widths and hardware scheduling, not merely FLOP reduction. Methods that eliminate the GPU tail effect have demonstrated $11-27\%$ latency reduction and $2.5-4\%$ accuracy improvement over baseline pruning/NAS.

Further, runtime-aware predictors such as MAPLE-Edge (Nair et al., 2022) use a dense set of hardware counters normalized by operator latency, achieving up to $+49.6\%$ accuracy gain for latency prediction in edge deployment with only 10 samples. Operation-wise predictors (Li et al., 2022) decompose networks into kernel-level constituents, modeling overhead such as kernel fusion, and achieve mean absolute percentage errors (MAPE) below $3.2\%$ (CPU) and $6.7\%$ (GPU) even with minimal profiling.

3. Caching with Delayed Hits: Mean and Variance-Aware Latency Optimization

Modern caching systems, notably in CDNs and MEC, must contend with "delayed hits"—a surge in requests during miss-induced fetches that lead to bursty user-perceived latency. Analytical results under Poisson arrivals with constant latency $z_i$ and rate $\lambda_i$ yield: $E[D_i] = z_i + \lambda_i z_i^2$

$\text{Var}(D_i) = z_i^2 + 6 \lambda_i z_i^3 + 5 \lambda_i^2 z_i^4$

for deterministic $z_i$ (Jiang et al., 29 Apr 2025), and similar results extend to exponential $z_i$ (fetches) (Jiang et al., 21 May 2025).

Variance-aware ranking functions for eviction are formulated as: $f_i = \frac{E[D_i] + \omega \sigma[D_i]}{R_i s_i}$ with online estimations for residual inter-arrival time $R_i$ , object size $s_i$ , and variability weight $\omega$ . Such methods achieve $3\%-30\%$ latency reduction in synthetic workloads and up to $7\%$ on real traces, outperforming mean-only estimators especially in bursty, high-variance traffic (Jiang et al., 21 May 2025).

4. Control of Latency in Model Inference, Streaming, and Keyword Spotting

In streaming and real-time applications, latency is directly linked to user experience and is optimized jointly with reliability or utility objectives.

Keyword Spotting: A latency-aware loss function introduces a hyperparameter $b$ , shifting the detection peak earlier along the audio stream via:

$t = \max(\arg\max_i p_{y,i} - \beta, 0), \quad \beta \sim \text{Bernoulli}(b)$

Adjusting $b$ yields a tunable trade-off between detection latency and false accept rates, showing a $25\%$ relative improvement for fixed latency targets over baseline losses (Jose et al., 2022).

Speech Recognition: In sequence transducers (e.g., RNN-T, Conformer-T), the minimum latency training regime augments the loss with an expected delay term. The expected latency is computed along diagonals in the lattice using forward-backward probabilities, with gradients efficiently calculated and added as regularization:

$L_{MLT} = L_{trans} + \lambda_{MLT} \cdot \ell_{t+u+1}$

This approach cut streaming latency from $220$ms to $27$ms (PR90) with less than $0.7\%$ WER degradation (Shinohara et al., 2022).

Streaming S2S Translation: Latency spikes are predominantly caused by model hallucinations, which are minimized using strategies such as enforcing a minimum input window (e.g., $0.7$s), lookback context, and commitment duration constraints. Latency evaluation uses average lagging (AL) and its differentiable counterpart (DAL) as metrics (Wilmet et al., 2 Sep 2024).

5. Latency Analysis in Multi-Traffic and Adaptive Networked Systems

Latency analysis under multi-traffic (eMBB, URLLC) coexistence is especially prominent in 5G industrial deployments. Dynamic TDD (time division duplexing) frame selection, QoS-aware user scheduling, and fine-tuning of UL power control significantly improve latency profiles:

Service-aware TDD with buffered traffic partitioning and head-of-line delay-based scheduling achieves up to $68\%$ reduction in URLLC outage latency compared to conventional schemes (Esswie et al., 2020).
In microgrid adaptive protection, wired links still yield sub-4ms end-to-end delays, but with the maturation of 5G URLLC (12–20ms with 99.999% reliability), wireless methods with multi-connectivity and network slicing will become central for ultra-low latency protection responses (Gutierrez-Rojas et al., 2020).

6. Evaluation, Benchmarking, and Practical Recommendations

The assessment of CUAs involves empirical latency benchmarking (e.g., OSWorld-Human for computer-use agents (Abhyankar et al., 19 Jun 2025), DASH.js for video streaming (O'Hanlon et al., 2023)), which reveal that existing agents and algorithms, while successful in task completion or video quality, often incur much higher latencies than human or lower-level system baselines.

Computer-Use Agents: The dominant component of latency (75%-94%) is attributed to large-model planning and reflection calls, with prompt size (history length) causing later steps to take up to $3\times$ longer. Weighted Efficiency Score (WES) is introduced to quantify efficiency in step counts relative to human trajectories:

$\text{WES}^+ = \sum_t {r_t (t_{\text{exp}}/t_{\text{actual}})}, \quad \text{WES}^- = \sum_t {-(1 - r_t)(t_{\text{actual}}/S)}$

Even top agents require $1.4$– $2.7\times$ more steps than human baselines, amplifying the LLM planning/reflection latency (Abhyankar et al., 19 Jun 2025).

Adaptive Bitrate Streaming: At very low latency targets (e.g., 3s), video players such as dash.js experience increased stalling. Default Dynamic ABR algorithms maintain latency closer to target and higher QoE than alternatives (L2A-LL, LoL+), with modifications targeting throughput outlier filtering and state update suppression further improving performance (O'Hanlon et al., 2023).

7. Perspectives and Future Research

Emerging directions include:

Extension of variance-aware latency models to non-Poisson or heavy-tailed fetches in caching.
Adaptive parameter estimation and tuning (e.g., variance penalty, residual estimation window size) for robust real-world deployment under traffic non-stationarity.
Cross-layer and holistic latency models that jointly consider transmission, computation, system scheduling, and application behaviors.
Learning-based or hybrid analytical-ML predictors for latency in highly heterogeneous, composable CUA environments.
Benchmarking frameworks (e.g., OSWorld-Human, QoE models for streaming) to drive efficiency improvements and prioritize latency instead of accuracy alone.

Advances in latency analysis continue to rigorously formalize, quantify, and optimize the end-to-end responsiveness of CUAs, spanning physical layer transmission, computational blocks, and application-level agents. Incorporating the full spectrum of computational and protocol constraints—along with empirical benchmarking—remains essential for future progress in latency-sensitive systems.

Summary Table: Key Analytical Latency Models

Domain	Principal Latency Formula	Main Variables (examples)
Wireless Comm.	$d_t = nT_s + k c T_b$	$n,T_s,k,c,T_b$
Edge DNN/GPU	$L = A_l \lceil B/S \rceil$	$A_l,B,S$
Caching	$E[D_i] = z_i + \lambda_i z_i^2$	$z_i,\lambda_i$
Speech Trans.	$L_{MLT} = L_{trans} + \lambda_{MLT} \ell$	$L_{trans},\lambda_{MLT},\ell$