QuickDivergence: Fast Divergence Computation Methods

• QuickDivergence is a design principle that enables rapid estimation of divergence-like quantities by harnessing structured models and asymptotic rates.
• It replaces costly operations like alignment, Monte Carlo probing, and sorting with efficient streaming, spectral, or greedy thresholding techniques.
• Applications include PFSA-based time-series comparisons, fast process distinguishability via co-emission rates, AMP divergence estimation, and linear-time monetary risk measures.

QuickDivergence is a label used in several technically distinct research programs to denote fast procedures for computing, estimating, or exploiting divergence-like quantities. In the literature represented here, it refers most prominently to a PFSA-based time-series comparison method built from Sequence Likelihood (SL) divergence and implemented as Smash2.0; to a process-distinguishability framework based on divergence rates and, in particular, the co-emission divergence rate; to low-overhead divergence estimators for Onsager corrections in scalable message passing; and to an expected linear-time algorithm for computing certain $\varphi$-divergence monetary risk measures such as CVaR and TVaR for discrete random variables (Huang et al., 2019). The shared theme is computational acceleration: each version replaces a more costly baseline—such as alignment, Monte Carlo probing, sorting, or long-horizon path comparison—by a construction whose dominant operations are linear-time passes, low-dimensional spectral calculations, or one-shot offline corrections.

1. Time-series QuickDivergence as Smash2.0

In "Data Smashing 2.0: Sequence Likelihood (SL) Divergence For Fast Time Series Comparison" (Huang et al., 2019), QuickDivergence denotes the fast sequence-level realization of SL divergence. The setting is discrete-valued stochastic processes indexed by time $t\in\mathbb{N}_+$ over a finite alphabet $\Sigma$, assumed strictly stationary and ergodic. Continuous-valued time series are first quantized into a finite alphabet before modeling. The hidden-generator class is the probabilistic finite-state automaton (PFSA) $G=(\Sigma,Q,\delta,\tilde{P})$, with deterministic next-state map $\delta(q,\sigma)$ defined when $\tilde{P}(q,\sigma)>0$; PFSA strictly generalize finite-order Markov chains and approximate hidden Markov models arbitrarily well.

The central divergence is the Sequence Likelihood divergence, defined for two PFSA generators $P$ and $Q$ by

$$D_{\mathrm{SL}(P\|Q)}=\lim_{n\to\infty}\frac{1}{n}\,\mathbb{E}_{x\sim P}\Big[\log \frac{P(x)}{Q(x)}\Big] = D_{\mathrm{KL}(P\|Q)}.$$

This is a per-symbol KL divergence rate for stationary ergodic sequential data rather than an i.i.d. distributional divergence. For irreducible PFSA $P,Q$ and $t\in\mathbb{N}_+$0, the per-symbol negative log-likelihoods satisfy

$t\in\mathbb{N}_+$1

so that

$t\in\mathbb{N}_+$2

QuickDivergence appears as Option A in the paper’s algorithmic procedure. Rather than inferring a generator for every sequence pair, Smash2.0 fixes a small base set of PFSAs, $t\in\mathbb{N}_+$3, and maps each observed sequence $t\in\mathbb{N}_+$4 to

$t\in\mathbb{N}_+$5

The sequence-level distance is then taken as the $t\in\mathbb{N}_+$6 distance

$t\in\mathbb{N}_+$7

Each $t\in\mathbb{N}_+$8 is computed in a single streaming pass through the sequence, yielding time $t\in\mathbb{N}_+$9 and memory $\Sigma$0, where $\Sigma$1 is sequence length. The paper explicitly identifies this as the source of the method’s speed: “no alignment (DTW) and no anti-stream algebra (original data smashing)” (Huang et al., 2019).

The same work also gives a generator-level estimate

$\Sigma$2

which is consistent under stationarity, ergodicity, and irreducibility, and a closed-form PFSA expression

$\Sigma$3

The resulting methodology is explicitly featureless, alignment-free, and domain-free, with preprocessing limited to symbolization, optional detrending or normalization, and quantizer selection via the ratio $\Sigma$4 on labeled training data.

2. Divergence rates and process distinguishability

A second use of QuickDivergence concerns fast distinguishability measures for stochastic processes. In "Measures of distinguishability between stochastic processes" (Yang et al., 2019), the relevant framework is the family of divergence rates

$\Sigma$5

where $\Sigma$6 is a continuous, normalized metric distance between the length-$\Sigma$7 sequence distributions of two stationary ergodic processes.

The paper’s principal instance is the co-emission divergence rate (CDR),

$\Sigma$8

with co-emission probability

$\Sigma$9

For HMM representations of $G=(\Sigma,Q,\delta,\tilde{P})$0 and $G=(\Sigma,Q,\delta,\tilde{P})$1, the method becomes spectral. Defining transfer matrices

$G=(\Sigma,Q,\delta,\tilde{P})$2

and similarly $G=(\Sigma,Q,\delta,\tilde{P})$3, with leading eigenvalues $G=(\Sigma,Q,\delta,\tilde{P})$4, one obtains

$G=(\Sigma,Q,\delta,\tilde{P})$5

This construction is “quick” in a different sense from Smash2.0. Its complexity is polynomial in model size and independent of the observed sequence length $G=(\Sigma,Q,\delta,\tilde{P})$6: building the transfer matrix costs $G=(\Sigma,Q,\delta,\tilde{P})$7, and each power-iteration step costs $G=(\Sigma,Q,\delta,\tilde{P})$8 (Yang et al., 2019). The method is symmetric, finite unless perfect discrimination is possible in finite $G=(\Sigma,Q,\delta,\tilde{P})$9, and continuous in model parameters. The paper emphasizes that CDR remains well-behaved when KL divergence is infinite or undefined due to support mismatch, alphabet mismatch, or topology differences.

A plausible implication is that this version of QuickDivergence occupies a complementary niche to PFSA-based SL divergence: both are process-level rather than path-level, but one is organized around per-symbol log-likelihood rates and the other around exponential decay of similarity.

3. QuickDivergence in scalable message passing

In "Divergence Estimation in Message Passing algorithms" (Skuratovs et al., 2021), QuickDivergence refers to low-cost estimators of the denoiser divergence required by the Onsager correction in AMP-, OAMP-, and VAMP-type algorithms. The inverse problem is

$\delta(q,\sigma)$0

with $\delta(q,\sigma)$1, $\delta(q,\sigma)$2, and large-system-limit assumptions on $\delta(q,\sigma)$3 and the denoiser $\delta(q,\sigma)$4. The normalized divergence is

$\delta(q,\sigma)$5

Conventional estimation uses the BB-MC Hutchinson-style probe

$\delta(q,\sigma)$6

which requires an extra denoiser call per iteration.

The paper proposes two alternatives requiring no additional denoiser executions. The algebraic estimator is

$\delta(q,\sigma)$7

with practical choices such as $\delta(q,\sigma)$8. The polynomial estimator derives from a quadratic indication equation

$\delta(q,\sigma)$9

where

$\tilde{P}(q,\sigma)>0$0

and $\tilde{P}(q,\sigma)>0$1 abbreviates $\tilde{P}(q,\sigma)>0$2. Root selection is given by

$\tilde{P}(q,\sigma)>0$3

with fallback $\tilde{P}(q,\sigma)>0$4 when the roots are complex.

This QuickDivergence is computationally defined by replacement of BB-MC denoiser probes with inner products or reused matrix–vector products. The algebraic estimator adds only $\tilde{P}(q,\sigma)>0$5 work; the polynomial estimator is dominated by matrix–vector multiplies with $\tilde{P}(q,\sigma)>0$6 and $\tilde{P}(q,\sigma)>0$7, often already available in the surrounding iteration. Empirically, the polynomial method gives similar or superior correction compared to BB-MC and about $\tilde{P}(q,\sigma)>0$8 speedup in AMP and MF-OAMP when denoiser cost dominates (Skuratovs et al., 2021).

4. QuickDivergence for $\tilde{P}(q,\sigma)>0$9-divergence monetary risk measures

In "Computing Monetary Risk Measures in Linear Time" (Agrawal et al., 6 Jul 2026), QuickDivergence is an expected linear-time algorithm for computing a class of $P$0-divergence risk measures on a finite probability space $P$1 with $P$2 and reward vector $P$3. The generic $P$4-divergence is

$P$5

and the associated risk measure is

$P$6

The algorithmic tractability comes from a polymatroid formulation. For coherent and comonotonic risk measures,

$P$7

and the paper identifies a class of Element-Wise-Separable capacities

$P$8

QuickDivergence solves

$P$9

without sorting, by first calling QuickVaR to locate the threshold value $Q$0 at quantile level $Q$1 and then performing a single pass that allocates mass according to the EWS capacity.

The method specializes directly to CVaR and TVaR. For CVaR at level $Q$2, the capacity is EWS$Q$3 and

$Q$4

For TVaR, the capacity is EWS with $Q$5 and

$Q$6

The resulting runtime is expected $Q$7: one expected-linear QuickVaR call plus linear passes for set construction and weight assignment. The paper reports an order-of-magnitude speedup for large domains and provides implementations in RiskMeasures.jl (Agrawal et al., 6 Jul 2026).

A plausible implication is that this use of QuickDivergence is conceptually distinct from information-theoretic or dynamical divergence estimation: here “divergence” refers to the risk-ball constraint $Q$8, and the computational novelty lies in exploiting EWS polymatroid geometry rather than estimating a discrepancy between empirical generators.

5. Common algorithmic motifs across usages

Despite the heterogeneity of the applications, the different QuickDivergence constructions share several structural motifs. First, each reduces a high-cost comparison or correction problem to a lower-complexity primitive. Smash2.0 replaces pairwise model inference or alignment by streaming log-likelihood coordinates (Huang et al., 2019). CDR replaces long-sequence distribution comparison by leading eigenvalue computation on transfer matrices (Yang et al., 2019). AMP/OAMP divergence estimation replaces Monte Carlo probing by algebraic identities or a quadratic equation (Skuratovs et al., 2021). The monetary-risk version replaces sorting or generic convex optimization by Quickselect-style thresholding and polymatroid greedy structure (Agrawal et al., 6 Jul 2026).

Second, each method works with an “intensive” quantity. SL divergence is explicitly a per-symbol rate, invariant to sequence-length scaling. Divergence rates quantify exponential decay per observed symbol. Onsager divergence is normalized by $Q$9. The risk-measure version scales linearly in domain size rather than in the number of optimization iterations. This suggests that QuickDivergence is best understood not as a single algorithmic family, but as a recurrent design principle: extract the relevant asymptotic or structural rate, then compute it using a model class or combinatorial representation that admits efficient evaluation.

Third, each formulation is paired with explicit assumptions delimiting its valid regime. Stationarity and ergodicity are essential in the PFSA and divergence-rate settings (Huang et al., 2019, Yang et al., 2019). Large-system-limit randomness assumptions underlie the message-passing estimators (Skuratovs et al., 2021). EWS capacity structure is required for the expected linear-time monetary-risk algorithm (Agrawal et al., 6 Jul 2026). Consequently, the adjective “quick” should not be read as assumption-free; the speed gains arise precisely because the papers exploit restrictive but interpretable structure.

6. Limitations, scope, and terminological ambiguity

The term QuickDivergence is not standardized across arXiv usage. In the time-series setting it names Smash2.0, a universal metric over observed sequences built from PFSA likelihood coordinates (Huang et al., 2019). In process distinguishability it is naturally associated with efficient computation of divergence rates, especially the co-emission divergence rate (Yang et al., 2019). In message passing it denotes fast divergence estimators for denoisers within Onsager-corrected iterative algorithms (Skuratovs et al., 2021). In monetary risk it denotes an expected linear-time solver for $$D_{\mathrm{SL}(P\|Q)}=\lim_{n\to\infty}\frac{1}{n}\,\mathbb{E}_{x\sim P}\Big[\log \frac{P(x)}{Q(x)}\Big] = D_{\mathrm{KL}(P\|Q)}.$$0-divergence risk measures with EWS capacities (Agrawal et al., 6 Jul 2026). The same string therefore indexes distinct mathematical objects: KL-rate generalizations, metric-induced process distinguishability rates, Jacobian traces of denoisers, and risk-ball optimization under convex divergences.

Each usage carries specific limitations. PFSA-based QuickDivergence assumes stationarity and ergodicity, depends on symbolization for continuous data, and inherits the non-metric character of KL-based symmetrizations unless one uses the coordinate embedding (Huang et al., 2019). CDR requires HMM representations and transfer-matrix construction, with complexity polynomial in hidden-state size; its attractive properties do not eliminate model-specification issues (Yang et al., 2019). The algebraic Onsager estimator is sensitive at moderate problem sizes, and the polynomial estimator can yield complex roots near fixed points, requiring a stationary-point fallback (Skuratovs et al., 2021). The linear-time monetary-risk algorithm applies only to $$D_{\mathrm{SL}(P\|Q)}=\lim_{n\to\infty}\frac{1}{n}\,\mathbb{E}_{x\sim P}\Big[\log \frac{P(x)}{Q(x)}\Big] = D_{\mathrm{KL}(P\|Q)}.$$1-divergence risk measures admitting EWS capacities; EVaR does not fall into that subclass, although TVaR provides a lower bound (Agrawal et al., 6 Jul 2026).

For that reason, “QuickDivergence” is best treated as a disambiguation term rather than a single canonical concept. What unifies the literature is not one invariant definition of divergence, but a recurrent algorithmic agenda: compute a divergence-like object rapidly by exploiting hidden generator structure, transfer-matrix spectra, large-system identities, or polymatroid capacity geometry.