Adaptive KDE for Real-Time Thresholding: Prioritized Queues for Financial Crime Investigation

Abstract: We study the problem of converting a stream of risk scores into one or more review queues under explicit intake constraints[cite: 6]. Instead of top-$K$ or manually tuned cutoffs, we fit an online adaptive kernel density to the score stream, transform the density into a tail-mass curve to meet capacity, and ``snap'' the resulting cut to a persistent density valley detected across bandwidths[cite: 7]. The procedure is label-free, supports multi-queue routing, and operates in real time with sliding windows or exponential forgetting[cite: 8]. On synthetic, drifting, multimodal streams, the method achieves competitive capacity adherence while reducing threshold jitter[cite: 9]. Updates cost $O(G)$ per event with constant memory per activity

• The paper introduces a method using online adaptive Kernel Density Estimation (KDE) on the unit interval, tailored to financial crime investigations' capacity constraints and designed for real-time thresholding.
• Experiments demonstrate competitive capacity adherence within an error rate of 10%, outperforming baseline methods in threshold variability, intake variability, and backlog management, which highlights the durability of the methodology for operational needs.
• A notable limitation is the lack of quantitive performance metrics provided in tables; furthermore, the authors emphasized uncertainty and assumptions involved, creating an additional repository of challenges.

Problem and motivation

Detection systems in financial-crime operations emit continuous streams of risk scores that must be converted into review queues under explicit intake (capacity) constraints. The prevailing practice—top-$K$ selection or hand-tuned cutoffs—is fragile under distributional drift: a fixed cutoff can migrate from a sparse region of the score space into a dense one, producing abrupt swings in alert volume, and each additional queue requires ad-hoc retuning. Butvinik, Boateng, and Hackmon propose a density-anchored alternative: thresholds are placed not directly on scores but on an online kernel density estimate, at persistent valleys of the estimated density, subject to capacity targets. The procedure is label-free, streaming, and supports multi-queue routing.

The paper's contribution is a formalization rather than new statistical theory. It assembles classical tools—Rosenblatt–Parzen KDE, Epanechnikov kernels, Sheather–Jones plug-in bandwidths, Abramson adaptive bandwidths, boundary reflection, SiZer-style persistence, and Bayesian Blocks—and specializes them to the unit interval with online updates, then evaluates the result on operational criteria: capacity adherence, intake variability, cut jitter, backlog behavior, and runtime.

Method

Online adaptive KDE on $[0,1]$

Each business activity (BA) produces scores in $[0,1]$; the density is maintained on a fixed grid of $G$ points via exponential forgetting,

$$\hat{f}_t(x) = (1-\alpha)\hat{f}_{t-1}(x) + \alpha\left[K_{h_t}(x - s_t) + K_{h_t}(x + s_t) + K_{h_t}(x - (2-s_t))\right],$$

where the two reflected terms implement boundary correction. Reflection conserves mass exactly ($\int_0^1 \hat{f}_t = 1$) and avoids local regression at the edges; a sliding window of width $W$ is offered as an alternative with faster reaction but noisier estimates. Local bandwidths follow Abramson's square-root rule, $h_t(x) = h_{0,t}\sqrt{g_t/\tilde{f}_{b,t}(x)}$, computed from a pilot estimate updated identically, with clipping to $[h_{\min}, h_{\max}]$. The global scale $h_{0,t}$ is refreshed on a cadence using the Sheather–Jones plug-in (or an AMISE surrogate with effective sample size $[0,1]$0 under forgetting). Per-event cost is $[0,1]$1 with constant memory per BA.

Valley detection

Valleys are grid points where the first difference changes sign and the second difference is positive. Two filters suppress spurious minima: scale, enforced by requiring the sign pattern of $[0,1]$2 to persist over a bandwidth neighborhood (SiZer-style consistency), and salience, measured as persistence against bracketing maxima, $[0,1]$3, retained only above a data-driven threshold from a local variance proxy. Valley identity is tracked across time by nearest-neighbor matching with drift constraints.

Capacity matching and routing

For a target intake ratio $[0,1]$4, the capacity-true cut is the quantile $[0,1]$5 read off the tail-mass curve. The deployed cut snaps to the nearest admissible valley below $[0,1]$6 with sufficient tail mass, minimizing local elasticity $[0,1]$7; since intake sensitivity is $[0,1]$8, anchoring at minima directly minimizes volume sensitivity to cut motion. Overshoot from snapping is resolved by within-band percentile trimming. Two-threshold operation (Escalation / Standard / Hibernation) selects the pair of valleys jointly minimizing summed density while meeting both tail-mass constraints, again with intra-interval fine-tuning. Guardrails include hysteresis (a cut moves only if elasticity drops by a fraction $[0,1]$9 or the unconstrained quantile crosses a valley midpoint), minimum-support requirements, edge exclusion $[0,1]$0, and fallback to a plain quantile cut when no valleys survive. An optional label-aware refinement replaces the stability-only objective with per-band utility maximization, but the primary formulation is explicitly score-only.

Evaluation

Experiments use synthetic per-BA score streams generated as Beta mixtures with regime shifts, seasonal oscillations, and discretization, covering unimodal skewed, bimodal narrow-valley, and trimodal crowded-tail shapes. Baselines comprise batch top-$[0,1]$1, sliding-window quantiles (Greenwald–Khanna sketch), fixed-bandwidth KDE with reflection, EWMA cuts, and non-reflected KDE (to isolate edge bias). Metrics include relative capacity deviation, intake coefficient of variation, cut jitter, elasticity at deployment, backlog exceedance probability under a fixed review rate, tie-break volatility under discretized scores, and cross-BA dispersion of elasticities.

The reported findings are qualitative rather than tabulated: the method achieves competitive capacity adherence within a $[0,1]$2 tolerance band while reducing threshold jitter relative to quantile baselines, with controlled backlog under routine variability. Ablations isolate adaptive versus fixed bandwidth, reflection versus none, snapping versus plain quantiles, and hysteresis on versus off, alongside stress tests for score surges, mode merging, and coarser rounding. Runtime profiling confirms $[0,1]$3 per-event updates with constant memory per BA. All results are reported as medians and interquartile ranges over random seeds. Notably, no numerical tables appear in the paper; quantitative claims are supported only through figures, which limits independent verification of the magnitude of jitter reduction or adherence improvement.

An optional label-aware check attaches synthetic outcomes monotone in the true score to confirm two invariants: within-queue ordering remains score-monotone, and stability gains do not depend on labels.

Limitations and open questions

The authors are explicit that the study is score-only: no claim is made about precision, false-positive rates, or risk re-ordering, since those require labels or utility models. Several further concessions bear directly on the results:

• Valley reliability: in sparse windows or heavily discretized streams, apparent minima may be unreliable or vanish; guardrails reduce jitter but add latency, so short shocks can be under-reacted to.
• Boundary bias: reflection is fast and mass-conserving but can distort shapes when true densities accumulate near 0 or 1; streaming local-linear correction is heavier.
• Design knobs: window length, decay rate, refresh cadence, and hysteresis strength trade responsiveness against stability without principled tuning rules.
• No uncertainty quantification: deployed thresholds and valley locations carry no confidence statements, limiting formal change-control arguments.
• Per-BA scope: cross-BA capacity allocation is exogenous; joint optimization across activities or tenants is unsolved.
• Evaluation scope: all evidence comes from synthetic streams; no production or real transaction data is reported.

Open questions the paper leaves include bootstrap or SiZer-style confidence bands for streaming valleys, stability theorems bounding intake sensitivity under windowed KDE with decay, conditional densities $[0,1]$4 for covariate-aware cuts, sketch-based acceleration for burst rates, and coupling capacity matching with queueing control as a stochastic control problem.

Conclusion

The paper offers a transparent, label-free mechanism for converting streaming risk scores into capacity-matched queues: an online adaptive KDE with reflection, valley detection with scale and salience filters, quantile cuts snapped to persistent valleys, and guardrails against jitter. Its guarantees are operational—capacity adherence and reduced threshold sensitivity—not statistical performance gains, and its empirical support is confined to synthetic scenarios presented without numeric tables. Within those bounds, it provides a defensible, auditable alternative to top-$[0,1]$5 and manual cutoffs for real-time investigative queue management.