---
title: 'Adaptive-GraphSketch: Real-time Anomaly Detection'
url: https://www.emergentmind.com/topics/adaptive-graphsketch
type: topic
---

# Adaptive-GraphSketch: Real-time Anomaly Detection

ADAPTIVE-GRAPHSKETCH is a lightweight and scalable framework for real-time anomaly detection in streaming edge data. It is designed for dynamic graphs in which timestamped edges \(e=(u,v,t)\) arrive continuously, and it combines temporal multi-tensor sketching with Count-Min Sketch using Conservative Update (CMS-CU), Bayesian inference for probabilistic anomaly scoring, and Exponentially Weighted Moving Average (EWMA) thresholding tuned to burst intensity. The framework is intended to compactly track edge frequency patterns with bounded memory, mitigate hash collision issues, and adapt to evolving traffic patterns in settings such as cybersecurity and power grids [2509.11633].

## 1. Problem setting and detection objective

The framework addresses edge-level anomaly detection in real time. In the stated problem setting, modern systems such as cybersecurity, fraud, social media, and power grids generate massive streams of timestamped edges \(e=(u,v,t)\), and the task is to flag every incoming \((u,v,t)\) whose frequency pattern deviates sharply from its recent history. The motivating constraints are throughput, memory, adaptivity, and probabilistic interpretability: the stream may involve tens of millions of edges per second, the full adjacency matrix or long edge histories cannot be stored, network behavior drifts over time, bursts appear and subside, and the detector should provide principled thresholds and uncertainty-aware scores [2509.11633].

Within this formulation, the framework is not a batch detector over static graphs. Its basic unit of analysis is the incoming edge event, and its anomaly score is computed online from a compressed temporal summary rather than from explicit per-edge histories. This design aligns bounded-memory graph summarization with online detection, rather than with retrospective graph mining.

## 2. Multi-layer tensor sketching and CMS-CU integration

The core summary structure is a three-dimensional Count-Min style sketch
\[
\mathcal{S}\in\mathbb{R}^{d\times w\times W},
\]
where \(d\) is the number of hash functions, \(w\) is the number of buckets per hash, and \(W\) is the number of time-bins in the sliding window [2509.11633].

Time is divided into contiguous intervals of width \(\Delta\). For an edge timestamp \(t\), the bin index is
\[
b_t=\Big\lfloor\tfrac{t}{\Delta}\Big\rfloor,\quad b_t\in\{1,2,\dots,W\}.
\]
Given \(d\) pairwise-independent hash functions \(\{h_1,\dots,h_d\}\) mapping each unordered pair \((u,v)\) into \(\{1,\dots,w\}\), the arrival of edge \((u,v,t)\) identifies the \(d\) candidate cells
\[
\mathcal{S}[\,i,\;h_i(u,v),\;b_t\,],\quad i=1\ldots d.
\]

The update rule uses Conservative Update rather than standard Count-Min increments. Standard CMS increments all \(d\) counters for an item, which produces overestimates under collisions. In CMS-CU, only those rows whose counter value equals the current minimum are incremented:
\[
v_i = \min_{j=1}^d v_j\;\;\Longrightarrow\;\;v_i\leftarrow v_i+1,
\]
or equivalently,
\[
v_i \leftarrow
\begin{cases}
v_i+1 & \text{if }v_i = \min_{j}v_j,\\
v_i & \text{otherwise.}
\end{cases}
\]
In ADAPTIVE-GRAPHSKETCH, CMS-CU is applied separately on each time layer \(b\) of the 3D tensor, which reduces per-bin collision bias [2509.11633].

The frequency estimators are defined at two temporal scales. The estimated count in the current bin is
\[
\hat a_{uv}(t)=\min_{i=1}^d\mathcal{S}[\,i,h_i(u,v),b_t\,],
\]
and the cumulative count up through bin \(b_t\) is
\[
\hat s_{uv}(t)=\sum_{k=1}^{\,b_t}\hat a_{uv}(k).
\]

To emphasize recent behavior and bound memory, the framework applies exponential decay and sliding-window pruning before processing bin \(b_t\). Every cell is decayed as
\[
\mathcal{S}[i,j,b]\leftarrow \gamma\;\mathcal{S}[i,j,b],\quad \gamma\in(0,1),
\]
and any bin \(b\) with \(b_t-b>W\) is zeroed out. The resulting memory guarantee is \(\mathcal{O}(d\cdot w\cdot W)\), while the per-edge update cost is \(O(d)\) [2509.11633].

## 3. Probabilistic anomaly scoring and adaptive thresholding

After computing \(\hat a_{uv}(t)\) and \(\hat s_{uv}(t)\), the framework assigns a probabilistic score to the burstiness of the edge. Under the normal model,
\[
\mu=\tfrac{\hat s}{t},\qquad \sigma^2=\tfrac{\hat s}{t^2},
\]
while under the anomaly model the mean is shifted and the variance is inflated:
\[
\mu_A=\mu+\delta,\qquad \sigma_A^2=4\,\sigma^2.
\]
With prior \(P(\text{Anom})=p_0\) and evidence
\[
P(a)=p_0P(a\mid\text{Anom})+(1-p_0)P(a\mid\text{Normal}),
\]
the posterior anomaly probability is
\[
P(\text{Anom}\mid a)=\frac{p_0\,P(a\mid\text{Anom})}{P(a)}.
\]
This posterior \(P(\text{Anom}\mid a)\) is used as the edge’s raw anomaly score \(X_t\) [2509.11633].

Binary alarms are produced by EWMA smoothing together with an adaptive threshold. The EWMA statistic is
\[
Z_t=\lambda\,X_t+(1-\lambda)\,Z_{t-1},\qquad \lambda\in(0,1),\;Z_1=X_1.
\]
Smaller \(\lambda\) places more weight on history, while larger \(\lambda\) reacts faster to bursts. In parallel, the running mean and variance of the score stream are maintained as
\[
\mu_t=\frac1t\sum_{i=1}^tX_i,\qquad
\sigma_t^2=\frac1t\sum_{i=1}^t(X_i-\mu_t)^2,
\]
and the threshold is set to
\[
\tau_t=\mu_t+k\,\sigma_t.
\]
By Chebyshev’s inequality,
\[
\Pr(X_t>\tau_t)\le 1/k^2,
\]
so the parameter \(k\) directly trades off sensitivity against false-alarm rate. The decision rule is to flag edge \(t\) as anomalous if \(Z_t>\tau_t\) [2509.11633].

This scoring-and-thresholding stack gives the framework a probabilistic interpretation at two levels. The posterior score encodes the relative support for the anomaly model, while the EWMA–Chebyshev stage converts that score stream into adaptive alarms tuned to burst intensity.

## 4. Complexity, memory profile, and implementation characteristics

The stated per-edge time complexity is \(O(d)\) hashes plus \(O(d)\) conservative-update checks, which is \(O(1)\) for fixed \(d\). Memory usage is \(\mathcal{O}(d\cdot w\cdot W)\). In the reported experiments, \(d=10\), \(w\approx 10^2\)–\(10^3\), and \(W\approx10^2\), so the total memory is a few MB. The update cost is one pass over the \(d\) rows per edge, plus a short loop of length \(W\) for decay, which can be amortized or implemented incrementally [2509.11633].

These parameters define the practical operating regime of the method. The sketch is fixed-size, the number of hash functions is small, and the temporal dimension is explicit rather than implicit. The design therefore separates three concerns that are often conflated in streaming systems: collision control through CMS-CU, recency weighting through decay and pruning, and decision calibration through posterior scoring and EWMA thresholding.

A plausible implication is that the framework’s bounded-memory behavior is not an auxiliary implementation detail but part of the detection model itself. Older events are attenuated twice: numerically through \(\gamma\)-decay and structurally through zeroing of bins older than \(W\). This makes the detector explicitly recent-history centric.

## 5. Empirical evaluation

The experimental evaluation uses four real-world intrusion detection datasets and reports ROC-AUC, end-to-end processing time, and memory footprint as the primary metrics. The baselines are AnoEdge-G/L, MIDAS-R, F-FADE, PENminer, SedanSpot, and DenseStream [2509.11633].

| Dataset | Scale | Reported observation |
|---|---:|---|
| DARPA | 4.5 M edges, 25,525 nodes | Included in evaluation |
| ISCX-IDS2012 | 1.1 M edges | Included in evaluation |
| CIC-IDS2018 | 7.9 M edges | AUC gain of up to 6.5% over the best prior method |
| CIC-DDoS2019 | 20.3 M edges | AUC gain of up to 15.6% |

Across these datasets, the reported quantitative findings are that ADAPTIVE-GRAPHSKETCH processes 20 million edges in under 3.4 seconds using only \(d=10\) hash functions, while using a few megabytes of sketch storage. Trade-off plots of AUC versus runtime place the method on the Pareto front, with higher accuracy and lower latency than all baselines. The scalability study reports near-linear runtime growth in the number of edges and in the number of hashes, with slope \(\approx 1\) in log–log plots [2509.11633].

The empirical profile therefore combines three claims that are often separated in the streaming anomaly-detection literature: real-time throughput, bounded memory, and improved ROC-AUC on large intrusion-detection streams. This suggests that the method is intended as a deployment-oriented streaming detector rather than only as a sketching primitive.

## 6. Relation to other graph-sketching paradigms and scope of the term “adaptive”

A recurring source of ambiguity is that closely related literatures use the language of sketching and adaptivity for different objectives. In graph-stream query estimation, gSketch partitions a global sketch into localized sketches to estimate edge-frequency and aggregate subgraph queries, emphasizing partitioning based on sampled graph structure and optional workload information [1111.7167]. In labeled and time-sensitive graph streams, LSketch preserves vertex and edge labels, introduces a sliding window, and supports structure-based and time-sensitive queries in sub-linear storage space [2304.02897]. In labeled-graph stream summarization under skewed label distributions, SBG-Sketch maintains \(L\) matrices together with a rank-vector eviction mechanism so that labels can borrow and evict cells across matrices, thereby self-balancing memory across time-varying label frequencies [1709.06723].

A different branch of work uses sketching for dynamic graph algorithms rather than anomaly detection. HybridSCALE sketches only the dense core of a graph and stores the sparse periphery losslessly for dynamic connectivity, with hybrid algorithms for fully-dynamic and semi-streaming connectivity and space
\[
O(\min\{V+E,\;V\log V\log(2+E/V)\})
\]
with high probability [2605.15173]. Another distinct direction studies adjacency sketches in adversarial environments, where adaptivity refers to an adversary that may request labels of up to \(k\le n-2\) vertices before attempting a forgery on two previously unqueried vertices; in that model, label size is tightly related to maximum degree [2309.03728].

Within this broader landscape, ADAPTIVE-GRAPHSKETCH occupies a narrower and more specific position. Its adaptive behavior is realized through temporal decay, posterior anomaly scoring, and EWMA thresholding tuned to burst intensity, rather than through adaptive adversaries, hybrid core-periphery storage, or label-aware reachability and query processing [2509.11633]. This suggests that the framework should be read primarily as a real-time streaming detection pipeline built on graph-sketching components, not as a generic theory of adaptive sketches.

Source: https://www.emergentmind.com/topics/adaptive-graphsketch