Dynamic GUS: Graph Building with ScaNN

• The paper introduces Dynamic GUS, which transforms Grale’s bucket-based candidate generation into a sparse embedding framework for dynamic approximate nearest-neighbor retrieval.
• It achieves rapid graph construction with sub-second query freshness and up to 4x reduction in detection latency through efficient candidate filtering and IDF weighting.
• The system supports continuous updates with hundreds of thousands of point mutations per second while preserving high-quality, task-specific neighborhood queries.

Dynamic Grale Using ScaNN, usually abbreviated Dynamic GUS, is a dynamic graph-building system that combines the multimodal, learned pairwise similarity framework of Grale with the low-latency dynamic retrieval capabilities of ScaNN. It is designed for environments in which points are inserted, deleted, and modified continuously, yet neighborhood queries must remain fresh within seconds at the $99$th percentile and typically complete in the tens of milliseconds. Its central idea is to convert Grale’s bucket-based candidate-generation structure into a sparse embedding, use ScaNN over that sparse space to retrieve candidates dynamically, and then apply a pre-trained machine-learning model over the original features for final edge scoring. In this way, Dynamic GUS aims to preserve the graph quality associated with Grale while making graph construction practical in rapidly evolving production settings (Almeida et al., 14 Jul 2025).

1. Conceptual lineage and problem setting

Dynamic GUS is best understood as a dynamic serving formulation of Grale. Grale was introduced as a system for graph design in semi-supervised learning, where the graph is not assumed to be given but instead learned from multimodal data. In its offline formulation, Grale has three main steps: train a machine-learning model that scores the similarity of two points, generate candidate pairs using an LSH-style bucketization, and score those candidate pairs with the trained model. The 2025 Dynamic GUS work describes Grale as deployed in more than 50 different industrial settings at Google and able to build graphs for billions of points in hours in offline environments; the earlier Grale paper reported more than 20 industrial settings at Google, including datasets with tens of billions of nodes and hundreds of trillions of potential edges to score (Almeida et al., 14 Jul 2025, Halcrow et al., 2020).

The original Grale formulation is centered on multimodal graph construction. Each point may contain multiple types of features, and the edge score is learned as a task-specific fusion of modality-level similarities rather than inherited from a single pre-existing embedding. In the Grale framework, the learned similarity function is written as

$$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$

with edge weights defined by

$w_{ij} := \log G(x_i, x_j).$

This formulation ties graph construction directly to task-specific homophily rather than to a fixed geometric notion of nearness (Halcrow et al., 2020).

Dynamic environments make the standard Grale workflow prohibitive. Although bucket generation for a single point is fast, maintaining the bucket data structures and recomputing or rescoring all candidate pairs induced by those buckets is very costly when the data evolves continuously. The motivating examples include harmful-app detection with many new apps arriving continuously, social media moderation with thousands of uploads per second, and recommendation systems with thousands of new entities per second. The target operating point is explicit: continuous inserts, deletes, and modifications; low-staleness updates; sub-second neighborhood queries; freshness within seconds for the $99$th percentile of queries; and practical latency in the tens of milliseconds (Almeida et al., 14 Jul 2025).

A central distinction from conventional dynamic ANN systems follows from this lineage. Standard dynamic ANN infrastructure generally assumes that similarity is already captured by one embedding and one fixed metric. Dynamic GUS instead preserves Grale’s premise that similarity is learned from multiple features or modalities, with ANN used only as a dynamic candidate-generation layer. This is why it is not merely a dynamic nearest-neighbor index.

2. System architecture and serving workflow

Dynamic GUS has a three-component architecture. The Embedding Generator computes a sparse embedding from a point’s own features only and must be very fast because it lies on the critical path for both mutations and neighborhood queries. The Neighbors Computation component uses ScaNN to retrieve nearest neighbors in sparse embedding space. The Similarity Computation component then applies a pre-trained machine-learning model to score the query point against the retrieved candidates. This separation between fast candidate retrieval and richer multimodal scoring is the architectural core of the system (Almeida et al., 14 Jul 2025).

The mutation path is direct. For an insert or update RPC, the client sends a point $p$, Dynamic GUS computes the sparse embedding $M(p)$, inserts or updates $(p, M(p))$ in ScaNN, and returns an acknowledgment. For a delete RPC, the point is removed from ScaNN and an acknowledgment is returned. Neighborhood computation is similarly structured: the client sends a point $p$, Dynamic GUS computes $M(p)$, queries ScaNN for points $Q$ that are close to $$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$0 in the sparse embedding space, computes learned similarity scores between $$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$1 and each point in $$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$2, and returns $$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$3 together with the scores (Almeida et al., 14 Jul 2025).

The system is designed for very high dynamic throughput. Its stated capabilities include hundreds of thousands of point insertions, modifications, and deletions per second; hundreds of thousands of neighborhood queries per second; sub-second latency; freshness within seconds at the $$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$4th percentile; and neighborhoods similar to those obtained by rerunning offline Grale from scratch. The paper also states at a high level that mutation “typically takes up to a second,” while the reported single-machine benchmarks are much smaller; both descriptions are part of the stated system characterization (Almeida et al., 14 Jul 2025).

Several design choices explain the latency regime. Embedding generation uses only local information; sparse ANN retrieval with ScaNN avoids bucket-induced candidate explosion; the serving path typically retrieves only top-$$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$5 neighbors rather than all points with negative distance; popular-bucket filtering reduces noisy collisions and embedding dimensionality; and the trained model, filtered bucket list, and IDF lookup table are kept in memory. The system can run in parallel and distributed settings for larger datasets, although the reported experiments are intentionally sequential and centralized (Almeida et al., 14 Jul 2025).

3. Sparse embedding construction and the formal bridge to Grale

The defining technical move in Dynamic GUS is the reinterpretation of Grale’s bucket structure as a sparse embedding. If Grale assigns bucket IDs $$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$6 to a point $$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$7, then the embedding $$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$8 is a sparse vector with nonzero coordinates indexed by those bucket IDs. In the initial basic embedding, each active coordinate has value $$G(x_i, x_j) = f(\kappa_1(x_i, x_j), \kappa_2(x_i, x_j), \dots, \kappa_d(x_i, x_j)),$$9. This construction creates an exact bridge between Grale candidate generation and sparse inner-product search: two points have positive dot product if and only if they share at least one active bucket coordinate (Almeida et al., 14 Jul 2025).

The distance used in ScaNN is the negative inner product: $w_{ij} := \log G(x_i, x_j).$0 When the embedding is understood, the notation is abbreviated to $w_{ij} := \log G(x_i, x_j).$1. ScaNN supports both top-$w_{ij} := \log G(x_i, x_j).$2 retrieval and threshold retrieval. Threshold retrieval is defined as

$w_{ij} := \log G(x_i, x_j).$3

and top-$w_{ij} := \log G(x_i, x_j).$4 retrieval is defined as finding a set $w_{ij} := \log G(x_i, x_j).$5 of size $w_{ij} := \log G(x_i, x_j).$6 such that

$w_{ij} := \log G(x_i, x_j).$7

Under the unit-weight bucket embedding, the distance equals minus the number of non-zero dimensions shared by the two points (Almeida et al., 14 Jul 2025).

This yields the formal equivalence result stated as Lemma 4.1: for any point $w_{ij} := \log G(x_i, x_j).$8, the neighborhood of $w_{ij} := \log G(x_i, x_j).$9 is exactly the same in Grale and Dynamic GUS if all points with negative distance to $99$0 are retrieved in ScaNN. The result is immediate from the fact that $99$1 if and only if the dot product is positive, which occurs if and only if the points share at least one bucket ID, which is exactly the Grale condition for being a candidate scoring pair. Exact equivalence therefore holds in the unbounded threshold-retrieval regime (Almeida et al., 14 Jul 2025).

Two refinements alter the sparse embedding to improve practical candidate quality. The first is filtering overly popular bucket IDs. If a bucket is present in too many points, it becomes weakly discriminative; Dynamic GUS can therefore ignore the corresponding sparse dimensions. This is parameterized by Filter-P, where Filter-P $99$2 means that the top $99$3 most frequent buckets are ignored. The second is inverse document frequency weighting. If $99$4 is the set of points and $99$5 is the number of points containing bucket ID $99$6, the weight of dimension $99$7 is

$99$8

A bounded-memory approximation, IDF-S, computes only the top $99$9 IDF weights and assigns the $p$0-th highest weight to all remaining buckets so that the lookup table remains memory-resident (Almeida et al., 14 Jul 2025).

These mechanisms clarify the role of ScaNN in Dynamic GUS. ScaNN is not the final semantic similarity model; it is the dynamic substrate for retrieving a recall-oriented candidate set whose geometry is aligned with Grale’s bucket logic. Final edge weights still come from the learned pairwise scorer over the original feature representation.

4. Evaluation methodology and empirical behavior

The empirical study uses two Open Graph Benchmark datasets and a common pairwise scorer architecture: a two-layer neural network with 10 hidden units per layer. The central quality metric is the distribution of edge weights assigned by the learned similarity model to the returned edges, plotted as edge weight versus edge percentile after sorting edges by score. Dynamic-system evaluation also measures end-to-end query latency, average CPU time per request, maximum memory usage, and insertion latency (Almeida et al., 14 Jul 2025).

Dataset	Size and features	Selected dynamic results
ogbn-arxiv	169,343 CS arXiv papers; publication year; 128-dimensional feature vector from averaged word embeddings over title and abstract	median query latency around 10–20 ms; average CPU about 14 ms; max memory about 501–688 MiB; median insertion 0.29 ms, 95th percentile 0.54 ms
ogbn-products	2,449,029 Amazon products; co-purchased products; 100-dimensional embedding from bag-of-words product descriptions reduced by PCA	median query latency around 5–25 ms; average CPU about 33 ms; max memory about 13,086–33,871 MiB; median insertion 0.42 ms, 95th percentile 0.78 ms

The first experiment validates the exact bridge to Grale. When Grale is run without a bucket-size limit and Dynamic GUS retrieves all points with negative distance from ScaNN, the edge sets are identical on both datasets, empirically validating Lemma 4.1. The reported total retrieved edges are $p$1 for ogbn-products and $p$2 for ogbn-arxiv (Almeida et al., 14 Jul 2025).

The second experiment studies the effects of ScaNN-NN in $p$3, Filter-P in $p$4, and several IDF-S settings. Dynamic GUS often returns higher-quality edges than practical Grale baselines because Grale controls bucket explosion by random bucket splitting, whereas Dynamic GUS retrieves nearest points in sparse embedding space. On ogbn-products, one reported example states that, with Grale, less than 30% of edges have weight above $p$5 for all bucket sizes, while with Dynamic GUS and Filter-P $p$6, more than 97% of edges have weight above $p$7. Another example, from Fig. 4a, states that at the 20th percentile the edge weight is almost $p$8 without filtering or IDF, $p$9 when IDF is used, and $M(p)$0 when filtering is used (Almeida et al., 14 Jul 2025).

Selectivity is visible in the returned edge counts. For ogbn-products with ScaNN-NN $M(p)$1, Filter-P $M(p)$2 returns $M(p)$3 edges, whereas Filter-P $M(p)$4 returns $M(p)$5. For ogbn-arxiv with the same ScaNN-NN settings, the counts are $M(p)$6 and $M(p)$7, respectively. These examples illustrate how strongly popular-bucket suppression can reduce candidate volume (Almeida et al., 14 Jul 2025).

The top-$M(p)$8 pruning experiments connect the system to downstream graph materialization. For top-$M(p)$9, ogbn-products shows high and comparable edge weights between Grale and Dynamic GUS, with $(p, M(p))$0 edges for Grale Top-K $(p, M(p))$1, Bucket-S $(p, M(p))$2, and $(p, M(p))$3 for Dynamic GUS Top-K $(p, M(p))$4, NN $(p, M(p))$5, Filter-P $(p, M(p))$6. For top-$(p, M(p))$7, Dynamic GUS has significantly higher quality on ogbn-products even though Grale scores $(p, M(p))$8–$(p, M(p))$9 more edges; the reported counts are $p$0 for Grale and $p$1 for Dynamic GUS. On ogbn-arxiv, Dynamic GUS is described as slightly lower at top-$p$2, but still much more attractive computationally (Almeida et al., 14 Jul 2025).

The dynamic experiments were conducted on a single Google Cloud machine, n2d-standard-48, with CPU platform AMD EPYC 7B12; all tasks ran on a single core, one experiment at a time, with sequential queries over 10,000 random sampled points per dataset. Under this controlled setup, smaller ScaNN-NN and larger Filter-P improve latency. The benchmark therefore locates the practical quality-latency frontier primarily in the ScaNN-NN setting, with filtering acting as one of the strongest quality and performance levers (Almeida et al., 14 Jul 2025).

5. Operational role and downstream uses

Dynamic GUS has over 10 deployments at Google and is described as a critical component of dynamic graph systems. It is also identified as the backbone of downstream dynamic graph algorithms including clustering, label propagation, and graph neural networks. In this operational framing, Dynamic GUS should not be understood as a standalone nearest-neighbor service; it supplies dynamic local neighborhoods that other graph analytics and learning systems consume (Almeida et al., 14 Jul 2025).

A prominent deployment is in Google Play and Android Security and Privacy. The use case is dynamic risk modeling for potentially harmful apps. The reported production impact is a $p$3 action rate improvement and a $p$4 reduction in detection latency, summarized as enabling harmful applications to be captured 4 times faster before they can reach users. This case makes the relevance of freshness explicit: when graph updates arrive quickly enough, suspicious new items can be connected to harmful neighborhoods materially earlier (Almeida et al., 14 Jul 2025).

The system’s operational pattern depends on a separation between slowly refreshed global metadata and rapidly updated per-point state. Before serving begins, Dynamic GUS computes embeddings for the initial dataset, loads them into ScaNN, trains the pairwise model, computes popular buckets for filtering, and computes IDF weights. The trained scorer, the set of overly popular bucket IDs, and the IDF table are kept in memory. These components can be retrained or recomputed periodically so that they remain approximately consistent as the dataset evolves (Almeida et al., 14 Jul 2025).

This architecture also clarifies a common misconception. Dynamic GUS does not replace learned multimodal graph construction with generic ANN retrieval. Rather, it preserves the learned pairwise scoring framework of Grale and moves only the candidate-generation substrate into ScaNN. The distinction matters because the value proposition of Grale, as formulated in the original system, is precisely that graph edges are learned from multiple weak signals and task-specific similarity rather than from one fixed metric or one dense vector alone (Halcrow et al., 2020).

6. Limitations, misconceptions, and open questions

Dynamic GUS inherits Grale’s quality only to the extent that the underlying bucket generation is good. The sparse embedding is derived directly from bucket IDs, so weak bucket assignments lead to weak ANN retrieval. The system does not redesign bucket generation; it assumes that part from Grale or another compatible algorithm. This means that the dynamic advance lies in serving and candidate maintenance, not in redefining the underlying notion of bucketized similarity (Almeida et al., 14 Jul 2025).

Exact equivalence to Grale holds only in the unbounded retrieval regime. Lemma 4.1 applies when all points with negative distance are retrieved. In practice, Dynamic GUS typically uses bounded top-$p$5 retrieval, so the operational system is approximate. Its quality then depends on ScaNN retrieval behavior, the selected $p$6, and the choices of filtering and IDF weighting. No formal approximation ratio or recall theorem beyond the equivalence lemma is provided (Almeida et al., 14 Jul 2025).

Global statistics are refreshed periodically rather than maintained fully online. Popular-bucket filters and IDF weights are computed from an initial dataset and periodically recomputed so that they remain approximately consistent. If the data distribution shifts quickly, those statistics may become stale before reload. The paper presents this as a practical freshness-versus-cost trade-off rather than as an exact consistency mechanism (Almeida et al., 14 Jul 2025).

Resource usage is also nontrivial. On ogbn-products, maximum memory in the experiments ranges from roughly $p$7 MiB to $p$8 MiB depending on the configuration. Latency therefore improves substantially relative to offline graph rebuilding, but large-scale deployments still face a meaningful memory budget, especially with larger or richer ScaNN configurations (Almeida et al., 14 Jul 2025).

The theoretical analysis remains limited. The paper gives an exact equivalence lemma but does not provide broader guarantees on approximation quality under top-$p$9 retrieval, end-to-end recall of the highest-weight Grale edges, update complexity bounds, or consistency semantics under concurrent heavy mutation and query loads. The empirical evaluation is also restricted to two OGB datasets and controlled single-machine tests. This suggests several open questions about behavior under more modalities, highly skewed streaming patterns, adversarial bucket distributions, and large-scale distributed deployment, although these questions are not resolved in the reported study (Almeida et al., 14 Jul 2025).