HIT-Leiden: Incremental Tree Community Detection

• HIT-Leiden is an incremental, hierarchical, and parallel community detection algorithm designed to maintain quality community partitions under frequent graph updates.
• It uses a tree-structured hierarchy and dynamic connectivity techniques to localize and bound update computations, enhancing efficiency over traditional methods.
• Empirical evaluations demonstrate significant speedups and modularity preservation, making HIT-Leiden effective for dynamic network applications from social graphs to biological datasets.

Hierarchical Incremental Tree Leiden (HIT-Leiden) is an incremental, hierarchical, and parallel community detection algorithm for large, dynamic networks that maintains high-quality community partitions under frequent updates. HIT-Leiden is distinguished by a hierarchical, tree-structured approach to managing community structure and update propagation, as well as provable boundedness and efficient locality—qualities that address the inefficiency and unboundedness of previous Leiden-based incremental methods. It has demonstrated significant speedups and modularity preservation in large-scale experimental evaluations across a variety of dynamic graph scenarios (Lin et al., 13 Jan 2026, Bokov et al., 20 Feb 2025).

1. Theoretical Foundation and Problem Setting

In HIT-Leiden, the input is a dynamic, undirected, weighted graph $G=(V, E)$, where each node $v \in V$ has weighted degree $d(v)$, and the global edge-weight sum is $m = \sum_{(u,v)\in E}w(u,v)$. The community detection objective is modularity maximization:

$$Q(G, f, \gamma) = \sum_{C} \left[ \frac{w(C, C)}{2m} - \gamma \frac{d(C)^2}{4m^2} \right]$$

where $f:V\to\{1,\ldots,k\}$ is the community assignment, $w(C,C)$ the total internal edge weight, $d(C)$ the total degree of $C$, and $\gamma$ is the resolution parameter.

The algorithm processes batch updates $\Delta G$ of edge insertions and deletions. The principal challenge addressed by HIT-Leiden is efficient community maintenance under these updates, without recomputing community structure from scratch. This is formalized in Problem 1: given $(G, f)$ and update $\Delta G$, efficiently compute updated communities for $G' = G \oplus \Delta G$ while preserving high modularity.

A key theoretical concept is boundedness: an incremental algorithm is bounded if its update time is polynomially related to the size of the affected region (AFF) and the community structure size $|f|$. Existing incremental methods (DF-Leiden, ND-Leiden, DS-Leiden) are unbounded, requiring work proportional to the whole graph on each update (Lin et al., 13 Jan 2026).

2. Hierarchical Data Structures and Connectivity Maintenance

HIT-Leiden implements a hierarchical community structure using $P$ levels (typically $P \leq 10$), each representing progressively coarser meta-communities. At each level $p$, the nodes (supernodes) correspond to communities formed at level $p-1$, and edges are induced by aggregating weights between underlying vertices. Supernodes are linked to parents via pointers, forming a tree.

Subcommunity structure is maintained by a dynamic connectivity index $\Psi$ (e.g., DND-Tree) over a subgraph $G_\Psi$ comprising intra-sub-community edges. Each connected component is a subcommunity; edge updates or vertex moves that split a component are tracked, and the smaller piece receives a new subcommunity ID. This enables efficient detection of subcommunity splits/merges and localizes updates.

A representative hierarchy:

Level	Entity	Nodes Represent
$L$	Community Nodes	Meta-communities
$L-1$	Refined Nodes	Subcommunities
$0$	Ground Nodes	Singleton vertices

Edges exist only within levels and inherit weights from underlying substructure (Bokov et al., 20 Feb 2025).

3. Incremental Update Algorithms

HIT-Leiden processes batch updates via efficient local routines at each hierarchical level. The update pipeline consists of:

1. Inc-movement: For a batch $\Delta G$, identifies affected vertices, maintains a working set $A$ of potentially moved vertices, and greedily applies modularity-improving moves based on the gain

$$\Delta Q(v \to C', \gamma) = \frac{w(v, C') - w(v, C)}{2m} + \gamma \frac{d(v)[d(C) - d(v) - d(C')]}{4m^2}$$

All such moves are performed until no positive $\Delta Q$ remains. The process marks both the “community-changed” region ($B$) and sub-community splits ($K$).

1. Inc-refinement: For each vertex in $K$, if split from its subcommunity, reassigns it optimally based on local modularity, ensuring (nearly) $\gamma$-connected, locally optimal subcommunity partitions.
2. Inc-aggregation: Lifts the batch of edge and subcommunity changes from level $p$ to $p+1$ by aggregating the updates on supernodes, maintaining consistent hierarchy representations.
3. Deferred hierarchy update: Changes detected at higher levels are propagated downward: affected children inherit updated community labels, preserving hierarchical consistency.

A global driver sequentially processes each hierarchical level, applying inc-movement, inc-refinement, inc-aggregation, and final deferred updates (Lin et al., 13 Jan 2026, Bokov et al., 20 Feb 2025).

4. Modularity Optimization and Parallelization

In each hierarchical level, HIT-Leiden applies a Leiden-style process comprised of Move and Refine stages:

• MoveStage: Computes, for each affected node and its neighbors, the best allowed move based on $\Delta Q$. Candidate moves are collected, sorted by reward, and greedily filtered to a non-conflicting set for simultaneous application.
• RefineStage: Restricts optimization to moves within the same parent community, refining substructure.

Parallelization leverages the largely local nature of these operations: affected node sets and their 2-hop neighborhoods are partitioned across threads. Most steps, including computation of modularity rewards and candidate moves, are parallel. Only the decoupling (conflict filtering) step is strictly sequential. Memory accesses remain localized, and synchronization cost is minimal (Bokov et al., 20 Feb 2025).

5. Time and Space Complexity

Update cost per batch is a function of the number of unique affected nodes $\delta$, maximum degree $d$, number of hierarchy levels $L$, inner iterations $N$, and Move/Refine iterations $M$. Formally,

$$O\left(N n (d + \log n)\right), \quad n \leq d^{(2L+1)M} \delta$$

where $n$ is the number of supernodes touched. Since $\delta \ll |V|$ in practical dynamic workloads, and $d,L,M$ are constants, time per update is effectively $O(\delta)$. This establishes HIT-Leiden as relatively bounded. Space overhead is $O((L+1)(|V|+|E|))$, with $L \leq 4$ in practice (Bokov et al., 20 Feb 2025, Lin et al., 13 Jan 2026).

6. Empirical Performance and Applications

HIT-Leiden demonstrates scalability and efficiency across multiple domains:

• On datasets with up to 201M nodes and 4B edges, HIT-Leiden achieves up to $10^5 \times$ speedup over DF-Leiden and $10^3$–$10^4 \times$ over ND/DS-Leiden for batch sizes $b=1000$.
• Modularity matches static Leiden within $0.01$ and achieves $>99\%$ $\gamma$-density.
• As batch size $b$ decreases, runtime grows sublinearly, validating dynamic locality, whereas baseline methods remain linear in $|V|+|E|$.
• In long-term experiments over 999 update batches, HIT-Leiden remains both fast and quality-stable.
• In question-answering over graphs (Graph-RAG on HotpotQA), HIT-Leiden-RAG is 56$\times$ faster than static Leiden-RAG, with summary token cost dropping below $1\%$ and no deterioration in QA accuracy (Lin et al., 13 Jan 2026).

The parallel implementation (LD-Leiden) achieves 7–49$\times$ single-thread speedup over prominent baselines and scales to 64 threads with maintained or improved modularity (Bokov et al., 20 Feb 2025).

7. Relation to Previous Methods and Significance

HIT-Leiden’s design addresses the central deficiency of prior incremental Leiden algorithms—unboundedness—by tightly confining computation to the actual affected subregions and propagating changes only when necessary. This is achieved through integration of hierarchical representation, dynamic connectivity tracking, and efficient modularity optimization. The locality inherent in HIT-Leiden also enables efficient parallelization on shared-memory systems.

As the first relatively bounded incremental Leiden algorithm with provable update complexity, HIT-Leiden offers a foundational methodology for community detection in steaming or continuously evolving massive networks encountered in knowledge graphs, anomaly detection, biological datasets, and LLM-powered retrieval-augmented generation systems (Lin et al., 13 Jan 2026, Bokov et al., 20 Feb 2025).