Incremental Sequential/Resection–Intersection

• Incremental Sequential/Resection–Intersection is a paradigm that incrementally builds solutions by fusing data through sequential insertions and intersection operations, ensuring structural invariance.
• It utilizes efficient data structures and hash-based techniques to achieve sublinear expected time for set intersection in dynamic, insertion-only settings.
• Enhanced sequential covariance intersection in multi-sensor fusion maintains unbiased estimates under asynchronous and bursty data arrivals through optimal analytic weighting.

Incremental Sequential/Resection–Intersection describes a class of computational and data fusion methodologies that systematically build up solutions via a sequence of insertion, intersection, or fusion operations. In this paradigm, incoming data, sets, or estimates are incrementally added (sequentially or in bursts), with the core operations being either resection (addition or update) or intersection (fusion, search for overlaps, or witness discovery). Two major instantiations are: (1) incremental set intersection in dynamic data structures on the word-RAM model, and (2) enhanced sequential covariance intersection (ESCI) in multi-sensor data fusion. Each provides rigorous performance guarantees and structural invariance under incremental growth or asynchronous incoming data streams.

1. Formalization of Incremental Operations

The incremental model is defined by a state that evolves via a sequence of operations. In the set intersection case, the state is a family $F$ of sets, where only insertions (of elements) are permitted; deletion is not allowed in the incremental setting (Kopelowitz et al., 2014). Witness or intersection queries are executed at any time on any current pair of sets.

In the sequential covariance intersection context, the state space consists of a collection of estimate–covariance pairs (EPs) $\{x_j, P_j\}$ generated by a network of sensors. Incremental operations correspond to the arrival and fusion of new EPs—either singly or in batches—at a fusion node. The order of arrival and batching pattern may be unpredictable (Hu et al., 2021).

In both settings, the challenge is to enable efficient and correct query answering and fusion, while minimizing sensitivity to the operation sequence, data structure, or fusion structure.

2. Incremental Set Intersection: Data Structure and Algorithm

Incremental set intersection, as analyzed in the word-RAM model (Kopelowitz et al., 2014), introduces a data structure supporting insertions and “witness” queries. Sets are classified into three size classes—small, medium, large—with thresholds set via parameters involving the total insertion count $N$ and the word size $w$:

• Small: $|S|<\sqrt{N/\tau_q}$,
• Medium: $\sqrt{N/\tau_q}\le|S|<\sqrt{N\tau_q}$,
• Large: $|S|\ge\sqrt{N\tau_q}$, where $\tau_q = w/\log^2 w$.

Each set $S$ supports:

• Insert operation: Add element $e$.
• Witness query: For any $\{x_j, P_j\}$0, return any $\{x_j, P_j\}$1 if nonempty; else report empty.

Small-set primitive uses packed, bucketed, pairwise-independent hashing with word-parallel operations to answer witness queries in $\{x_j, P_j\}$2 expected time, where $\{x_j, P_j\}$3, and insert in $\{x_j, P_j\}$4 time. Medium and large sets maintain a “stash” of newest $\{x_j, P_j\}$5 elements managed by the small-set primitive. Large sets also maintain a witness array for constant-time lookup against other large sets.

Query and update complexities are:

Set Size	Insert Complexity	Witness Query Complexity
Small	$\{x_j, P_j\}$6	$\{x_j, P_j\}$7
Medium	$\{x_j, P_j\}$8 (amortized)	$\{x_j, P_j\}$9
Large	$N$0	$N$1

Operations use expected time bounds over randomized hash functions and exploit word-level parallelism for sublinear scaling with $N$2 as $N$3 increases. Limitations include being inherently incremental; supporting deletions requires a different, generally slower scheme.

3. Incremental Intersection-Fusion in Multi-Sensor Estimation

Sequential covariance intersection (CI) fusion provides an incremental, structure-invariant method for fusing estimates with unknown correlations (Hu et al., 2021). Each estimate–covariance pair $N$4 from a sensor is collected at a fusion node, possibly in unpredictable order and in varying batch sizes. The fusion structure is parametrized by an order vector $N$5 and batch vector $N$6.

Batch CI fuses all $N$7 pairs at once after receiving all data. Sequential CI fuses each new pair as it arrives, updating the fused estimate and covariance recursively. ESCI generalizes this to arbitrary batch sizes, supporting “bursty” arrivals and fully distributed or asynchronous settings.

Key formulas for batch CI: $N$8 with nonnegative weights $N$9 summing to one.

Sequential and enhanced sequential CI recurrences involve weight selection at each step. The fusion output $w$0 after all EPs have arrived only depends on the set of input pairs if the weights are chosen analytically to be structure-independent: $w$1 where

$w$2

for an importance function $w$3. Choices for $w$4 include minimizing total variance ($w$5), minimizing covariance volume ($w$6), or maximizing total information ($w$7).

4. Structural Invariance and Performance Guarantees

A fundamental property in both incremental set intersection and ESCI fusion is structural invariance: the correctness and quality of output do not depend on the sequence, order, or batching of incremental operations, provided the analytic weighting or class partitioning scheme is followed. In the set intersection scheme, queries and insertions operate correctly regardless of prior operation sequences, due to the use of hash-based structures and stashes.

In ESCI, if ESCI weights are chosen as prescribed analytic functions of input statistics, the final fused output is unbiased, conservative, and—crucially—independent of both the order and grouping of fusions. This addresses a known deficiency of classical sequential CI fusion, where variable results can arise depending on the fusion structure. Objective choices of $w$8 lead to familiar interpretations (such as minimizing mean square error or maximizing information volume).

5. Practical Considerations and Implementation

Both domains emphasize efficient incremental processing for scalable real-time systems.

• In set intersection (Kopelowitz et al., 2014), implementation on word-RAM leverages $w$9-time bitwise and arithmetic operations, achieves $|S|<\sqrt{N/\tau_q}$0 expected time for each operation, and attains significant speedups when $|S|<\sqrt{N/\tau_q}$1 is large (e.g., on 64-bit architectures).
• The ESCI fusion method (Hu et al., 2021) distributes computational cost evenly over time and supports fully asynchronous sensor networks. Simulation results in synthetic 2D cases, Kalman filter tracking, and nonlinear mobile robot localization confirm that ESCI achieves fusion outputs invariant to batch/arrival structure and typically outperforms classical sequential CI in accuracy and efficiency, while closely approaching batch CI optimality for practical analytic weight choices.

6. Extensions, Limitations, and Trade-offs

The methods described are optimized for incremental or insertion-only regimes. Supporting deletions in set intersection requires fundamentally different data structures with higher complexity (Kopelowitz et al., 2014). Time-space tradeoffs are available for space-constrained dynamic intersection by scaling word-packing parameters or batching strategies.

In CI fusion, the choice of importance function $|S|<\sqrt{N/\tau_q}$2 directly mediates the statistical optimality of the fused estimate, with some tradeoff between performance and computation (e.g., minimizing $|S|<\sqrt{N/\tau_q}$3 versus maximizing $|S|<\sqrt{N/\tau_q}$4). Structure-invariant ESCI is particularly advantageous for resource-constrained or highly asynchronous fusion settings encountered in sensor networks.

7. Applications and Implications

Incremental sequential/resection–intersection paradigms are central to large-scale graph algorithms (e.g., triangle listing using dynamic intersection), scalable set-indexing, and decentralized estimation and control in adversarial or unreliable communication environments.

The sublinear expected-time scaling in set intersection supports applications such as streaming graph analytics where both space and latency are critical (Kopelowitz et al., 2014). ESCI fusion advances robust estimation in distributed, delay-prone, or multi-perspective sensor systems, providing guarantees independent of network timing or data batching (Hu et al., 2021).

These methodologies exemplify how careful design of incremental operations, structural decompositions, and analytical weighting can yield both efficiency and invariance—central principles for modern distributed computing and estimation.