Rateless Bloom Filter (RBF) Overview

• The paper introduces RBF as an adaptive protocol that streams one-hash slices to achieve near-optimal metadata reduction in set reconciliation.
• RBF is characterized by its incremental, slice-based approach that dynamically reduces false positives without prior knowledge of divergence.
• Experimental results show up to 92% metadata reduction and improved communication efficiency compared to static Bloom filters in diverse similarity regimes.

Searching arXiv for the specified papers and closely related Bloom-filter work. arxiv_search query: (Gomes et al., 31 Oct 2025) Rateless Bloom Filter (RBF) denotes, in the set-reconciliation formulation introduced in 2025, a Bloom filter transmitted incrementally as a stream of one-hash slices, so that the receiver can stop exactly when another increment would no longer reduce total communication cost. The term is not used in that sense in the Bloom-filter optimization survey of 2018, which instead treats growable Bloom-filter families such as Dynamic Bloom Filter, Scalable Bloom Filter, Dynamic Bloom Filter Array, and Par-BF, while separately covering Retouched Bloom Filter, a different design that is sometimes abbreviated as RBF in other literature (Gomes et al., 31 Oct 2025, Luo et al., 2018).

1. Terminology and position within Bloom-filter literature

The 2018 survey does not introduce a variant named Rateless Bloom Filter. Its closest matches to a “rateless” or growable Bloom filter are the dynamic-set constructions in Section 6.2: Dynamic Bloom Filter (DBF), Scalable Bloom Filter (SBF), Dynamic Bloom Filter Array (DBA), and Par-BF. These are presented as responses to the “elasticity” challenge, namely dynamic capacity extension without fixing a priori upper bounds. By contrast, the survey’s Retouched Bloom Filter is a fixed-filter false-positive reduction technique based on selectively clearing bits and accepting controlled false negatives; it is orthogonal to rateless growth (Luo et al., 2018).

Within the classical Bloom-filter framework, the survey gives the standard approximations $p_{FP} \approx (1-e^{-kn/m})^k$, $k^* \approx (m/n)\ln 2$, and $m/n \approx -\ln p_{FP}/(\ln 2)^2$. It also notes that the approximation is a lower bound and provides a more accurate exact expression using Stirling numbers. These relations supply the parameter vocabulary used by both growable Bloom filters in the survey and the later rateless reconciliation construction (Luo et al., 2018).

The 2025 RBF paper narrows the meaning of “rateless” to an adaptive communication protocol rather than merely a dynamically extensible membership structure. In that setting, ratelessness means that the sender can, in principle, stream an unbounded number of Bloom-filter slices, while the receiver determines the stopping point from observed marginal utility. This shifts the design objective from storage elasticity to communication optimality under unknown difference cardinality $d$ (Gomes et al., 31 Oct 2025).

2. Problem setting: set reconciliation with unknown divergence

RBF is defined for set reconciliation between two replicas holding sets $S_A$ and $S_B$. The objective is to identify the symmetric difference

$$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$

so that both replicas can converge to $S_A \cup S_B$ while minimizing communication. The formulation assumes a symmetric difference cardinality $d = |\Delta|$ with $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$ (Gomes et al., 31 Oct 2025).

The 2025 paper emphasizes variable-sized elements as the motivating regime. Practical workloads such as files, database rows, and transactions are not fixed-size, so the usual reduction is to reconcile fixed-size digests and fetch actual elements only after the differing items are identified. Reconciliation then becomes a metadata minimization problem: the protocol should identify the differing elements at minimum metadata cost, after which the payload transmission is unavoidable (Gomes et al., 31 Oct 2025).

Existing approaches are described as effective only in restricted regions of the parameter space. Fixed-size digest reconciliation schemes such as PinSketch achieve asymptotic $k^* \approx (m/n)\ln 2$0 metadata but still incur at least $k^* \approx (m/n)\ln 2$1 bits of metadata and may require $k^* \approx (m/n)\ln 2$2 decoding, which becomes impractical beyond small $k^* \approx (m/n)\ln 2$3. Static Bloom filters can be effective when their parameters are tuned to the realized $k^* \approx (m/n)\ln 2$4, but the optimal choice depends on $k^* \approx (m/n)\ln 2$5, which is unknown and may vary widely. IBLT-only approaches, including rateless IBLT, are parameter-free with respect to $k^* \approx (m/n)\ln 2$6 and exact, but for large $k^* \approx (m/n)\ln 2$7 their communication remains $k^* \approx (m/n)\ln 2$8 with a relatively large constant, such as $k^* \approx (m/n)\ln 2$9–$m/n \approx -\ln p_{FP}/(\ln 2)^2$0 symbols per difference, each symbol carrying $m/n \approx -\ln p_{FP}/(\ln 2)^2$1, $m/n \approx -\ln p_{FP}/(\ln 2)^2$2, and count fields. RBF is proposed as the missing prefilter that sends “just enough” Bloom filter without prior knowledge of $m/n \approx -\ln p_{FP}/(\ln 2)^2$3 (Gomes et al., 31 Oct 2025).

A plausible implication is that RBF is not primarily a new approximate-membership data structure for static databases; rather, it is a rate-adaptive communication primitive specialized for the case in which false positives can be cleaned up by a second-stage exact reconciler.

3. Construction and protocol mechanics

The core construction is a partitioned Bloom filter decomposed into one-hash slices. If each slice uses $m/n \approx -\ln p_{FP}/(\ln 2)^2$4 and size $m/n \approx -\ln p_{FP}/(\ln 2)^2$5, then $m/n \approx -\ln p_{FP}/(\ln 2)^2$6 slices emulate a partitioned Bloom filter with $m/n \approx -\ln p_{FP}/(\ln 2)^2$7 and total size $m/n \approx -\ln p_{FP}/(\ln 2)^2$8. The paper chooses

$m/n \approx -\ln p_{FP}/(\ln 2)^2$9

for a set of size $d$0, which yields the halving law

$d$1

because $d$2. Each additional slice therefore halves the false-positive rate (Gomes et al., 31 Oct 2025).

Operationally, the sender streams slices continuously. Slice $d$3 is a bit array of $d$4 bits, built by hashing each element $d$5 with a shared hash function salted as $d$6 and setting one bit. The receiver applies the slice to its current candidate pool $d$7: elements that hit a zero bit are moved to the true-negative set $d$8, while elements that hit a one remain in $d$9. The receiver then decides whether the most recent slice justified its transmission cost (Gomes et al., 31 Oct 2025).

The full protocol is a two-stage hybrid. In Phase 1, replica $S_A$0 sends RBF slices to $S_A$1, which partitions $S_A$2 into $S_A$3 and $S_A$4. In Phase 2, replica $S_A$5 sends RBF slices back to $S_A$6, but now builds the filter over $S_A$7; replica $S_A$8 analogously obtains $S_A$9 and $S_B$0. In Phase 3, the remaining suspected common subsets are reconciled with a rateless IBLT operating on fixed-size digests. The RBF stage removes cheap true negatives, while the rateless IBLT resolves the surviving false positives exactly and identifies which side is missing which elements (Gomes et al., 31 Oct 2025).

This protocol architecture differs from the dynamic Bloom-filter families in the survey. DBF, SBF, DBA, and Par-BF grow by allocating additional subfilters as a set expands over time, with insertions directed to an active layer and queries checking multiple layers. Their goal is elasticity of a membership structure. RBF instead uses incremental slices as a transmission schedule over a fixed reconciliation instance, even though both designs exploit layered Bloom-filter composition (Luo et al., 2018).

4. Analysis, stopping rule, and near-optimality

The static Bloom-filter baseline in the 2025 analysis uses the standard false-positive rate $S_B$1, optimal hash count $S_B$2, and size relation $S_B$3. If one direction transmits a static Bloom filter of size $S_B$4, the metadata cost is $S_B$5, the expected unresolved non-members are $S_B$6, and the second-stage cost is $S_B$7. Under symmetric roles, the total cost is

$S_B$8

Minimizing over $S_B$9 yields

$$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$0

which depends explicitly on the unknown $$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$1 (Gomes et al., 31 Oct 2025).

RBF eliminates this parameter-selection problem by operating in discrete one-slice increments. After $$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$2 slices, the false-positive rate is $$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$3, total Bloom bits are $$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$4, and the total symmetric cost becomes

$$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$5

The expected newly revealed true negatives at slice $$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$6 are

$$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$7

The stopping rule is to stop when

$$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$8

or equivalently continue while

$$\Delta = (S_A \setminus S_B) \cup (S_B \setminus S_A)$$9

This yields

$S_A \cup S_B$0

slices, so that, up to rounding,

$S_A \cup S_B$1

The paper states that this matches the optimizer’s $S_A \cup S_B$2 up to rounding and keeps communication within at most one slice $S_A \cup S_B$3 of the optimum, without requiring prior knowledge of $S_A \cup S_B$4 (Gomes et al., 31 Oct 2025).

The survey’s layered false-positive analysis clarifies why this discrete rate adaptation is tractable. For a layered Bloom filter that reports positive if any layer reports positive, the total false-positive rate is

$S_A \cup S_B$5

For homogeneous layers, this reduces to

$S_A \cup S_B$6

which grows with the number of layers. That behavior is the baseline weakness of DBF and DBA and is precisely what motivates SBF’s parameter scheduling and, in the reconciliation setting, RBF’s slice-by-slice stopping policy (Luo et al., 2018).

5. Relation to dynamic, scalable, and retouched Bloom filters

The survey’s dynamic-set family supplies the closest antecedents to RBF. DBF uses multiple homogeneous Bloom filters activated over time; one active filter is filled to a precomputed upper bound, then a fresh filter is activated, and queries check layers sequentially. DBA organizes homogeneous Bloom filters into groups, supports scalable parallel queries, and handles deletion in batch using stale-element thresholds per filter. Par-BF employs parallel subfilter lists whose parameters are chosen to meet a global false-positive target while supporting deletion, scalability, and parallel query. SBF uses heterogeneous subfilters whose designed false-positive rates are increased geometrically, with lengths and utilizations adapted “to maintain a given false positive rate” as layers are added (Luo et al., 2018).

Among these, SBF is the strongest conceptual analogue to “rateless” growth as understood in the survey. It allows unbounded growth and controls aggregate false positives through per-layer planning rather than through fixed homogeneous replication. The survey explicitly contrasts this with DBF, whose aggregate false-positive rate rises as $S_A \cup S_B$7 unless compensated by larger $S_A \cup S_B$8 or smaller per-layer $S_A \cup S_B$9 (Luo et al., 2018).

The 2025 RBF differs in three ways. First, it is a communication protocol rather than an in-memory capacity-extension strategy. Second, its unit of adaptation is a one-hash slice chosen to make the false-positive rate decay by a factor of two per increment. Third, the stopping criterion is receiver-driven and cost-aware, using the estimated downstream exact-reconciliation cost $d = |\Delta|$0 rather than a preset false-positive target (Gomes et al., 31 Oct 2025).

The most common terminological confusion concerns Retouched Bloom Filter. Retouched BF, covered in Section 4.3 of the survey, reduces false positives by selectively clearing 1-bits and accepting controlled false negatives through random or selective clearing strategies. It does not address capacity growth and is not a rateless scheme. Stable Bloom Filter is likewise not a growable design; it maintains a stable fraction of 1s under streams by decay or eviction and accepts false negatives. Counting Bloom Filter enables deletions via counters, but does not by itself solve growable or rateless operation without adding counters and additional space. These distinctions matter because “RBF” may refer either to Retouched Bloom Filter in some older usage or to Rateless Bloom Filter in the 2025 reconciliation protocol (Luo et al., 2018).

6. Complexity, implementation, and empirical behavior

The 2025 paper gives the following complexity profile. At the sender, each RBF slice requires $d = |\Delta|$1 hashing-and-bit-setting with $d = |\Delta|$2, for an overall RBF cost of $d = |\Delta|$3. At the receiver, each slice requires one membership test per element in the current $d = |\Delta|$4, which shrinks geometrically, giving amortized $d = |\Delta|$5. The second-stage rateless IBLT has encoding cost $d = |\Delta|$6 symbols, decoding cost $d = |\Delta|$7, and expected decoding thresholds of approximately $d = |\Delta|$8 cells for small $d = |\Delta|$9 and approximately $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$0 as $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$1 grows (Gomes et al., 31 Oct 2025).

The same paper specifies several implementation choices. For RBF membership bits, the shared hash function is salted by slice index $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$2 to obtain per-slice independence. The paper’s experiments used 64-bit digests and 64-bit $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$3/$|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$4 fields; for adversarial settings or very large universes it recommends 128-bit or 256-bit digests. Slices are indexed and independent, so they can be applied out of order, and duplicates are idempotent. In adversarial settings, strong keyed hashing is recommended to prevent chosen-input false-positive inflation, while IBLT purity checks rely on $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$5 collision probabilities remaining negligible (Gomes et al., 31 Oct 2025).

The survey adds engineering guidance that also applies to layered or rateless deployments. Less Hashing uses two seeds to generate $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$6 hashes and reduces CPU cost. One Hash BF reduces hashing cost further at the expense of higher false positives. Bloom-1 and OMASS show how bit-array partitioning and blocking can align accesses to fixed-size words or blocks, enabling single-access lookups and parallelism. For large-scale storage systems, BloomStore and forest-structured Bloom filters layer RAM and flash, suggesting that multi-filter growth policies can be coupled to page and cache-line organization (Luo et al., 2018).

Empirically, the 2025 evaluation used $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$7 elements, varied $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$8 to sweep Jaccard similarity from $|S_A \setminus S_B| = |S_B \setminus S_A| = d/2$9 to $k^* \approx (m/n)\ln 2$00, and drew variable-sized elements uniformly in $k^* \approx (m/n)\ln 2$01 bytes. Against static Bloom filters, RBF reportedly tracks the theoretical best static Bloom filter across all $k^* \approx (m/n)\ln 2$02 without parameter tuning, while misconfigured static Bloom filters can overshoot metadata by $k^* \approx (m/n)\ln 2$03–$k^* \approx (m/n)\ln 2$04 depending on the regime. In the hybrid RBF–RIBLT protocol, for Jaccard indices below about $k^* \approx (m/n)\ln 2$05, total communication is reduced by up to over $k^* \approx (m/n)\ln 2$06 relative to the state-of-the-art minimum-metadata baseline, and compared with computationally tractable baselines the total-data reduction exceeds $k^* \approx (m/n)\ln 2$07 at large $k^* \approx (m/n)\ln 2$08; metadata reductions of up to $k^* \approx (m/n)\ln 2$09 are reported as $k^* \approx (m/n)\ln 2$10 grows. The same evaluation also identifies a narrow high-similarity regime in which the first RBF slice imposes a visible fixed cost of approximately $k^* \approx (m/n)\ln 2$11 KB at $k^* \approx (m/n)\ln 2$12, making pure RIBLT or PBS more efficient when similarity is at least $k^* \approx (m/n)\ln 2$13 (Gomes et al., 31 Oct 2025).

These results delimit the practical scope of RBF. It is designed for moderate-to-large divergence, especially after outages or partitions, when $k^* \approx (m/n)\ln 2$14 is unknown and static Bloom-filter tuning is unreliable. When replicas are almost identical, the fixed per-slice cost can dominate, so pure RIBLT, PBS, or a static Bloom filter with very high $k^* \approx (m/n)\ln 2$15 may be preferable. The paper therefore positions RBF not as a universal replacement for all reconciliation schemes, but as a near-optimal adaptive Bloom-filter front end whose value increases precisely when the difference cardinality is uncertain and potentially large (Gomes et al., 31 Oct 2025).