Correlated Linear Bloom Filters

• Correlated linear Bloom filters are Bloom-filter-like data structures that incorporate deliberate bit and hash correlations within a regular linear (or blocked) memory layout.
• They use occupancy frameworks that model joint bit distributions to achieve precise false-positive rate analysis beyond independent random hash assumptions.
• Applications include range queries and multi-stage filtering, where local order preservation and structured layouts enhance memory locality and overall filtering efficiency.

Correlated linear Bloom filters are not a standardized named data structure in the cited literature. As an Editor’s term, the expression is most useful for a family of Bloom-filter-like constructions in which the usual picture of independently dispersed hash placements is replaced by structured dependence across bit positions, neighboring keys, filter stages, or memory blocks, often together with a linear, blocked, local, or order-preserving layout. The closest foundations are the exact occupancy analysis of dependent Bloom-filter bits, the implementation literature on blocked and one-hash Bloom filters, dyadic range filters with piecewise-monotone hashing, multi-set and weighted Bloom-like structures, and chained or learned filters whose later decisions are conditional on earlier ones (Burns, 2019, Riegger et al., 2020, Concas et al., 2019, Bercea et al., 2022, Li et al., 2023, Luo et al., 2018, Fidalgo et al., 24 Jul 2025).

1. Terminological status and conceptual scope

The available literature is explicit on one negative point: no canonical variant is introduced under the exact name “Correlated Linear Bloom Filter.” The survey “Optimizing Bloom Filter: Challenges, Solutions, and Comparisons” states that it does not explicitly mention a “Correlated Linear Bloom Filter” (CLBF) and likewise does not define a canonical “Linear Bloom Filter” as a named family (Luo et al., 2018). The Daisy Bloom filter paper is equally explicit that Daisy is distribution-aware, but not correlated and not linear in the linear-algebraic sense (Bercea et al., 2022). The learned LSM-tree work states that its learned filters are not linear Bloom filters in the usual sense, even though they exploit correlations between keys, levels, and membership (Fidalgo et al., 24 Jul 2025). By contrast, bloomRF is described as a close relative of a “correlated” or “order-aware” Bloom-filter variant, but not merely a classic Bloom filter with correlated linear hashes (Riegger et al., 2020).

This terminological instability matters because the phrase conflates at least two orthogonal design axes. The first is correlation, which in the cited works can mean dependence among occupancy indicators, dependence among hash-derived positions, dependence among neighboring values, or dependence among stages of a composed filter. The second is linearity, which can mean a regular linear memory organization such as words, blocks, partitions, or matrices; a locally order-preserving mapping; or, less appropriately in this literature, a linear-algebraic sketch. This suggests that “correlated linear Bloom filter” is best understood as an overview label for Bloom-filter-like structures that deliberately give up fully independent, uniformly scattered probing in favor of structured dependence plus regular layout.

A recurrent misconception is that Bloom filters are “independent” objects in a stronger sense than the classical analysis supports. The occupancy analysis of Bloom filters makes the opposite point: even in the standard construction, the bit-occupancy indicators are typically exchangeable but dependent, and exact false-positive analysis already depends on that distinction (Burns, 2019). A second misconception is that any use of a model or prior distribution creates a correlated filter. The Daisy construction shows that a filter can be sharply distribution-aware while still assuming independent draws with replacement from a product distribution (Bercea et al., 2022). A third misconception is that correlation is necessarily harmful. bloomRF is expressly designed around structured dependence and local order preservation, and uses that dependence to support range queries (Riegger et al., 2020).

2. Occupancy dependence as the analytic core

The cleanest mathematical foundation for correlated Bloom-filter analysis is the occupancy framework of “On Occupancy Moments and Bloom Filter Efficiency” (Burns, 2019). In that model, Bloom filters are treated as balls-into-bins processes with $m$ distinguishable urns, occupancy indicators

$$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$

and occupancy number

$X=\chi_1+\cdots+\chi_m.$

The essential standing assumption is not independence of the $\chi_i$, but exchangeability: $$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$ where $p_s$ depends only on $s$. Under that assumption,

$$\mathbb{E}\!\left[\binom{X}{r}\right]=\binom{m}{r}\,\mathbb{P}\!\left[\prod_{l=1}^r \chi_l=1\right].$$

This identity is the main bridge from classical Bloom filters to correlated variants, because it shows that exact moment formulas depend on joint occupancy probabilities rather than on a fictitious independence of occupied bits.

The paper distinguishes two constructions. A standard Bloom filter uses $k$ independent hash functions and allows within-item collisions, so inserting $n$ items corresponds to throwing $$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$0 independent balls into $$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$1 urns. A classic Bloom filter instead maps each item to exactly $$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$2 distinct bit positions, so each insertion is a batch of $$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$3 balls forced into distinct urns. In both cases the resulting occupancy indicators are dependent, but the source of dependence differs. This distinction is directly relevant to correlated linear variants because a correlated construction may break the assumptions of independent hash outputs, identical joint occupancy of bit positions, or exchangeability itself (Burns, 2019).

For exact false-positive analysis, the same paper gives the standard Bloom-filter formula

$$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$4

and, for the classic no-within-item-collision model,

$$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$5

The conventional approximation

$$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$6

is not exact. For the standard Bloom filter it is a lower bound on the exact false-positive rate; for the classic Bloom filter the analogous “Bloom-style” estimate is an upper bound (Burns, 2019). The paper further shows that the conventional rule

$$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$7

overestimates the optimal number of hash functions for both standard and classic Bloom filters, and that Bloom-filter efficiency is monotonic in the number of hash functions in the specific standard-filter sense formalized there.

For correlated linear Bloom filters, the most transferable statement is therefore not a closed form for one specific design, but a methodological reduction: if a construction preserves enough symmetry, then the central quantities are the $$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$8-point occupancy probabilities

$$\chi_i(\omega)= \begin{cases} 1, & \text{if urn } i \text{ is occupied},\ 0, & \text{otherwise}, \end{cases}$$9

From these one obtains factorial moments, binomial moments, the occupancy distribution by inclusion–exclusion, and false-positive rates whenever the query rule depends only on occupancy count. If exchangeability fails, the paper is equally clear about the limitation: one must replace $X=\chi_1+\cdots+\chi_m.$0 by position-dependent quantities $X=\chi_1+\cdots+\chi_m.$1, and false positives may depend on the occupied-bit configuration rather than merely on $X=\chi_1+\cdots+\chi_m.$2 (Burns, 2019).

3. Linearity as blocked, local, or order-preserving structure

The literature closest to the “linear” part of the phrase does not usually mean linear algebra; it means structured placement in words, blocks, partitions, traces, or matrices. The Bloom-filter survey groups many relevant variants under implementation optimization and bit-vector or hash-function optimization rather than under a named CLBF taxonomy (Luo et al., 2018). The recurring idea is to reduce the cost of $X=\chi_1+\cdots+\chi_m.$3 independent random memory accesses by introducing structured placement and deliberate dependence among locations.

Construction	Structural mechanism	Closest CLBF aspect
Less hashing	$X=\chi_1+\cdots+\chi_m.$4	Dependent hash derivation
One Hash BF	One hash reduced modulo $X=\chi_1+\cdots+\chi_m.$5 uneven partitions	Partitioned linear layout
Bloom-1 / Bloom-$X=\chi_1+\cdots+\chi_m.$6	One selected word, then in-word bit positions	Block-local probing
Shifting BF	Auxiliary bits at offset positions	Deterministic position dependence
bloomRF	Dyadic Trace-Trees and piecewise-monotone hashes	Local order preservation

These variants make the dependence explicit. Less hashing, same performance uses two base hash functions to derive $X=\chi_1+\cdots+\chi_m.$7 locations by arithmetic progression, preserving asymptotic false-positive performance while removing the need for $X=\chi_1+\cdots+\chi_m.$8 independent base hashes (Luo et al., 2018). One Hash BF uses one machine-word hash value, then reduces it modulo the sizes of $X=\chi_1+\cdots+\chi_m.$9 uneven partitions; the survey notes that such partitioning “slightly damages randomness” and can increase collisions and false positives, although the gap decreases as $\chi_i$0 grows (Luo et al., 2018). Bloom-1 and Bloom-$\chi_i$1 choose one or a small number of words and then place multiple bits inside those words, reducing memory accesses to $\chi_i$2 or a small constant. OMASS, UFBF, and MPCBF likewise use blocks or words so that all or most work stays within a selected local region, often exposing SIMD or one-access behavior (Luo et al., 2018). Shifting BF makes auxiliary bit positions deterministic offsets of primary positions, which is one of the clearest examples of intentional position dependence within a single array (Luo et al., 2018).

bloomRF takes the same design philosophy further, but for order-aware point and range queries rather than flat set membership (Riegger et al., 2020). Its domain is the dyadic interval hierarchy

$\chi_i$3

and each key belongs to a chain of covering intervals across dyadic levels. Instead of inserting those intervals explicitly into a standard Bloom filter, bloomRF groups consecutive dyadic levels into Trace-Trees (TTs) and stacks them into Dyadic Trace-Trees (DTTs). A trace of size $\chi_i$4 has height $\chi_i$5, and one set bit in the trace implicitly encodes the entire path to the root of that TT (Riegger et al., 2020). The second key ingredient is the use of piecewise-monotone hash functions, which are not globally monotone but are locally order-preserving within each trace. This makes neighboring values or intervals map to neighboring bit positions inside a trace, which in turn allows a range query to be represented as a contiguous bitmask and checked by bitwise masking. The paper is explicit that this structure introduces correlation in two senses: horizontal/local correlation among neighboring values in a trace, and vertical correlation among the representatives of a key across DTT layers (Riegger et al., 2020).

The significance of this strand of work is that it reverses a standard Bloom-filter instinct. Classical Bloom-filter engineering usually tries to approximate independent uniform randomness; blocked, shifted, partitioned, and dyadic designs instead introduce dependence to improve locality, exploit order, or make query masks meaningful. This suggests that a correlated linear Bloom filter, in the strongest structural sense, is less a single known object than a design program in which memory layout and dependence are co-designed rather than treated as secondary implementation details.

4. Related architectures beyond flat correlated hashes

Several Bloom-filter-like structures are relevant to correlated linear Bloom filters even though they are not themselves correlation-aware in the strict sense. The Daisy Bloom filter is a weighted Bloom filter for sets generated by $\chi_i$6 independent draws with replacement from a distribution $\chi_i$7, with queries drawn from a possibly different distribution $\chi_i$8 (Bercea et al., 2022). It chooses an element-specific number of probes $\chi_i$9 from a piecewise rule based on $$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$0, including the possibility $$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$1, which means that some elements are effectively always answered YES. Its expected total number of hash probes is

$$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$2

and the construction uses

$$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$3

bits while matching the corresponding lower bound up to lower-order terms (Bercea et al., 2022). Yet the same paper states plainly that Daisy is not a correlated Bloom filter and not a linear Bloom filter in the standard technical senses. Its importance here is conceptual: it shows how far one can go by exploiting marginals $$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$4 and $$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$5 alone, without modeling pairwise or higher-order dependence. That also delineates the boundary beyond which a genuinely correlated model would need new machinery.

The Bloom multifilter literature extends the discussion from one set to many overlapping sets. “Multiple Set Matching and Pre-Filtering with Bloom Multifilters” introduces Bloom Matrix and Bloom Vector for the problem of returning the set of identifiers associated with a queried label (Concas et al., 2019). Bloom Matrix is a binary matrix $$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$6 of size $$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$7 together with an ordering $$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$8 and a family of hash functions; insertion ORs an $$\mathbb{P}\!\left[\chi_{i_1}\chi_{i_2}\cdots \chi_{i_s}=1\right]=p_s,$$9-bit item vector into each hashed row, and lookup ANDs the relevant rows and decodes the result. Bloom Vector instead stores one Bloom filter per item and tests every item-specific filter on query. The paper’s main empirical conclusion is distribution-sensitive: Bloom Matrix is faster, but Bloom Vector more effectively exploits Zipf-like skew for space reduction, and Bloom Matrix may fail to meet an expected false-positive rate below $p_s$0 when the represented data or labels are not uniformly distributed (Concas et al., 2019). This is directly relevant to the “linear” side of the term because Bloom Matrix is a fixed rectangular bit-matrix, and to the “correlated” side because its practical behavior is driven by overlap and non-uniform incidence across many sets.

Taken together, Daisy and Bloom multifilters show two adjacent but distinct uses of structure. Daisy reallocates error budget across elements using known marginals, but retains a standard hash-based bit-array semantics (Bercea et al., 2022). Bloom Matrix and Bloom Vector store many overlapping incidences and reveal that homogeneous sizing is fragile under skew (Concas et al., 2019). Neither paper gives a full theory of correlated occupancy in the sense of joint bit probabilities, but both demonstrate that once filtering departs from a single homogeneous set, global independent-hash intuition ceases to be a reliable guide.

5. Multi-stage and learned compositions

Another major route to correlation is not within a bit array, but across stages. “ChainedFilter: Combining Membership Filters by Chain Rule” gives an information-theoretic formulation in which a membership problem $p_s$1 can be decomposed by the chain rule

$p_s$2

with the lower-bound function

$p_s$3

for general membership (Li et al., 2023). In operational terms, a first-stage filter eliminates most negatives but leaves a false-positive residual, and a second-stage filter is constructed specifically for that residual. The stages are therefore conditionally dependent by design. ChainedFilter is not a linear Bloom filter in the sketching sense, but it is a precise theory of correlated stage composition. Later stages encode the mistakes of earlier stages rather than acting as independent replications.

The learned-filter literature introduces a different form of dependence: correlation between key features and membership or level placement. In “Learned LSM-trees: Two Approaches Using Learned Bloom Filters,” the first method uses a per-level GradientBoostingClassifier to decide whether a Bloom filter probe can be skipped, and the second replaces the ordinary Bloom filter at each level by a learned classifier plus a backup Bloom filter containing the model’s false negatives (Fidalgo et al., 24 Jul 2025). The classifier-skipping design bypasses about one-third of Bloom-filter checks and yields speedups from $p_s$4 to $p_s$5, but it incurs nonzero false-negative rates. The learned replacement reports zero false negatives and achieves 70–80% per-level memory reduction, while retaining baseline latency (Fidalgo et al., 24 Jul 2025). The same paper is explicit that these learned filters are not linear Bloom filters, and that the exploited correlations are between features and membership, not between Bloom bits.

This stage-based and learned work corrects another common misconception: correlation need not arise from neighboring bits or linear offsets alone. A filter can also be correlated because its second stage is built on the false positives of the first, or because a predictor exploits stable associations between key structure and membership. For correlated linear Bloom filters, this suggests that “correlated” should be read broadly enough to include conditional residual designs, while “linear” should be reserved for the subset of such systems whose state remains a regular bit-array or matrix-like layout.

6. Limitations, misconceptions, and open directions

The central limitation of the current literature is that the phrase itself lacks a settled formal target. The survey literature gives neighboring classes, the occupancy literature gives the correct dependence-aware mathematics, bloomRF gives an order-aware correlated range filter, Daisy gives a marginal-aware but non-correlated baseline, Bloom multifilters expose skew and overlap in linear layouts, ChainedFilter gives conditional composition, and learned LSM filters exploit feature-membership correlations (Luo et al., 2018, Burns, 2019, Riegger et al., 2020, Bercea et al., 2022, Concas et al., 2019, Li et al., 2023, Fidalgo et al., 24 Jul 2025). What is missing is a single canonical model that jointly fixes: the placement mechanism, the dependence structure, the query functional, and the precise meaning of linearity.

The most important technical caution is that the occupancy-number reduction can fail. The occupancy framework of (Burns, 2019) works cleanly when exchangeability holds and the query rule depends only on total occupancy $p_s$6. But the same source notes that more general models may lose exchangeability, and explicitly identifies positional non-uniformity, local correlations, and non-exchangeable occupancy indicators as unresolved for this framework. It also states that for a genuinely correlated linear Bloom filter, false positives may depend on the configuration of occupied bits, not just on the total count $p_s$7 (Burns, 2019). bloomRF reinforces this point operationally: its range false positives are shaped by trace overlap, layer saturation, mask pruning, and threshold heuristics, so a single scalar occupancy statistic is not enough (Riegger et al., 2020).

A second caution is that product-distribution optimality does not transfer automatically to correlated sources. Daisy’s lower bound and construction are tied to sets generated by independent draws from $p_s$8, with a false-positive objective weighted by $p_s$9; the paper states that under correlated set sources the source entropy and coding structure would be different, and $s$0 would no longer be the obvious benchmark (Bercea et al., 2022). The same issue appears in learned systems, where empirical gains depend on stable feature-membership regularities that are external to classical Bloom-filter occupancy theory (Fidalgo et al., 24 Jul 2025).

The open problems therefore follow directly from the cited works. A full theory of correlated linear Bloom filters would need, at minimum, a model for joint bit occupancy beyond exchangeable $s$1, possibly position-dependent probabilities $s$2, a false-positive functional sensitive to pattern geometry rather than only occupancy count, and design rules that connect blocked or order-preserving layouts to exact or bounded false-positive behavior (Burns, 2019). For non-product data sources, it would also need entropy or lower-bound arguments that respect correlations in set generation rather than only marginals (Bercea et al., 2022). For staged or learned variants, it would need to state clearly whether correlation is in the bit array, in the composition rule, or in the predictor that mediates access to the bit array (Li et al., 2023, Fidalgo et al., 24 Jul 2025).

In that sense, correlated linear Bloom filters are best viewed not as one finished data structure, but as the intersection of three mature research themes: exact dependence-aware Bloom-filter analysis, structured linear or locally ordered layouts, and conditional composition across stages or models. The literature already shows that each axis materially changes false-positive behavior, memory locality, or achievable space; what remains open is a unified theory that treats all three simultaneously.