Learned Bloom Filter Design

• Learned Bloom filter design is defined by combining a learned scoring function with a backup Bloom filter to guarantee no false negatives while reducing memory usage.
• It utilizes various architectural variants such as sandwiched, adaptive, partitioned, and cascaded designs to balance false positive rates, space constraints, and reject times under different workload complexities.
• Recent optimization techniques like fast PLBF and adversarial hardening methods enhance construction speed and robustness against distribution shifts and adaptive attacks.

Learned Bloom filter design is the study of approximate membership data structures that combine a learned scoring function with one or more Bloom-filter-based components in order to exploit structure in stored keys and query traffic more effectively than a classical Bloom filter. In the canonical formulation, a model $f:U\to[0,1]$ produces a score, a threshold $\tau$ determines the model’s accept region, and a backup filter repairs the model’s false negatives so that the overall structure preserves the no-false-negative property; later work generalized this formulation to sandwiched, adaptive, partitioned, cascaded, multidimensional, neural, and adversarially secure designs (Mitzenmacher, 2018, Mitzenmacher, 2018, Vaidya et al., 2020, Sato et al., 6 Feb 2025, Almashaqbeh et al., 2024).

1. Formal model and semantic guarantees

A standard learned Bloom filter can be written as $(f,\tau,B)$, where $f:U\to[0,1]$ is a learned function, $\tau$ is a threshold, and $B$ is a backup Bloom filter that stores exactly the keys

$\{z : z \in {\cal K},\ f(z) < \tau\}.$

The query rule is equally standard: return positive if $f(y)\ge \tau$; otherwise query $B$; return positive if $B$ accepts and negative otherwise. This construction has no false negatives because every true key rejected by $\tau$0 is inserted into the backup filter, and its total space is $\tau$1 (Mitzenmacher, 2018).

The defining theoretical distinction from a classical Bloom filter is that the false positive behavior is query-distribution-dependent. For a query distribution $\tau$2 over $\tau$3, the false positive rate is

$\tau$4

where $\tau$5 is the backup filter’s false positive rate. The consequence is that a learned Bloom filter does not admit the same query-independent guarantee that a standard Bloom filter does; empirical false positive estimates are meaningful when test queries and future queries are drawn from the same distribution, in which case concentration bounds justify transfer from test to deployment (Mitzenmacher, 2018).

This modeling perspective also clarifies when learning helps. Space savings require a compact model and a small residual false-negative set, because the backup filter may itself need a lower false positive rate than a classical Bloom filter would. A recurring misconception is that the learned component is merely an interchangeable pre-classifier; the formal model instead makes the learned Bloom filter a workload-coupled composite object whose guarantees depend on both model quality and query population (Mitzenmacher, 2019).

2. Design objectives and dataset dependence

Learned Bloom filter design is not a single-metric optimization problem. One line of work explicitly formulates the problem as multi-criteria design under constraints involving false positive rate (FPR), space usage, and reject time, and treats classifier selection as part of the filter design rather than as a preprocessing choice (Malchiodi et al., 2022). In that formulation, the threshold $\tau$6 is a central design variable because it simultaneously controls classifier false positives, classifier false negatives, backup-filter size, and reject time. For a standard learned Bloom filter, the empirical false positive rate is written as

$\tau$7

with $\tau$8 the classifier false positive rate on non-keys and $\tau$9 the backup filter’s false positive rate (Malchiodi et al., 2022).

The same work emphasizes that dataset classification complexity materially changes which learned Bloom filter configurations are preferable. It uses the feature-based measure $(f,\tau,B)$0 and the class-imbalance measure $(f,\tau,B)$1, and reports two broad empirical regimes: easy datasets, roughly $(f,\tau,B)$2, and hard datasets, roughly $(f,\tau,B)$3. On easy datasets, simple classifiers are often preferred because more complex models consume budget that could otherwise be assigned to backup filters; on hard datasets, more expressive models become necessary, and simple classifiers can become useless or even worse than a baseline Bloom filter (Malchiodi et al., 2022).

A second recurrent design issue is the balance between model size and filter size. In the standard model, total space is $(f,\tau,B)$4, so the “best” classifier for prediction is not necessarily the best classifier for a learned filter. This is why later work treats model storage, backup-filter storage, and in some cases reject latency as jointly optimized resources rather than isolated quantities (Mitzenmacher, 2018, Sato et al., 6 Feb 2025).

3. Architectural families

The learned Bloom filter literature now contains several recurring architectural templates, each defined by how it uses model scores and where it places Bloom-filter-based stages.

Design	Key mechanism	Stated emphasis
Standard LBF	classifier first, then backup Bloom filter for false negatives	baseline learned Bloom filter (Mitzenmacher, 2018)
Sandwiched LBF	initial Bloom filter $(f,\tau,B)$5 learned function $(f,\tau,B)$6 backup Bloom filter	lower false positives and robustness to distribution shift (Mitzenmacher, 2018)
Ada-BF / disjoint Ada-BF	multiple score regions with region-specific hash counts or separate Bloom filters	full use of score spectrum (Dai et al., 2019)
PLBF	consecutive score partitions with region-specific backup FPRs	memory minimization under overall FPR constraint (Vaidya et al., 2020)
CLBF	trunk filters, model cascade, branch filters, final filters	joint optimization of model size, filter size, and reject time (Sato et al., 6 Feb 2025)

The sandwiched learned Bloom filter inserts an initial Bloom filter before the learned model and retains a backup Bloom filter after it. Its analytical false positive rate is

$(f,\tau,B)$7

where $(f,\tau,B)$8 and $(f,\tau,B)$9 are bits per original key assigned to the initial and backup filters, $f:U\to[0,1]$0 and $f:U\to[0,1]$1 are learned-model false positive and false negative rates, and $f:U\to[0,1]$2 is the Bloom-filter false-positive base. The notable design result is that the optimal backup-filter size

$f:U\to[0,1]$3

is constant with respect to the total budget $f:U\to[0,1]$4; once the backup is large enough, extra memory should go to the initial Bloom filter rather than the backup filter (Mitzenmacher, 2018).

Adaptive Learned Bloom Filters (Ada-BF) replace the single threshold by $f:U\to[0,1]$5 score regions

$f:U\to[0,1]$6

and assign each region a possibly different number of hash functions $f:U\to[0,1]$7. In the single-filter version, all keys share one Bloom filter but are inserted using score-dependent $f:U\to[0,1]$8; in the disjoint version, each group gets its own Bloom filter. The original learned Bloom filter is a special case with $f:U\to[0,1]$9, $\tau$0, and $\tau$1, so Ada-BF is a strict generalization of threshold LBFs (Dai et al., 2019).

Partitioned Learned Bloom Filters (PLBFs) go further by explicitly partitioning the score range into $\tau$2 contiguous regions and assigning a distinct backup-filter false positive rate $\tau$3 to each region. This architecture turns score usage into a constrained optimization problem, and the paper explicitly states that sandwiching is a special case of PLBF with two regions (Vaidya et al., 2020).

Cascaded Learned Bloom Filters (CLBFs) generalize both the single-model and partitioned settings by alternating Trunk Bloom Filters, ML stages, Branch Bloom Filters, and Final Bloom Filters. The query path can stop early at several points, so CLBF is designed not only for memory efficiency but also for fast rejection. The paper states that Sandwiched LBF can be seen as a special case with one trunk and one final filter, while PLBF can be seen as a special case with no trunk or branch filters and multiple final filters (Sato et al., 6 Feb 2025).

4. Optimization criteria over scores, regions, and stages

For Ada-BF, the expected groupwise false positive rate is

$\tau$4

with overall expected false positive rate

$\tau$5

where $\tau$6 is the probability that a non-key query falls into score group $\tau$7. The design principle is that lower-score regions, which typically contain more non-keys, should receive stronger Bloom-filter treatment than higher-score regions (Dai et al., 2019).

For PLBF, the optimization problem is written directly in terms of score-region masses. If

$\tau$8

are the key and non-key masses in region $\tau$9, then the objective is

$B$0

subject to

$B$1

In the relaxed problem, the optimal per-region false positive rates satisfy

$B$2

Substituting this into the objective yields a KL-divergence form, so threshold optimization reduces to maximizing

$B$3

The paper interprets the resulting space saving over an ordinary Bloom filter as

$B$4

which makes the design criterion explicitly information-theoretic: learned filters help when the score partition induces enough separation between key and non-key distributions to pay for the model (Vaidya et al., 2020).

For CLBF, the optimization target includes memory and reject time simultaneously. The objective is

$B$5

where $B$6 is total memory, $B$7 is expected reject time, and $B$8 sets the memory–latency trade-off. The expected reject time is approximated as

$B$9

so trunk-filter false positive rates directly determine how many non-keys survive to later, more expensive model stages (Sato et al., 6 Feb 2025).

A plausible implication is that learned Bloom filter design has bifurcated into at least three optimization regimes: threshold allocation, score-partition allocation, and cascade-structure allocation. The literature treats these as structurally different problems because the placement of model and Bloom-filter stages changes both the feasible memory budget and the attainable reject-time profile (Dai et al., 2019, Vaidya et al., 2020, Sato et al., 6 Feb 2025).

5. Construction algorithms and practical scalability

The most acute construction bottleneck emerged in PLBF. In its original form, PLBF discretizes the score space into $\{z : z \in {\cal K},\ f(z) < \tau\}.$0 segments, clusters them into $\{z : z \in {\cal K},\ f(z) < \tau\}.$1 consecutive regions, and solves a dynamic program for every $\{z : z \in {\cal K},\ f(z) < \tau\}.$2, yielding worst-case construction complexity

$\{z : z \in {\cal K},\ f(z) < \tau\}.$3

The construction cost is the main practical limitation because larger $\{z : z \in {\cal K},\ f(z) < \tau\}.$4 improves the approximation to the best partition but makes optimization prohibitively expensive (Vaidya et al., 2020).

“Fast Partitioned Learned Bloom Filter” removes this bottleneck without changing the exact PLBF objective. fast PLBF observes that the last dynamic-programming table already contains the information required to recover optimal thresholds for every $\{z : z \in {\cal K},\ f(z) < \tau\}.$5, so the same final data structure can be obtained in

$\{z : z \in {\cal K},\ f(z) < \tau\}.$6

rather than $\{z : z \in {\cal K},\ f(z) < \tau\}.$7. The paper states that fast PLBF preserves the same optimization problem, the same thresholds $\{z : z \in {\cal K},\ f(z) < \tau\}.$8, the same false-positive-rate assignment logic, and therefore exactly the same final PLBF as the original method (Sato et al., 2023).

The same paper introduces fast PLBF++, which computes dynamic-programming layers by exploiting monotone-matrix structure and achieves

$\{z : z \in {\cal K},\ f(z) < \tau\}.$9

It need not always produce the identical structure as PLBF, but it is proved to coincide with PLBF under the ideal ordering condition

$f(y)\ge \tau$0

On real-world datasets, the paper reports that fast PLBF and fast PLBF++ can construct the data structure up to 233 and 761 times faster than PLBF, that fast PLBF achieves the same memory efficiency as PLBF, and that fast PLBF++ achieves almost the same memory efficiency; with $f(y)\ge \tau$1 and $f(y)\ge \tau$2, PLBF takes more than 3 minutes to construct, while fast PLBF and fast PLBF++ take under 2 seconds (Sato et al., 2023).

“Fast Construction of Partitioned Learned Bloom Filter with Theoretical Guarantees” adds fast PLBF#, which uses SMAWK under total monotonicity and brings total complexity to

$f(y)\ge \tau$3

The paper proves that fast PLBF++ and fast PLBF# are equivalent to PLBF under ideal score distributions and gives an upper bound on the objective loss for non-ideal distributions. Experiments report speedups of up to 233, 761, and 778 for fast PLBF, fast PLBF++, and fast PLBF#, respectively, while fast PLBF maintains the same data structure as PLBF and the approximate variants remain nearly identical in memory efficiency (Sato et al., 2024).

These results changed the practical status of PLBF-style design. Before the fast-construction work, PLBF was primarily a strong optimization target; after it, large $f(y)\ge \tau$4 and larger $f(y)\ge \tau$5 became operationally reasonable rather than merely asymptotically appealing (Sato et al., 2023, Sato et al., 2024).

6. Robustness, reject time, and adversarial security

Robustness has two distinct meanings in this literature. The first is robustness to dataset complexity and classifier variability. A critical empirical study reports that the Sandwiched Learned Bloom Filter (SLBF) is the most robust variant under changing data complexity and classifier performance variability, and that it often has smaller reject times. The same study also reports that ADA-BF can be strongest in false-positive-rate terms, especially on benchmark URL data, but often suffers from the largest reject time (Malchiodi et al., 2022).

The second meaning is robustness to distribution shift and adaptive attack. Sandwiching already improves robustness in a non-cryptographic sense because the initial Bloom filter screens many queries before they reach the learned model, reducing sensitivity to mismatch between the distribution used to estimate $f(y)\ge \tau$6 and the true query distribution (Mitzenmacher, 2018). A stronger adversarial problem remained open until “Adversary Resilient Learned Bloom Filters,” which defines adaptive security notions for learned Bloom filters, including full and partial adaptivity, and learned extensions of Always-Bet and Bet-or-Pass models (Almashaqbeh et al., 2024).

That paper proposes two secure learned-Bloom-filter constructions, PRP-LBF and Cuckoo-LBF, and the detailed construction discussion centers on the Downtown Bodega Filter. In the main secure design, both the model-positive branch and the model-negative branch are routed through PRP-secured backup filters, so the final membership decision is not exposed directly to adversarial probing of the learned model. The security guarantee is written as

$f(y)\ge \tau$7

assuming pseudo-random permutations exist, and the main theorem states that a secure learned Bloom filter can be built with

$f(y)\ge \tau$8

bits of extra memory over the sandwiched learned Bloom filter (Almashaqbeh et al., 2024).

The literature therefore separates benign-workload optimization from adversarial hardening. A design that is memory-optimal under stable, sampled non-key traffic is not automatically secure under adaptive querying, and the secure constructions explicitly spend memory on cryptographic hardening rather than on classifier or backup-filter refinement (Mitzenmacher, 2018, Almashaqbeh et al., 2024).

7. Systems deployments and emerging variants

A prominent systems application is the LSM-tree. “Learned LSM-trees: Two Approaches Using Learned Bloom Filters” uses a per-level learned Bloom filter in which a classifier $f(y)\ge \tau$9 predicts whether a key belongs to level $B$0, and a per-level backup Bloom filter catches classifier false negatives. The lookup path is $B$4 and the paper reports zero false negatives for the learned Bloom filter across all workloads, baseline latency of about 1.00× or within 1%, and 70–80% memory reduction per level, with learned Bloom filters using about 530–903 KB per level (Fidalgo et al., 24 Jul 2025). The paper explicitly frames this design as a memory optimization rather than a latency optimization.

Another extension is the Neural Bloom Filter, a meta-learned memory-augmented architecture for one-shot approximate set membership. It replaces fixed hashing with learned addressing, using additive writes

$B$1

and a learned nonlinear readout. Because the neural model can produce false negatives, the design uses a backup Bloom filter to restore the usual correctness guarantee. The paper reports substantial compression gains on structured familiarity tasks and a database task, while also noting that neural models are much slower than classic Bloom filters on CPU and become more competitive only with batching and GPU execution (Rae et al., 2019).

For multidimensional membership, “Compressing (Multidimensional) Learned Bloom Filters” reduces model size by losslessly splitting high-cardinality columns into smaller subcolumns before embedding. On the Airplane dataset, the compressed model reduces learned-filter memory from 4.06 MB to as low as 0.53 MB while accuracy changes from 0.98 to 0.95 at the most aggressive compression; on the DMV dataset, memory falls from 1.97 MB to 0.36 MB while maintaining 0.98 accuracy (Davitkova et al., 2022). The design lesson is that in learned Bloom filters with embeddings, input representation can dominate memory more than the backup filter.

A more recent variant uses Counting Bloom Filters (CBFs) as certainty-aware evidence sources rather than binary backstops. “Learning Filters with Certainty” derives a posterior membership estimate from counter values and combines it with learned priors in four architectures. The decision rule is based on

$B$2

with the counter product $B$3 acting as a sufficient signal for certainty under the paper’s model. Unlike the classical learned Bloom filter, these certainty-aware pipelines may introduce false negatives when the posterior threshold is strict, shifting the design space from one-sided error to a tunable false-positive/false-negative trade-off (Banoun et al., 22 Jun 2026).

Taken together, these deployments and extensions show that learned Bloom filter design is no longer restricted to the original single-model-plus-backup pattern. It now spans storage engines, multidimensional indexes, meta-learned memory systems, and certainty-aware counting-filter hybrids, but the central design questions remain constant: how scores are used, where probabilistic correction is placed, which guarantees are preserved, and how memory is divided between learned and non-learned components (Fidalgo et al., 24 Jul 2025, Rae et al., 2019, Davitkova et al., 2022, Banoun et al., 22 Jun 2026).