nnFilterMatch: Learned Filter-Aware ANN Search

• nnFilterMatch is a learned filter-aware nearest neighbor retrieval approach that integrates label filtering into vector similarity for multi-label AND queries.
• The method learns a penalty weight via constrained linear optimization, balancing vector distance and label mismatch during both querying and graph construction.
• Empirical results on datasets like YFCC1M and Wiki35M report a 5-10% improvement in accuracy over fixed-penalty methods while reducing unnecessary vector comparisons.

nnFilterMatch is a learned, filter-aware nearest-neighbor retrieval scheme for filtered approximate nearest neighbor (ANN) search with multi-label AND queries. In this setting, a query carries a set of discrete labels $S_q$, and a result vector $v$ is valid only if it satisfies all query labels, i.e. $S_q \subseteq S_v$. The method replaces hand-tuned or fixed penalties with a data-driven distance metric that explicitly trades off ordinary vector similarity against filter satisfaction, learning the penalty weight from training data through a constrained linear optimization problem. The learned weight is then used during both querying and index construction, with reported improvements of about 5-10% in accuracy over fixed-penalty baselines (Sutradhar et al., 6 Nov 2025).

1. Problem formulation and filter semantics

nnFilterMatch is defined for filtered ANN retrieval with multiple discrete filters under AND semantics. Each query $q$ is associated with a label set $S_q \subseteq [m]$, and each data vector $v$ has its own label set $S_v \subseteq [m]$. A candidate is valid only when it contains all query labels:

$S_q \subseteq S_v.$

This formulation treats multiple filters as a set-membership condition rather than as independent post-processing predicates (Sutradhar et al., 6 Nov 2025).

For each query-candidate pair, the method uses two quantities. The first is a vector distance $d(q,v)$, such as Euclidean or cosine distance. The second is a label-match score

$m(q,v) = \frac{|S_q \cap S_v|}{|S_q|},$

so that $v$0 if and only if the candidate satisfies all labels in the query. The learned filter-aware distance is

$v$1

with $v$2 a learned penalty weight. If a candidate matches all labels, the second term is zero; if it violates some labels, it is penalized in proportion to the amount of mismatch (Sutradhar et al., 6 Nov 2025).

This design implements soft ranking rather than strict hard filtering or crude post-filtering. Candidates that are slightly farther in embedding space but satisfy the query labels can outrank closer candidates that do not. The method therefore integrates label satisfaction directly into the search objective instead of treating filtering as an external constraint applied only after traversal.

2. Learned distance metric and linear optimization

The central methodological step in nnFilterMatch is the learning of $v$3 from training queries. The paper constructs pairwise preference constraints from ground-truth nearest neighbors obtained by exhaustive search. For each query $v$4, the ground-truth top-$v$5 neighbors are denoted by $v$6. Positives are defined as

$v$7

that is, ground-truth neighbors that satisfy the filter. For each positive $v$8, negatives are

$v$9

namely candidates that are closer in vector space but fail the filter (Sutradhar et al., 6 Nov 2025).

The weight is learned by solving a linear program with variable $S_q \subseteq S_v$0 and slack variables $S_q \subseteq S_v$1. The objective is

$S_q \subseteq S_v$2

where

$S_q \subseteq S_v$3

The constraints are

$S_q \subseteq S_v$4

for all $S_q \subseteq S_v$5, with margin $S_q \subseteq S_v$6 (Sutradhar et al., 6 Nov 2025).

Because $S_q \subseteq S_v$7 for positives, the left-hand side reduces to $S_q \subseteq S_v$8, while negatives incur a penalty term. The optimization therefore chooses the smallest $S_q \subseteq S_v$9 that still separates filter-satisfying neighbors from closer but invalid candidates, while allowing violations through slack. The paper states that $q$0 controls the trade-off between minimizing the penalty weight and the total slack, is selected by grid search, and uses $q$1 (Sutradhar et al., 6 Nov 2025).

The interpretation of the learned weight is straightforward. A small $q$2 makes filter mismatch less important, so behavior approaches standard ANN. A large $q$3 strongly prefers filter satisfaction and, in the extreme, approximates hard filtering.

3. Search procedure and graph construction

A defining feature of nnFilterMatch is that the learned weight is used both at query time and during index construction. At query time, graph-search priority is based on the learned distance

$q$4

Search proceeds as a weighted greedy / best-first graph traversal. Neighbors are expanded according to this joint score, so nodes that better match the query labels are more likely to be visited and retained in the candidate set. This differs from applying penalties only after traversal, because the search path itself becomes label-aware (Sutradhar et al., 6 Nov 2025).

The same principle is injected into graph building. The method defines a label-aware distance between data points

$q$5

where

$q$6

This is an asymmetric Jaccard-style label overlap. It encourages edges between points that are not only close in embedding space but also similar in label structure (Sutradhar et al., 6 Nov 2025).

The paper uses this metric inside a modified greedy graph builder based on FilteredDiskANN. The stated intuition is that the index should already encode the filter structure, rather than relying on query-time heuristics to recover it. Two mechanisms are emphasized. First, traversal is guided toward nodes likely to satisfy the query labels, reducing wasted exploration on irrelevant graph regions. Second, the graph topology itself becomes label-aware, increasing the chance that relevant filtered neighbors are connected or reachable through short paths. The paper explicitly attributes improved generalization over fixed penalties to adaptation to the dataset’s label distribution and query selectivity.

4. Relation to fixed-penalty and heuristic filtered ANN methods

The paper positions nnFilterMatch against three common patterns in filtered ANN systems. The first is post-filtering: the index is built and searched using only vector distances, and filtering is applied afterward. The stated problem is that the search path is not label-aware and may miss good answers unless the system over-provisions the search (Sutradhar et al., 6 Nov 2025).

The second is fixed-penalty search, in which distance is a linear mix of vector distance and label mismatch with a manually chosen constant penalty. The paper argues that a single global penalty does not adapt to different datasets, query selectivities, or label semantics. The third is heuristic graph pruning or prioritization, where the search is biased toward filter-satisfying nodes or edges are pruned heuristically. The stated limitation is that such heuristics are data-independent and may break search paths or over-constrain the graph.

In the broader filtered ANN literature, these distinctions align with the general taxonomy of pre-filtering, post-filtering, and joint-filtering strategies. A unified experimental study of Filtering ANN search characterizes post-filtering as searching first and filtering candidates afterward, and joint-filtering as integrating filtering into traversal so that invalid paths can be pruned early (Li et al., 22 Aug 2025). Within that taxonomy, nnFilterMatch belongs to the joint-filtering family, but differs from heuristic joint-distance methods by learning the trade-off between vector similarity and filter match from data rather than fixing it a priori (Sutradhar et al., 6 Nov 2025).

A common misconception is to treat all filter-aware graph methods as variants of the same penalty rule. nnFilterMatch is more specific: it learns a single penalty weight through pairwise constraints derived from ground-truth filtered neighbors, then applies that weight consistently in both search and graph construction. This suggests a more principled calibration than manually setting a mismatch coefficient.

5. Experimental evaluation and reported results

The evaluation is conducted on two real-world datasets. YFCC1M contains 1 million image embeddings from Yahoo Flickr Creative Commons, with 192-dimensional CLIP embeddings and metadata labels such as camera model, year, and location. Its queries include single-label predicates and conjunctions of two labels. Wikipedia / Wiki35M contains about 35 million passages, 768-dimensional sentence embeddings, and labels derived from keywords, semantic label matching, and Wikipedia categories; it is designed for AND-style multi-label queries (Sutradhar et al., 6 Nov 2025).

The train/evaluation splits are reported explicitly. For YFCC1M, the paper uses 11,865 training queries and 5,718 evaluation queries. For Wikipedia, it uses 308 training queries and 464 evaluation queries. The reported metrics are Recall@10, defined as the fraction of ground-truth top-10 filtered neighbors that appear in the retrieved top-10, and the average number of vector distance comparisons, used as a hardware-independent proxy for search latency (Sutradhar et al., 6 Nov 2025).

The baselines are Fixed Penalty Search, which uses a normal graph build and a query-time fixed penalty combining distance and mismatch, and Post Filtering, which builds and searches by pure vector distance and filters the output afterward. The paper also reports unfiltered search behavior to show that filter-aware graph construction does not harm standard ANN retrieval (Sutradhar et al., 6 Nov 2025).

The main empirical result is that the learned filter-aware distance improves accuracy by about 5-10% over fixed-penalty methods, depending on dataset and query type, and reduces the number of distance comparisons needed to achieve a given recall. The paper reports specific learned weights of $q$7 for Wikipedia-35M and $q$8 for YFCC. These values are used to illustrate the claim that the optimal balance between vector similarity and filter match depends on dataset statistics and label selectivity (Sutradhar et al., 6 Nov 2025).

6. Operating regimes, scope, and relation to adjacent filtered-search systems

The paper identifies the settings in which nnFilterMatch is especially helpful. These include cases where filter-satisfying neighbors are rare or highly selective, label semantics vary across datasets, query predicates are AND-style and must satisfy multiple labels, and there is a meaningful interaction between label overlap and vector proximity. It further notes that when relevant filtered neighbors are sparse, the learned penalty tends to increase, making filter satisfaction more important; when labels are less selective, vector distance plays a larger role (Sutradhar et al., 6 Nov 2025).

This operating profile places nnFilterMatch within a narrower scope than systems built for arbitrary Boolean metadata predicates. For example, PathFinder targets filtered ANNS with arbitrary conjunctions and disjunctions over multiple attributes, lets users build attribute-specific indexes, and uses a cost-based optimizer to choose search plans (Wu et al., 2 Nov 2025). By contrast, nnFilterMatch is formulated directly for multi-label AND queries over discrete labels, with validity defined by $q$9 and mismatch measured by partial label overlap (Sutradhar et al., 6 Nov 2025).

The broader benchmark literature also underscores that selectivity strongly influences which filtered ANN design is most effective. The unified study of attribute filtering reports that joint-filtering is especially useful at moderate selectivity, while different families dominate under extremely low or very high selectivity depending on attribute type and graph design (Li et al., 22 Aug 2025). A plausible implication is that nnFilterMatch is most naturally interpreted as a learned joint-filtering mechanism specialized to discrete multi-label conjunctions, rather than as a universal solution for all filtered ANN query classes.

Within that scope, its contribution is conceptually tight: positives are defined as ground-truth neighbors with full label satisfaction, negatives are closer vectors with incomplete match, the penalty is learned by a constrained linear program, and the resulting score is used consistently throughout the search pipeline. The method therefore treats filter awareness not as a post hoc constraint but as part of the distance geometry used to traverse and construct the ANN graph (Sutradhar et al., 6 Nov 2025).