Predicate-Agnostic Filtered Vector Search

• Predicate-agnostic filtered vector search is a technique that performs similarity search on vector embeddings combined with dynamic, arbitrary filtering conditions.
• It integrates methods such as hidden vector encryption, filter-integrated transformations, and graph-based algorithms to support flexible, secure, and efficient queries.
• Practical systems leverage these techniques to enhance scalability and throughput in multi-tenant, encrypted, and hybrid databases while reducing computational overhead.

Predicate-agnostic filtered vector search refers to enabling similarity search over vector-encoded data in conjunction with arbitrary, potentially ad hoc, structured predicates, without advance knowledge of filter types, cardinalities, or correlations. This concept underpins a large body of recent research in secure, scalable, and high-performance retrieval systems where structured and unstructured data coexist. Predicate-agnostic approaches are designed to support expressive, dynamic query conditions (attribute constraints, user/tenant access, range predicates, etc.) while maintaining stringent requirements on efficiency, scalability, and, in some settings, security and privacy.

1. Core Principles and Definitions

Predicate-agnostic filtered vector search is characterized by the capacity to execute nearest neighbor (kNN) queries on high-dimensional embeddings, filtered by arbitrary combinations of attributes, labels, or tokens—regardless of their distribution, domain, or whether they have been seen during index construction.

In encrypted contexts, hidden vector encryption (HVE) (1702.07456) provides a cryptographic foundation for this functionality. HVE tags each ciphertext with an attribute vector $x$ and generates decryption tokens for predicates $f$; only when $f(x)=1$ does decryption succeed. The scheme supports conjunction, subset, and comparison predicates, while wildcards allow “don’t care” positions, making the system natively predicate-agnostic. For unencrypted search, recent vector database and system designs generalize this principle for fast attribute-value filtering.

Key definitions:

• Predicate-agnostic filtering: The process of performing vector search with filtering conditions that are arbitrary and not fixed in advance.
• Hybrid search: Combination of similarity search (vectors) and structured query predicates (attributes, labels).
• Wildcard/Token-based filtering: In HVE, wildcards in tokens provide flexibility to express a variety of predicates post-index or post-encryption.

2. Predicate-Agnostic Techniques: Algorithms and System Strategies

Multiple methodologies have emerged to realize predicate-agnostic filtered vector search. They differ in their foundational approach, performance profile, and system implications.

2.1 Predicate-Oblivious Indexing and Filtering

Hidden vector encryption (HVE) (1702.07456) tightly couples data protection with predicate-agnostic retrieval. Ciphertexts are constructed to hide underlying data and attributes completely; tokens, generated for arbitrary predicates, allow decryption if and only if the predicate matches. Notably, token size and decryption complexity are constant in cutting-edge HVE schemes, enabling scalable search even for large, high-dimensional attribute spaces.

In unencrypted settings, systems such as Curator (2401.07119) employ tenant clustering trees encoded as subtrees of a global clustering tree. Here, access control predicates are encoded structurally (using Bloom filters and per-tenant shortlists) rather than being evaluated per-query, conveniently decoupling index structure from specific filter conditions and thus supporting arbitrarily complex access patterns.

2.2 Filter-Integrated Vector Transformations

The filter-centric vector indexing (FCVI) approach (2506.15987) marries filter conditions with embedding vectors via a geometric transformation: $$\psi(v, f, \alpha) = [v^{(1)} - \alpha f,\ldots, v^{(d/m)} - \alpha f]$$ where $v$ is the vector, $f$ encodes filter attribute(s), $d/m$ is the number of segments, and $\alpha$ controls the strength of filter separation. This enables unmodified use of popular vector search algorithms (HNSW, FAISS, etc.) while guaranteeing that pairs sharing the same filter retain original distances and others are separated quadratically with respect to $\alpha$.

2.3 Partitioning and Optimized Predicate Pruning

Partition-based techniques (HQI (2304.01926), Elastic Index Select (2505.03212)) rely on partitioning the data jointly by attribute predicates and vector similarity. HQI builds a workload-aware index partitioned along both attributes and embedding centroids, using a qd-tree. Elastic Index Select formulates index selection as a set-cover-style optimization, selectively materializing partial indexes to cover all possible query predicates with guaranteed minimum selectivity (“elastic factor”), achieving massive space and query time efficiency gains over exhaustive index schemes.

2.4 Graph- and Subgraph-Oriented Search

Graph-based methods leverage navigable proximity structures for vector search, with extensions for predicate-agnostic support. ACORN (2403.04871) expands the neighbor lists of its HNSW index to ensure the predicate-induced subgraph at query time remains searchable for any predicate, not just those anticipated in advance. Predicate subgraph traversal allows efficient search within the matching subset, approximating the theoretical performance of building a bespoke index for each possible predicate.

SVG and SVG-L0 (2506.20584) apply kernel machine learning to learn navigable graph structures for arbitrary (including non-metric) similarity functions, providing theoretical navigability guarantees. The $\ell_0$-constrained SVG-L0 approach automates degree control without heuristic truncation, further enhancing efficiency and robustness.

3. Practical Systems and Performance

Several systems provide practical, high-performance deployments of predicate-agnostic filtered vector search concepts.

3.1 Vector Databases and Multi-Tenancy

Curator (2401.07119) enables multi-tenant vector search with index space and query time close to the minimum possible, leveraging partitioned clustering trees that efficiently encode per-tenant access patterns with Bloom filters and compact shortlists. Search speed is up to 55× faster than metadata filtering and nearly matches per-tenant approaches, but with only a single physical index.

3.2 Batch and Streaming Workloads

HQI (2304.01926) supports high-throughput, batch-optimized hybrid search on knowledge graphs. Its batching mechanisms, which group queries by filter templates and centroid assignments, enable more than 30× improvement in throughput compared to naive post-/pre-filtered approaches, and maintain these gains as batch size scales.

3.3 Elastic and Serverless Search

Serverless-oriented solutions such as SQUASH (2502.01528) employ scalar quantization (OSQ) for vectors and attributes, segment-based storage for memory efficiency, and bitwise attribute masking for filtering, distributing the workload over FaaS instances. This results in up to 18× higher throughput and 9× lower cost per query versus server or managed cloud alternatives, supporting arbitrary predicate combinations.

4. Security, Robustness, and Theoretical Guarantees

Security and robustness are central for predicate-agnostic filtered vector search in privacy-sensitive or adversarial environments.

4.1 Fine-Grained Access and Privacy

HVE (1702.07456) achieves privacy-preserving, fine-grained access to encrypted vectors. Its constant-size tokens permit efficient, scalable filtered search. Notably, filtering tokens can be issued at any time, supporting unplanned use cases and queries without re-encryption. The schemes are proven secure (full security, selective security) under bilinear group and Diffie-HeLLMan assumptions.

4.2 Distributional Robustness

Filter-centric transformations (FCVI (2506.15987)) offer stability to workload and distribution shifts. Unlike methods that rely on upfront, static segmentation, FCVI’s transformation is not tied to specific filter patterns, maintaining consistent accuracy and throughput even as filter distributions change; observed recall loss under strong distribution shift is less than 3%, while legacy approaches may lose 20% or more.

4.3 Theoretical Performance Bounds

ACORN (2403.04871) shows that, with appropriate neighbor expansion, search can be guaranteed to have complexity

$O((d+\gamma)\log (sn) + \log(1/s))$

where $s$ is predicate selectivity and $\gamma$ the expansion factor. FCVI provides bounds for selecting $\alpha$ to achieve desired separation and recall, and Elastic Index Select (2505.03212) offers theoretical guarantees via the elastic factor, bounding the query time by $O(C + k/e)$.

5. Application Domains and Architectural Implications

Predicate-agnostic filtered vector search is foundational in several application areas:

• Privacy-preserving search and encrypted databases: HVE’s token-based filtering is directly applicable to secure cloud search and audit-compliant data platforms.
• Multi-modal hybrid search: Systems like ACORN and HQI enable flexible retrieval in datasets that combine semantic embeddings (text, images, audio) with rich structured data (tags, attributes).
• Multi-tenant and access-controlled datastores: Curator and related partitioned-tree methods naturally encode complex, dynamic access control policies for collaborative environments.
• Elastic and serverless cloud deployment: SQUASH and similar architectures offer efficient hybrid search that adapts to workload bursts and scales cost-effectively in stateless environments.
• Graph databases and joint graph/vector search: NaviX (2506.23397) integrates filtered vector search with graph DBMS engines using a combination of disk-based HNSW and adaptive exploration heuristics, maintaining robustness across varying selectivities and filter correlation patterns.

6. Technical Comparison Table

System/Approach	Predicate Generality	Indexing Overhead	Scalability	Security/Robustness	Integration
HVE (1702.07456)	Any (tokens)	Low	High (const token)	Strong cryptographic proofs	Any pairing group
HQI (2304.01926)	Arbitrary	Moderate	High (batch, multi)	Relies on partition stability	Relational/graph DB
Curator (2401.07119)	Any tenant/filter	Very low	High	Adapts to data shifts	Vector DB
FCVI (2506.15987)	Any (plug-and-play)	Minimal	High	Stable across distributions	HNSW, FAISS, ANNOY
SQUASH (2502.01528)	Any	Very low	High (serverless)	Bitwise/SIMD filtering	FaaS, cloud native
ACORN (2403.04871)	Any (HNSW, γ tunable)	Low-Moderate	High (graph)	Theoretical query bounds	HNSW extension
Elastic Index Select (2505.03212)	Any label set	Moderate to high	High (DAG shared)	Theoretically bounded	Modular, any index
SVG/SVG-L0 (2506.20584)	Any kernel/similarity	Sparse, tunable	High (auto-degree)	Navigability in non-metric	Any similarity metric
NaviX (2506.23397)	Any (DBMS filters)	Moderate to high	High (disk+memory)	Adaptive to selectivity/clusters	Graph DBMS

7. Summary and Outlook

Predicate-agnostic filtered vector search is a foundational capability enabling expressive, scalable, and in some contexts secure, retrieval and analytics over rich, multi-modal datasets. The principal contribution of methods in this family is the decoupling of index (or cryptographic) structure from any preordained set of predicates or labels, whether for attribute filtering, access control, or multi-tenant partitioning. This is achieved through a suite of algorithmic innovations: token-based decryption (HVE), per-tenant partitioned trees (Curator), joint vector-attribute transformations (FCVI), batch-optimized index partitioning (HQI), predicate-agnostic subgraph search (ACORN), and kernelized or self-tuning proximity graphs (SVG-L0).

As the landscape of hybrid search continues to evolve, these methods offer theoretical guarantees, practical scalability, and architectural modularity, enabling robust and efficient deployment across encrypted, distributed, and production-grade search systems.