Policy-aware Vector Search: A Vision for Fine Grained Access Control in Vector Databases

Abstract: Vector databases are increasingly used in security sensitive contexts with Retrieval Augmented Generation and organizational AI pipelines; however, their security capabilities remain limited. Specifically, Fine-grained Access Control (FGAC) which is required to ensure that data access adheres to user-specific policies is not fully supported in modern vector databases. Unlike relational databases, vector databases combine structured and unstructured attributes to provide semantic, approximate query results, which complicates FGAC implementation. This creates an inherent tension between enforcing FGAC policies correctly, achieving high ANN search recall and maintaining low query latency. In this paper, we present a vision for Policy-aware Vector Search by formalizing the FGAC policy model in vector databases as well as the enforcement problem. We compare various enforcement strategies, present preliminary findings, and identify key open challenges for future research in policy-aware vector search.

• The paper presents formal models for FGAC that integrate vector-based retrieval with metadata filtering to ensure policy correctness.
• It compares enforcement strategies—pre-filtering, post-filtering, iterative, and parallel—highlighting trade-offs between high recall and low latency.
• Regression and random forest models are used to optimize query planning, enabling dynamic, scalable, and policy-compliant vector search.

Policy-aware Vector Search: Fine-Grained Access Control in Vector Databases

Motivation and Problem Scope

Vector databases are central to modern AI pipelines, particularly for RAG, semantic search, and recommendation systems. Ensuring FGAC in such systems is imperative when deployed in multi-tenant or security-sensitive environments. Unlike relational DBs, vector DBs operate on high-dimensional embeddings and unstructured data, making traditional FGAC strategies inadequate. The enforcement of FGAC demands not only the policy correctness, guaranteeing authorized access, but also optimal recall and minimal latency—requirements that are often conflicting due to the inherent characteristics of ANN search and the complexity of access control policies.

Formal Models for Data, Query, and Policy

The paper formalizes a data model where each record comprises a vector embedding alongside scalar metadata. Queries are expressed as tuples combining a vector search and optional metadata predicates, with top-$k$ retrieval semantics. FGAC policies are articulated in the ABAC framework, consisting of object constraints (metadata predicates), subject constraints (user attributes), and actions (allow/deny decisions). Policies may target metadata attributes or, more challengingly, vector features, necessitating advanced enforcement mechanisms.

Enforcement Strategies and Tradeoffs

The enforcement strategies are delineated as:

• Pre-filtering: Metadata filters are applied prior to ANN search. Guarantees policy compliance and high recall under selective policies but may underutilize index structures like HNSW, and incur overhead when selectivity is low.
• Post-filtering: ANN search is executed across the dataset, and policy predicates are enforced on the results. Exhibits low latency but potentially disastrous recall, particularly if policy selectivity is high and query-policy correlation is low.
• Iterative Post-filtering: Expands candidate space by repeated search and filtering until policy-compliant top-$k$ results are obtained. Improved recall over naive post-filtering but with substantial latency penalties.
• Parallel Post-filtering: Conducts multiple concurrent ANN searches using semantically distinct query variants, then aggregates and filters candidates. Achieves competitive recall at reduced latency compared to iterative approaches.

Empirical comparisons highlight that no single strategy dominates across all workloads. Pre-filtering excels in policy selectivity, whereas post-filtering favors low-selectivity queries. Hybrid methods offer intermediate latency-recall tradeoffs, but bring added complexity, especially regarding policy correlation and dynamic policy updates. The optimization problem is formalized as minimizing cost subject to policy correctness and user-specified recall thresholds.

Figure 1: Default Recall comparison between Pre-filtering and Naive Post-filtering as policy selectivity varies.

Analytical Cost and Recall Models

The authors introduce regression-based and random forest models for estimating recall and latency under various enforcement strategies. Linear models suffice for scalar filtering, whereas vector search latency is highly variable and benefits from RFR in log-space. These estimators feed a query optimizer capable of dynamically selecting the optimal enforcement strategy for a given query and policy scenario. In experiments, these models achieved low error rates in plan selection, with mispredictions arising primarily from high query-policy correlations.

Experimental Evaluation

Experiments were conducted on Pgvector with the arXiv dataset, embedding paper titles and abstracts. Four enforcement strategies were evaluated across diverse policy templates and selectivity levels. Key findings include:

• Pre-filtering achieved consistently high recall but incurred higher latency in low-selectivity scenarios.
• Naive post-filtering offered minimal latency but very poor recall, except where policies were highly correlated with queries.
• Iterative and parallel post-filtering improved recall and reduced latency, particularly for policies of medium selectivity.

Vector-based Object Conditions and Storage Challenges

While metadata-based object constraints are tractable, supporting enforcement over vector-based conditions necessitates novel index architectures. Dual-indexing (exact + ANN), policy subgraph construction, and joint embedding are proposed as potential directions. Dynamic policy management imposes additional overhead for storage and maintenance, particularly as policies change and impact selectivity or index structure.

Practical Implications and Theoretical Considerations

The paper underscores that FGAC in vector DBs is not merely a filtered ANN problem but introduces correctness as a fundamental requirement. Deployment in AI/NLP RAG pipelines mandates that unauthorized retrieval is eliminated, reinforcing security and regulatory compliance. Hybrid cost models and query planners are essential for balancing retrieval latency and recall with enforcement guarantees.

Theoretical implications include the need to reinterpret soundness, security, and maximality criteria from relational FGAC systems in the context of semantic vector retrieval (2606.19803). Approximate search makes correctness guarantees subtle, demanding rigorous approaches in policy modeling and enforcement.

Future Directions in AI and Vector Search

Future research will focus on unified cost/recall models for hybrid enforcement strategies, efficient policy storage and update mechanisms, and architectural innovations for vector-based policy enforcement. System-level validation across multiple DB implementations and datasets is essential for robustness. Co-designing access control and ANN retrieval from inception, rather than post-hoc enforcement, will be pivotal for secure, scalable vector search in organizational AI.

Conclusion

This paper advances the vision for FGAC in vector databases, formalizing policy-aware vector search and characterizing key enforcement strategies and tradeoffs. Analytical models guide optimal plan selection, and empirical results substantiate the need for hybrid approaches. The work establishes a comprehensive foundation for secure, efficient access control in vector-powered AI systems and fosters future developments for robust, policy-compliant retrieval in emerging multi-tenant environments.