HubScan: Security Scanner for RAG Systems

• HubScan is an open-source security scanner designed to detect hubness poisoning in retrieval-augmented generation systems by evaluating vector indices and document embeddings.
• It employs a multi-detector architecture, including statistical, cluster spread, and stability detectors, to analyze risk across diverse queries and embedding behaviors.
• HubScan integrates with vector search engines like FAISS and Pinecone to provide actionable risk scores and guide operational practices in RAG systems.

HubScan is an open-source security scanner for detecting hubness poisoning in retrieval-augmented generation (RAG) systems. Hubness refers to the phenomenon in high-dimensional vector spaces where certain points—"hubs"—appear disproportionately often among the top-k retrieved items for diverse queries. Attackers can exploit hubness to poison LLM outputs at scale by inserting documents that behave as universal or domain-specific hubs in the embeddings index, resulting in malicious or misleading content appearing across many unrelated queries. HubScan systematically evaluates vector indices and document embeddings to identify both universal and targeted hubs, providing a risk scoring system for large-scale production RAG systems (Habler et al., 25 Feb 2026).

1. Hubness in High-Dimensional Retrieval Systems

Hubness arises naturally in spaces with dimensionality $d \gg 1$, where some points—due to geometric concentration—have high "hub rate," appearing as nearest neighbors to many unrelated queries, while others ("anti-hubs") almost never appear. Given a gallery of $N$ items with embeddings $\{g_i\}$ and a set of $M$ queries $\{q_j\}$, the hub rate of item $i$ is defined as:

$$r_i = \frac{1}{M}\, \# \left\{ j: g_i \in \text{top-}k(\mathrm{sim}(q_j, \cdot)) \right\}$$

RAG systems, which leverage nearest-neighbor search in embedding space (using engines such as FAISS, Pinecone, Qdrant, or Weaviate), are highly vulnerable to hubness poisoning. A single adversarial embedding can be engineered to have high $r_i$ and dominate retrievals, causing widespread contamination in downstream LLM completions—examples such as Copilot and GeminiJack demonstrate the practical impact, including fact injection, prompt hijacking, and data exfiltration.

2. Multi-Detector Architecture of HubScan

HubScan employs a multi-detector pipeline, with each detector targeting a distinctive aspect of hubness behavior. The pipeline comprises:

1. Loading of embeddings, metadata, and vector index.
2. Query generation, sampling from random gallery items, cluster centroids, and optionally real user queries.
3. $k$-NN or hybrid search across sampled queries, accumulating per-item retrieval statistics.
4. Detector execution: statistical hubness, cluster spread, stability under perturbation, and deduplication, with support for domain-aware and modality-aware analysis.
5. Normalization and risk score fusion to produce a combined anomaly score.
6. Percentile-based verdict assignment (LOW/MEDIUM/HIGH risk).

Key Detector Algorithms

Detector	Definition/Workflow	Key Measure(s)
Statistical Hubness	Counts top-$k$ retrievals per item; robust z-score via median/MAD	$N$0
Cluster Spread	Entropy of retrievals over query clusters (MiniBatch KMeans)	$N$1
Stability	Repeat retrievals under Gaussian-perturbed queries	$N$2
Deduplication	Groups by text hash, embedding distance, or boilerplate suppression	Downweights near-duplicate clusters
Domain-aware	Domain-specific hub rate and Gini concentration coefficient	$N$3, $N$4
Modality-aware	Cross-modal hub score and penalty	$N$5

Signal Fusion

Detector scores are normalized to $N$6 and combined:

$N$7

with recommend weights such as $N$8, $N$9, $\{g_i\}$0, etc. Verdicts are assigned by percentile thresholds: HIGH ($\{g_i\}$1), MEDIUM ($\{g_i\}$2–$\{g_i\}$3), LOW (below $\{g_i\}$4).

3. Statistical and Semantic Hub Detection Algorithms

Statistical Hubness Detector

For each item $\{g_i\}$5, the total hit count $\{g_i\}$6 across all queries is calculated, optionally weighted by retrieval rank or distance. The robust z-score via median and MAD distinctly separates true hubs (with $\{g_i\}$7; $\{g_i\}$8 is a $\{g_i\}$9-in-$M$0 million event under normality).

Cluster Spread Detector

Queries are clustered (MiniBatch KMeans) to detect items evenly retrieved across semantic clusters, a hallmark of universal adversarial hubs. For document $M$1, the normalized entropy of its retrievals over clusters is:

$M$2

where $M$3 is the proportion of hits for cluster $M$4, and $M$5 is the number of clusters.

Stability under Query Perturbation

Top candidates by hubness are stress-tested by re-running retrieval with Gaussian noise added to queries. The stability score:

$M$6

measures robustness; centroid-based "brittle hubs" drop off under perturbations ($M$7), distinguishing them from gradient-optimized hubs.

Deduplication and Boilerplate Suppression

HubScan aggregates near-duplicates by exact text hash, close embedding distance, or detected boilerplate patterns. Large duplicate clusters are downweighted to avoid overcounting synthetic or copy-pasted attacks.

4. Domain-Aware and Modality-Aware Detection Components

Domain-targeted hubs manifest only within specific semantic partitions. HubScan clusters queries into domains (by metadata or embedding), computing per-domain hub rates and a Gini coefficient for domain concentration:

$M$8

Items with high $M$9 or $\{q_j\}$0 are flagged. Modality-aware detection extends this concept to cross-modal settings (e.g., text–image), penalizing items with anomalously high cross-modal hub rates.

5. Integrations with Retrieval Pipelines

HubScan supports the major vector search engines (FAISS, Pinecone, Qdrant, Weaviate) and is compatible with:

• Pure dense-vector search (cosine/inner product)
• Hybrid retrieval (dense + sparse/BM25, keyword overlap)
• Lexical search with cross-encoder reranking (e.g., Passage BERT)

Detectors requiring embedding-based features (e.g., cluster spread) are disabled for pure lexical modes. HubScan measures risk for exactly the retrievals shown to end users, including effects of reranking and hybridization.

6. Empirical Evaluation and Operational Best Practices

HubScan was benchmarked on Food-101 (CLIP-ViT-B/32 embeddings), MS-COCO, and FiQA, with adversarial hubs generated both via gradient-optimization (Zhang et al. objective) and centroid-based approaches. Key findings:

• At alert budget $\{q_j\}$1 ($\{q_j\}$2 flagged per 5,000 items), recall = 90% and precision ≈ 90% for universal attacks; $\{q_j\}$3 yields 100% recall.
• Without cluster spread, hubness-based detection alone attains 40% recall at $\{q_j\}$4, but cluster spread boosts recall by 10–20 percentage points.
• Stability detector is essential for brittle centroid hubs (observed 68% drop in hit rate under $\{q_j\}$5 perturbation).
• Domain-scoped scanning recovers 100% of domain-targeted attacks that evade global detectors.
• On 1M MS MARCO passages, clean 99th percentile score is 2.3 versus 13–17 for adversarial hubs (5.8× separation); 99.9th percentile is 5.0 versus 13–17 (2.7× separation). At $\{q_j\}$6, all top 1,000 clean items score $\{q_j\}$710; adversarial hubs exceed 13.
• Area under the ROC curve (AUC–ROC) stays ≈1.0 with adversarial content ≤2% of the corpus, degrading only above 5%.

Recommended Operational Practices

• Use 50% random queries plus 50% cluster centroids; supplement with real-world queries when available, for $\{q_j\}$8 queries per 5,000-item index.
• Set $\{q_j\}$9 for nearest neighbors.
• Flag items with $i$0 or combined risk above the 99.8th percentile.
• Maintain an alert budget of 0.2%–0.4% per scan to balance recall and review burden.
• Enforce domain-scoped scans when categories are at risk.
• For production at MS MARCO scale, use a combined anomaly score threshold of $i$1 for initial investigation.
• Run HubScan over the same retrieval/ranking pipeline used in end-user deployments.
• Full scans incur $i$2 performance overhead and can be run periodically or pre-indexing for new items.

By adhering to these integration and threshold recommendations, HubScan provides a defense-in-depth mechanism against hubness poisoning in all current RAG operational paradigms (Habler et al., 25 Feb 2026).