Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings

Abstract: LLMs in real-world applications often face the risks of specially crafted prompts designed to bypass the safety controls. Existing guardrail methods, such as LLM-as-a-judge and cloud-based safety APIs are able to detect unsafe content. However, they often add a delay of about 250-900 ms to each request. This delay is too high for real-time applications, when the system usually needs to respond in less than 100 ms. Furthermore, routing user prompts through external moderation endpoints raises significant data privacy concerns. This paper introduces Reflex-Guard, a lightweight guardrail that runs locally. It uses jailbreak-aware preprocessing, compact sentence-transformer embeddings, and seven fast binary classifiers. Together, these components enable high-accuracy prompt safety filtering with much lower latency than existing solutions. Through systematic evaluation on a strategically balanced dataset of 30,568 samples drawn from five complementary sources, we demonstrate that Reflex-Guard achieves 95.9% recall on harmful prompts at 37.6 ms end-to-end latency. It is faster than existing baselines, including Llama Guard 2 at 255 ms and SafeDecoding at 723 ms. It can detect 100% of GCG suffix attacks and Base64-encoded prompts using the default threshold. However, DrAttack structured prompts required lowering the threshold to 0.03 for optimal detection, as they produced a distinct probability distribution. Reflex-Guard achieves Reflex Efficiency Score (RES) scores up to 16.79, significantly outperforming Llama Guard 2 (11.90) and SafeDecoding (9.80). This analysis offers practical deployment advice and shows that different attack types occupy distinct regions in the embedding probability space.

• The paper implements a fully local, low-latency pipeline that uses dense semantic embeddings to detect and flag potentially malicious LLM prompts at sub-100ms latency with recall rates of 95.9% in in-distribution evaluation.
• Prompt preprocessing is deemed necessary to detect and decode encoded attacks, improvement in the distinction of harmless and harmful prompts can be automatic by embedding these semantic cues in the base embedding space
• Several lightweight classification methods are evaluated (e.g., Logistic Regression, LightGBM) for efficiently determining the malicious likelihood of prompts, and adaptive thresholding strategies are recommended for better robustness against structured attacks.

Motivation and problem statement

Reflex-Guard addresses the latency–accuracy–privacy trilemma in LLM input moderation. Existing guardrails—LLM-as-a-judge systems such as Llama Guard 2 (255–400 ms) and SafeDecoding (300–723 ms), or cloud moderation APIs (100–300 ms network overhead)—cannot meet the sub-100 ms budget required by real-time conversational applications, and cloud routing exposes user prompts to third parties. The paper proposes a fully local pipeline: jailbreak-aware preprocessing, a compact BGE-small sentence encoder (33M parameters, 384-dimensional L2-normalized embeddings), and one of seven lightweight binary classifiers with threshold-based decisions. A notable contribution is the Reflex Efficiency Score (RES), defined as $$\mathrm{RES} = \mathrm{Recall} \times 100 / \log_2(\mathrm{Latency}+1)$$, which unifies detection effectiveness and computational cost into a single comparable metric.

Architecture

The pipeline has three stages. Jailbreak-aware preprocessing (<0.1 ms overhead) detects Base64 payloads via a sliding-window indicator over valid Base64 character patterns, decodes them, and prepends a "DECODED" marker; it flags GCG-style suffix noise (retained rather than removed, since the BGE encoder tolerates it and its presence is itself adversarial evidence); and triggers sensitivity flags on DrAttack-like structural templates ("As a researcher", "For educational purposes"). Embedding applies mean pooling over token states followed by L2 normalization, so classification operates on angular similarity; the "query:" prefix follows BGE's training convention. This stage dominates the latency floor at 35.27 ms per request on a Tesla T4. Classification compares seven architectures (Logistic Regression, XGBoost, LightGBM, HistGradientBoosting, Random Forest, AdaBoost, KNN), each emitting a harm probability $p$ compared against a default threshold $\tau = 0.25$.

Evaluation setup

The benchmark comprises 30,568 prompts: 15,568 harmful from Anthropic Red-Team (training, undersampled from 160,583), JailbreakBench, JailbreakHub/DAN (MinHash-deduplicated to 968), and MultiJail Bengali (oversampled for multilingual evasion), plus 15,000 benign AlpacaEval prompts. Training uses only Anthropic Red-Team harmful data; JailbreakBench, JailbreakHub, and MultiJail are held out entirely for out-of-distribution evaluation. Experiments run on commodity hardware (T4 GPU, dual-core Xeon CPU) with pinned dependencies and fixed seeds; latency is reported as mean/median/P99 over 100 prompts after warm-up.

Results

In-distribution performance. HistGradientBoosting leads with F1 = 0.955, recall = 95.9%, precision = 95.2%, ROC-AUC = 0.990; LightGBM is nearly identical (F1 = 0.954). Logistic Regression reaches F1 = 0.942 using raw embeddings directly, indicating that harmful and benign prompts are already well separated in the BGE embedding space—a strong claim about the representation itself rather than the classifier. KNN shows the highest precision (96.3%) but the lowest recall (86.0%).

Out-of-distribution robustness. Detection is tri-modal across attack families:

Attack	Samples	Recall ($\tau=0.25$)	Recall ($\tau=0.03$)	Mean prob.
GCG suffix	60	100%	100%	0.79
Base64 encoded	100	100%	100%	0.63
DrAttack structured	60	0%	100%	0.05

Two findings stand out. First, without Base64 decoding, recall collapses from 100% to 7% (mean probability 0.09), confirming that encoded prompts are semantically indistinguishable from benign text in embedding space absent preprocessing. Second, DrAttack prompts form a tightly clustered distribution (mean 0.05, standard deviation 0.008) that falls entirely below the default threshold: detection drops from 100% at $\tau=0.03$ to 75% at 0.04 and 0% at $\tau \geq 0.07$. Lowering the threshold to 0.03 introduces zero false positives among 2,000 benign samples (benign maximum probability 0.027), because benign probabilities concentrate near zero (mean 0.0021). This sharp cliff motivates an adaptive, attack-aware thresholding strategy rather than a single global operating point—an honest concession that the default configuration fails completely on structured attacks.

Latency. End-to-end means range from 53.17 ms (Logistic Regression) to 193.97 ms (Random Forest). LightGBM has the lowest component latency (37.59 ms total). Tail latencies are nontrivial: P99 values reach 284–540 ms even for fast classifiers, reflecting GPU scheduling jitter—the paper reports these transparently rather than only medians.

Efficiency comparison. All Reflex-Guard configurations outperform baselines on RES: Logistic Regression achieves 16.79 and HistGradientBoosting 16.67, versus 11.90 for Llama Guard 2 and 9.80 for SafeDecoding—improvements of 41% and 71% respectively. HistGradientBoosting buys 1.16 percentage points of recall over Logistic Regression for only 3.95 ms additional latency, placing both on the security–latency Pareto frontier while LightGBM is dominated.

Limitations

The paper concedes several constraints that qualify the headline numbers. Baseline comparisons use latencies reported in prior work rather than measurements under identical hardware, so the RES gap, while large, is not apples-to-apples. The threat model excludes white-box adversaries who know the embedding model or classifier; adaptive attacks targeting the embedding space are untested. Multilingual coverage is limited to preliminary Bengali evidence. Other attack classes—multi-turn jailbreaks, indirect prompt injection, paraphrasing, translation-based evasion—are explicitly out of scope. The dataset, while balanced and multi-source, may not reflect production traffic diversity, and the harmful training signal derives predominantly from a single source (Anthropic Red-Team).

Conclusion

Reflex-Guard demonstrates that a 33M-parameter embedding model paired with lightweight classifiers can match the recall of LLM-based guardrails (95.9% vs. ~95%) at roughly 5–7× lower latency while keeping all inference local. Its most instructive empirical finding is the trimodal separation of attack families in probability space, which both exposes the failure of fixed thresholds against structured attacks and suggests a concrete remedy via preprocessing-triggered adaptive thresholds. Open questions include robustness under white-box attack, generalization beyond English, and principled automatic threshold calibration.