Gate AI: LLM Security Benchmark Evaluation Methodology and Results

Abstract: Published evaluations of prompt-injection and jailbreak detectors for LLMs often suffer from two systematic weaknesses: per-dataset threshold tuning and undisclosed operating points. We describe an evaluation harness that addresses both. The detector under evaluation is scored across 16 public benchmarks (12,111 samples) using 5-fold cross-validation. StratifiedKFold (by row) is the headline pass; a parallel StratifiedGroupKFold pass over a composite key (parent-prompt id plus MinHash + LSH near-duplicate clusters at Jaccard $\gtrsim 0.8$) runs alongside it as a leakage-premium diagnostic. A single global operating point is selected on the held-out folds (max F1 subject to FPR $\leq 1\%$) and applied uniformly to every dataset, so per-dataset results reflect one threshold rather than per-benchmark optimisation. Generalisation is examined through a battery of diagnostics (leave-one-dataset-out cross-validation, a random-label control, adversarial validation, permutation feature importance, length-bias correlation, classifier-head agreement, cross-source near-duplicate detection, threshold transferability, train-vs-OOF agreement, and a paraphrase-invariance probe), most with a quantitative pass threshold and the remainder with a stated failure mode. For every external comparison, the detector's threshold is re-tuned to the competitor's published false-positive rate so head-to-head values are evaluated at matched operating points.

• The paper introduces a reproducible global thresholding protocol that standardizes prompt injection detection across 16 text benchmarks.
• It employs multi-level cross-validation and rigorous statistical diagnostics, including 95% bootstrap confidence intervals, to ensure robust performance.
• Results show state-of-the-art micro-F1 of 97.4% and a median latency of 53 ms, outperforming competitors on security and speed.

Gate AI: Comprehensive Evaluation Methodology and Results for Prompt Injection Detection

Motivation and Problem Statement

The deployment of agentic LLM-based assistants with read/write access to trusted user data raises substantial risks from prompt injection and jailbreak attacks. Inputs submitted by an adversary, even as a benign email or embedded content, can trigger undesired model behaviors such as unauthorized message dispatch, data exfiltration, or arbitrary tool invocation. Addressing these risks requires concrete, reproducible evaluation protocols for detectors that identify and filter such attacks—in particular, to mitigate the prevalence of ad hoc datasets, per-dataset threshold tuning, and undisclosed operating points that confound cross-system comparison.

Methodological Innovations

Gate AI introduces a rigorously auditable evaluation architecture for LLM security detectors, designed to enforce methodological transparency and statistical validity.

• Dataset Corpus and Trace Handling: 16 public benchmarks (12,111 samples) are combined through hash-stable content loading with fixed sample caps, stratified by attack family and supporting direct, indirect, synthetic, and over-defense (false-positive) stress-testing.
• Multi-level Cross-validation Protocols: Out-of-fold metrics are computed via 5-fold StratifiedKFold, ensuring class-consistent splits. Parallel diagnostic runs employ StratifiedGroupKFold with composite keys (parent-prompt ID + MinHash/LSH clusters at Jaccard $\geq 0.8$) to detect and quantify near-duplicate and sibling-chunk leakage, isolating overestimated performance otherwise introduced by overlooked sample overlap.

  Figure 1: Per-fold ROC overlay for the headline 5-fold cross-validation, confirming stability and absence of leakage between folds.

• Global Operating Point Selection: Detector thresholds are not tuned per dataset. A single global threshold $\tilde\theta$ is selected based on maximizing $F_1$ under a constraint of FPR $\leq$ 1% on held-out validation slices. This operating point is then uniformly applied to all benchmarks, yielding system-wide, reproducible statistics instead of per-dataset optimal values.
• Validation Diagnostics: The evaluation harness performs adversarial validation (confirming no distributional shift between train and test partitions), per-fold threshold stability checks, train-vs-OOF agreement, and threshold transferability analysis. Generalization stress tests include leave-one-dataset-out (LODO) experiments, random-label controls, permutation-based feature importance, classifier-head agreement measurement (Cohen's $\kappa$), and length-bias correlation.

  Figure 2: Adversarial-validation AUC per fold, targeting approximately 0.5 to ensure distributional consistency between splits.

  

  Figure 3: Per-fold train vs OOF F1, used to detect overfitting or state leakage across the pipeline.

Aggregation and Metric Reporting

• Micro vs. Macro Aggregation: The system distinguishes between pooled metrics (micro) and unweighted per-source means (macro), surfacing discrepancies that point to source size imbalance or dataset-specific performance skews.

  Figure 4: Cascade F1 vs FPR sweep, plotting micro and macro aggregation on the same axes; large inter-curve gaps reveal source-dominated contributions.

• Matched-FPR Comparison Protocols: For head-to-head competitor evaluation, Gate’s threshold is realigned per dataset to the published FPR of the comparator. Where only a primary metric is published, surrogate matching is used.
• Bootstrap Confidence Intervals: Reported metrics (F1, precision, recall, FPR, AUC) are provided with 95% stratified bootstrap CIs, ensuring all differences bracket statistical noise and precluding overinterpretation of nominal wins in small-$n$ regimes.

Generalization and Diagnostic Results

• Leakage Controls: The empirical $\Delta F_1$ (StratifiedKFold minus StratifiedGroupKFold) is reported per release, quantifying any residual leakage premium.
• Random-Label Controls: Post-shuffling, metrics collapse to the expected random baseline (AUC $\approx$ 0.51, F1 $\approx$ 84.4%), providing empirical evidence of leak-robust evaluation.

  Figure 5: Random-label control; F1 scores against shuffled targets align with theoretical chance baseline, confirming lack of row-identity leakage.

• Length-Bias and Feature Importance: Pearson correlation between input length and model scores remains $|\rho| < 0.3$, negating reliance on trivial proxies. Permutation importance analysis identifies load-bearing features robustly.

  Figure 6: Length-bias Pearson correlation, overall and per-source, confirming the detector does not reduce to length heuristics.

• Classifier Ensemble Analysis: Pairwise Cohen’s $\tilde\theta$0 in the ensemble head predictions fall in $\tilde\theta$1, indicating non-redundant, independent signals.

  Figure 7: Pairwise Cohen's $\tilde\theta$2 between ensemble heads, confirming that individual classifiers contribute distinct information.

Benchmark and Numerical Results

• Aggregate Performance: At the headline operating point (FPR $\tilde\theta$3 1%), Gate achieves a pooled micro-F1 of 97.4% (macro 95.7%); at the unconstrained threshold (max-F1, $\tilde\theta$4), micro-F1 reaches 98.7% (macro 98.4%).
• Per-dataset Outcomes: Gate ranks first on 8 out of 16 datasets by primary metric, second or third on most others, with statistical ties to top competitors within reported CIs.
• Head-to-head with Lakera Guard: On every dataset where a true matched-FPR comparison is possible, Gate matches or exceeds Lakera’s performance at the same operating point, with the most significant improvements on multi-attack and macro-aggregate benchmarks.

  Figure 8: Gate vs Lakera Guard per-dataset values at matched FPR, illustrating consistent parity or improvement across all reported splits.

  

  Figure 9: Detection Error Tradeoff (DET) curve on probit axes, highlighting regime differentiation at low FPR crucial for deployment scenarios.

Latency and Deployment Considerations

• Latency Distribution: Median end-to-end latency is 53 ms, mean 104 ms, with p95 at 571 ms. This positions the cascade as the fastest detector among published competitors, with substantial margin over the next best (Lakera Guard mean 140 ms, range 51–305 ms) and orders-of-magnitude over slower approaches (e.g., Rebuff mean 23.9 s).

  Figure 10: End-to-end detection latency, Gate (blue) vs competitors (grey); log-scale spans two orders of magnitude across systems.

  

  Figure 11: Latency density distribution, with majority of requests handled in the fast-mode cluster (<100 ms).

Limitations and Practical Scope

The evaluation is limited to public text-based benchmarks, does not account for pixel/image-payload prompts, voice-driven or long-context scenarios, non-English attacks beyond benchmarked languages, or code-execution intent. Pretraining contamination is recognized as a critical open problem, with most datasets indexed by modern foundation models; purification via post-training attack synthesis is ongoing research.

The analysis framework is robust against implicit data leakage via chunked and near-duplicate samples but can be subverted by aggressive paraphrasing or encoding attacks not captured by MinHash/LSH clustering. All external comparisons are bounded by the latest available third-party metrics, and bootstrap intervals are reported for sampling parity.

Implications and Future Perspectives

From a practical perspective, Gate's robust global-threshold methodology addresses a key barrier to defensibility and reproducibility in LLM security detection. Enforcing per-system fixed operating points, matched-FPR head-to-head metrics, and comprehensive diagnostic reporting sets a new standard for evaluation rigor. The framework's extensibility supports ongoing additions as state-of-the-art benchmarks and attacks evolve. Theoretical implications include the demonstration that cascade-ensemble defenses, when evaluated auditably, generalize strongly across heterogeneous prompt distributions and resist overfitting even in adversarial or underdefined problem regimes.

Future work must focus on integrating semantic-near-duplicate detection beyond MinHash-scale shingling, expanding to non-text modalities, and sourcing benchmarks resistant to pretraining contamination. The field also calls for adversarially adaptive evaluation suites to further stress-test detector robustness against white-box attacks.

Conclusion

Gate AI constitutes a high-integrity, comprehensive methodology for benchmarking LLM prompt-injection and jailbreak detectors. The adoption of a cascade-ensemble approach combined with strict global operating point selection yields state-of-the-art performance across diverse public benchmarks, with robust statistical controls and unmatched latency. The evaluation architecture and associated results provide a reproducible reference model for LLM security detector validation, serving as a platform for future methodological and algorithmic advances in robust, real-world prompt injection defense systems.