Efficient and Adaptable Detection of Malicious LLM Prompts via Bootstrap Aggregation

Abstract: LLMs have demonstrated remarkable capabilities in natural language understanding, reasoning, and generation. However, these systems remain susceptible to malicious prompts that induce unsafe or policy-violating behavior through harmful requests, jailbreak techniques, and prompt injection attacks. Existing defenses face fundamental limitations: black-box moderation APIs offer limited transparency and adapt poorly to evolving threats, while white-box approaches using large LLM judges impose prohibitive computational costs and require expensive retraining for new attacks. Current systems force designers to choose between performance, efficiency, and adaptability. To address these challenges, we present BAGEL (Bootstrap AGgregated Ensemble Layer), a modular, lightweight, and incrementally updatable framework for malicious prompt detection. BAGEL employs a bootstrap aggregation and mixture of expert inspired ensemble of fine-tuned models, each specialized on a different attack dataset. At inference, BAGEL uses a random forest router to identify the most suitable ensemble member, then applies stochastic selection to sample additional members for prediction aggregation. When new attacks emerge, BAGEL updates incrementally by fine-tuning a small prompt-safety classifier (86M parameters) and adding the resulting model to the ensemble. BAGEL achieves an F1 score of 0.92 by selecting just 5 ensemble members (430M parameters), outperforming OpenAI Moderation API and ShieldGemma which require billions of parameters. Performance remains robust after nine incremental updates, and BAGEL provides interpretability through its router's structural features. Our results show ensembles of small finetuned classifiers can match or exceed billion-parameter guardrails while offering the adaptability and efficiency required for production systems.

• The paper introduces BAGEL, a routed and stochastic ensemble of specialized 86M-parameter Prompt Guard classifiers that detects harmful requests, jailbreaks, and prompt injections.
• BAGEL reaches 0.922 F1 with five of nine ensemble members, reducing inference compute by about 45% while balancing a 0.095 attack success rate and 0.066 false-positive rate.
• The framework maintains F1 above 0.92 across nine simulated dataset updates, but remains limited by single-turn evaluation, binary labels, and the need for sufficient data on emerging attacks.

Overview

BAGEL (Bootstrap AGgregated Ensemble Layer) is a framework for detecting malicious LLM prompts that departs from the prevailing trend of scaling monolithic safety models. Instead, it composes an ensemble of small fine-tuned binary classifiers — each an 86M-parameter instance of Meta's Prompt Guard 2, termed a "promptcop" — where each member is specialized on a distinct attack dataset. The design modifies two classical techniques: bootstrap aggregation (bagging), in which members are trained on entirely different datasets rather than resampled subsets of one dataset, and mixture-of-experts routing, in which a random forest router selects a predicted ideal member plus stochastic additional members rather than a single expert. The central claim is that malicious prompt detection should be modular, lightweight, and incrementally updatable, and that such an ensemble can match or exceed billion-parameter guardrails at a fraction of the cost.

The paper targets three attack categories: simple harmful requests with explicit intent, jailbreaks that manipulate the instruction hierarchy through role-play or obfuscation, and prompt injections that interleave benign and malicious substrings. The threat model assumes remote attackers without access to system internals, spanning novice to expert adversaries, and restricts the defender's task to binary classification (benign vs. malicious).

Architecture

Each incoming dataset $D_i$ is partitioned into a 70% fine-tuning set, a 10% calibration set, and a 20% held-out test set. A promptcop $M_i$ is created by fine-tuning Prompt Guard 2 on $D_i^{train}$ using its energy-based loss. Calibration subsets are accumulated into a global calibration set $\mathcal{C}_{global}$, which serves two purposes: training the router and re-calibrating the decision threshold after each ensemble update.

At inference, a random forest $\mathcal{R}$ trained on nine lightweight structural features (prompt length, whitespace/special-character/digit/uppercase proportions, average word length, code-keyword count, natural-language word count, and Shannon entropy) predicts the index $i^*$ of the ideal promptcop. The selection subset $S_x$ comprises $M_{i^*}$ plus $n-1$ uniformly sampled members; their probability outputs are averaged and compared against a threshold $\tau$. The threshold is recalibrated via a coarse-to-fine search over roughly 20 evaluations, assuming quasi-normal F1 behavior around the optimum.

This hybrid routing scheme is presented as "safety in depth": when the router errs (~80% accuracy), randomly selected peers compensate; conversely, even a perfectly known ideal promptcop alone yields high ASR, so aggregation remains necessary regardless of routing quality.

Evaluation

Experiments use nine public datasets totaling 839,140 samples covering simple harmful, jailbreak, and injection attacks ($M_i$0), evaluated on a combined test set of 167,828 unseen samples.

Selection efficiency. With $M_i$1, performance saturates around $M_i$2: increasing from $M_i$3 to $M_i$4 only reduces ASR from 0.096 to 0.080 and FPR from 0.067 to 0.051. Setting $M_i$5 therefore cuts inference compute by approximately 45% relative to the full ensemble while retaining near-optimal performance. The random-forest strategy tracks the oracle ("ideal") strategy closely despite imperfect routing, indicating that stochastic aggregation absorbs router errors.

Adaptability. Simulating temporal arrival of new attack datasets by growing $M_i$6 from 3 to 9, BAGEL maintains F1 above 0.92 across all nine incremental updates. For most intermediate $M_i$7, near-ideal performance is achievable at $M_i$8 alone; the final, largest dataset disrupts single-model routing, but bagging stabilizes performance — evidence that aggregation provides robustness when incoming data diverges substantially from prior distributions.

Interpretability. Spearman correlation analysis plus Ward-linkage hierarchical clustering reveals redundant feature clusters; pruning from 9 to 5 features drops router accuracy only marginally, from 0.806 to 0.794. This confirms that structural signals such as uppercase ratio and special-character proportion carry most of the discriminative information, and it further lightens the router.

Comparative benchmarking. Against five baselines on the full test corpus:

Method	ASR	FPR	F1	Params
BAGEL ($M_i$9, $D_i^{train}$0)	0.095	0.066	0.922	430M effective
ToxicDetector	0.045	0.326	0.847	300M + 7B
ShieldGemma (2B)	0.624	0.038	0.534	2B
OpenAI Moderation API	0.881	0.024	0.208	unknown
Perspective API	0.569	0.068	0.642	unknown
LastLayer	0.598	0.171	0.519	n/a

BAGEL achieves the highest F1 by balancing ASR and FPR, whereas competitors are skewed: ShieldGemma and OpenAI Moderation have low FPR but miss most attacks (ASR of 0.624 and 0.881 respectively), while ToxicDetector attains the lowest ASR at the cost of flagging a third of benign prompts. The authors attribute API failures to toxicity-oriented optimization, which polite role-play jailbreaks evade. Notably, OpenAI Moderation's F1 of 0.208 on this corpus is a strong claim about widely deployed infrastructure, though it reflects the benchmark's composition of jailbreak and injection attacks rather than toxicity categories the API was designed for.

Limitations

The authors identify several constraints. First, BAGEL performs binary classification only; deployments requiring policy-category attribution would need a secondary fine-grained classifier. Second, evaluation covers exclusively single-turn prompts, leaving multi-turn conversational attacks untested. Third, incremental adaptation depends on Prompt Guard 2's pre-training having covered jailbreak and injection classes; whether fine-tuning suffices for fundamentally novel attack paradigms remains open. Fourth, each new promptcop requires a sufficiently large labeled dataset, creating a detection gap between a novel attack's emergence and data curation. Additionally, the adaptability experiments simulate dataset arrival in a fixed order, and the comparative results depend on the specific nine-dataset corpus; generalization to other threat mixtures is not established.

Conclusion

BAGEL demonstrates that an ensemble of specialized 86M-parameter classifiers with random-forest routing and stochastic subset aggregation achieves an F1 of 0.92 with 430M effective parameters, outperforming billion-parameter guardrails and commercial moderation APIs on a diverse malicious-prompt benchmark, while supporting incremental updates whose F1 never falls below 0.92 across nine additions. Its main contribution is architectural: decoupling detection quality from model scale and enabling per-dataset specialization with cheap updates. The open questions it leaves — multi-turn moderation, robustness to genuinely novel attack paradigms, and few-shot handling of emerging threats before sufficient data exists — define the boundary conditions of its applicability.