Maatphor: Automated Prompt Injection Framework

• Maatphor is an automated framework that generates diverse adversarial prompt injection variants to test LLM guardrail vulnerabilities.
• It employs a closed-loop search mechanism with seed extraction, template generation, LLM-driven mutation, and both string matching and embedding-based scoring.
• The framework achieves rapid convergence in high-score variants, making it effective for continuous red-teaming and robust security assessments.

Maatphor is an automated framework for the generation and analysis of prompt injection variants in LLM systems, addressing the challenge of adversarial prompt evasion of guardrails. Unlike reactive or manual approaches, Maatphor introduces a closed-loop variant search mechanism driven by LLMs themselves, enabling both the automated creation of diverse adversarial prompts and black-box evaluation of their effectiveness against arbitrary LLM+guardrail stacks. The system supports extensible evaluation modes and provides a methodology for iteratively discovering, scoring, and cataloging injection variants, facilitating robust security assessment and dataset generation for prompt injection research (Salem et al., 2023).

1. System Architecture

Maatphor’s architecture is organized as a five-component closed loop:

1. Seed & Goal Extractor: Ingests a known prompt injection $P_0$—which may be ineffective—and the original LLM task $T$, extracting the adversary's intent or “real goal” $G$. Extraction can be performed manually or automatically.
2. Template Generator: Produces $N$ test templates from an HTML (or other) base, each embedding a single prompt variant $v$ at one “holder” slot, with others cleared. This supports localized injection and positional evaluation, as detailed in Algorithm 2.
3. Prompt Variation (Variant Generator): Utilizes an LLM (e.g., GPT-3.5-Turbo) with the “Seven Principles of Maat” system prompt. Inputs comprise the seed $P_0$, goal $G$, and the top-$K$ historical variants with scores. This module is responsible for context-switching, rephrasing, creative paraphrase, and command-style synthesis.
4. Victim Model Executor: Feeds each generated template containing variant $v_t$ to the target LLM (potentially via plugin-enabled wrappers such as LangChain + ReadFile/WriteFile), recording the output $o_t$.
5. Prompt Evaluation (Scorer): Employs two black-box evaluation strategies—string-matching and embedding-based similarity—to score each injected variant. After processing, top variants and their scores are looped back to the generator, steering subsequent search.

The following table summarizes the core components:

Component	Input(s)	Output(s)
Seed & Goal Extractor	$T$0, $T$1	$T$2
Template Generator	Template, $T$3, $T$4	$T$5 variant embeds
Prompt Variation	$T$6, $T$7, top-$T$8 history	Candidate $T$9
Victim Model Executor	Injected template, LLM	Output $G$0
Prompt Evaluation	$G$1, (reference sets/$G$2/ $G$3)	Score $G$4

2. Variant Generation Methodology

Maatphor’s core search process is an LLM-driven evolutionary hill-climber, though it avoids explicit genetic algorithm machinery. Each iteration leverages feedback to steer the LLM’s sampling distribution:

• Mutation/Recombination: Implicitly encoded in the prompt context, encouraging context-switching, instruction permutation, paraphrasing, stylistic drift, error simulation, and command-style injection.
• Sampling: Each variant $G$5 is produced by the LLM under temperature 0.5, treating probability as $G$6, with $G$7 the current history.
• Feedback: After evaluation, history $G$8 is updated by aggregating (and sorting) top-scoring variants, retaining the top $G$9 for subsequent prompt context.
• Early Stopping: The process can terminate upon reaching a performance threshold or maximum iteration bound.

The high-level process is:

$P_0$8

3. Variant Effectiveness Scoring

Effectiveness assessment in Maatphor is modular, supporting both symbolic and neural metrics:

1. Exact String Matching: Define $N$0 (must-include phrases) and $N$1 (must-exclude phrases). For each output $N$2 produced by $N$3:

$N$4

The variant’s score is $N$5 over $N$6 trials.

2. Embedding-Based Similarity (kNN): With reference outputs $N$7 (successful class) and $N$8 (unsuccessful), embedded via a model (e.g., MPNet), each output $N$9 is embedded as $v$0. Compute $v$1-nearest neighbors in $v$2:

$v$3

Variant score is again averaged over $v$4 runs. Effectiveness threshold is typically set at $v$5.

Both scorers facilitate continuous or threshold-based measures for injection success.

4. Integration with LLM Guardrails

Maatphor employs a fully black-box interface to evaluated systems, supporting flexible testing of diverse defense mechanisms:

• System Prompt Guardrails: Defensive prompts included during Victim Model Executor invocation.
• Input Classifiers/Sanitizers: If an input classifier filters $v$6, that variant cannot succeed.
• Output Classifiers: Applied in parallel or as replacement scorers on $v$7.
• The framework reports all successful evasions, enabling automated analysis of which variants bypass specific protections, and which layers are robust.

5. Iterative Dynamics and Convergence

Key operational parameters and process characteristics include:

• Variants per Round: 1 by default, parallelizable by drawing multiple LLM samples per iteration.
• Trials per Variant: $v$8 runs to control stochasticity.
• Max Iterations: $v$9 (up to 100 explored for coverage).
• Success Criteria: Max total iterations or attaining a required mean score (e.g., $P_0$0).
• Effectiveness Over Time: Performance tracked via $P_0$1. Empirically, $P_0$2 within 40 iterations on all benchmark tasks.

Performance characteristics indicate rapid convergence to high-efficacy variants even from ineffective seeds.

6. Experimental Evaluation

Maatphor was evaluated on three LLM red-teaming scenarios using GPT-3.5-Turbo with LangChain + AutoGPT and ReadFile/WriteFile plugins on two NVIDIA K80 GPUs. Hyperparameters comprised generation temperature 0.5, agent step bound 15, $P_0$3, $P_0$4, and $P_0$520 reference examples per class.

Task Overview and Outcomes

Task	Seed Success	Top Variant Success	Convergence (Iterations)
Q&A Misinformation (“Did Einstein win...”)	0%	60–72% (string/sim)	30–40
Fraudulent Link (“Embed http...”)	0%	100% (string match)	5–10
Style: Pirate Accent Summarization	0%	64–68% (similarity)	40–50

• Ablation Studies: Disabling feedback significantly reduces the frequency of high-scoring variants. Parallel runs (e.g., 3×100) improve solution diversity but 50 iterations suffices for most cases. Maatphor produces both novel injection types and incremental improvements on strong variants.

7. Operational Recommendations and Applications

Practical insights and guidelines emerging from empirical study include:

• The feedback loop is essential; top-$P_0$6 historical variants focus LLM generation toward effective mutations.
• Maintaining a “hall of fame” (top-$P_0$7 archive) prevents loss of high-performing strategies and promotes search diversity.
• Manual review of top candidates is recommended to detect boundary cases unrecognized by either scorer.
• Maatphor is suitable for integration in continuous red-teaming and CI pipelines, running on each guardrail or model update.
• Variant generation approaches are modular; operators can incorporate stylistic perturbations, suffix injections, or random string manipulations.
• Defense research can invert the framework to optimize system prompts or locked-down plugin environments.
• Plugin-heavy systems should complement linguistic guardrails with strict plugin access controls.

In summary, Maatphor automates the process of (1) generating candidate prompt injections, (2) evaluating them against protected LLM systems, and (3) iteratively steering generation based on empirical evasion success, equipping defenders and researchers to proactively discover and evaluate variants that bypass contemporary prompt injection mitigations (Salem et al., 2023).