GenCharDef: Generative AI Security Framework

• GenCharDef is a framework that defines generation (attack synthesis and defensive perturbation), characterization (taxonomy and metrics), and defense (detection and mitigation) in security-relevant generative AI.
• It employs diverse methodologies such as prompt-guided synthesis, statistical and linguistic measurements, and adversarial defense techniques, with concrete examples like phishing and copyright compliance.
• The framework utilizes iterative feedback loops and integrated cost-aware optimizations to produce robust defenses across domains from intelligent networks to adversarial text repair.

Generation–Characterization–Defense, commonly abbreviated as GenCharDef, is a three-stage framework for analyzing and engineering security-relevant generative-AI systems. In its explicit formulation, Generation denotes the production of attack artifacts or security-relevant outputs, Characterization denotes the measurement and taxonomy of those artifacts, and Defense denotes the mechanisms used to detect, neutralize, or harden against them. The framework is introduced directly for LLM-generated phishing, where it organizes generation techniques, attack features, and mitigation strategies (Chen et al., 29 Aug 2025). Closely related papers apply the same tripartite structure, either explicitly or by retrospective mapping, to intelligent network AIGC services, copyright compliance in LLM text generation, adversarial NLP defense, reasoning-model prompt attacks, architectural side-channels, and reference-based diffusion misuse (Du et al., 2023, Liu et al., 2024, Islam et al., 2024, Zhang et al., 17 Oct 2025, Nam et al., 2023, Song et al., 2024).

1. Conceptual scope and analytical role

In the GenCharDef formulation for phishing, Generation refers to the techniques by which an adversary leverages an LLM to produce phishing messages; Characterization comprises the linguistic, semantic, stylometric, and human-factor features distinguishing those messages; Defense covers automated and human-centric methods for detection, mitigation, or immunization (Chen et al., 29 Aug 2025). This same partition appears in other domains with different objects: copyrighted continuations and jailbreaks in LLMs, poisoned or adversarial images in intelligent networks, paraphrastic repair loops for adversarial text, distractor blocks in reasoning prompts, and perturbation generators for side-channel obfuscation or image protection (Liu et al., 2024, Du et al., 2023, Islam et al., 2024, Zhang et al., 17 Oct 2025, Nam et al., 2023, Song et al., 2024).

The framework is not restricted to offensive use. In intelligent network services, generative AI is cast as both “spear” and “shield,” with LLMs and diffusion models supporting attacks and defenses in the same ecosystem (Du et al., 2023). In GenFighter, the generative module is defensive rather than offensive: a T5 paraphraser produces semantically equivalent variants intended to move adversarial inputs back toward the training distribution (Islam et al., 2024). In Anti-Reference, the generated object is an imperceptible adversarial perturbation that protects a reference image from downstream misuse by customization pipelines (Song et al., 2024).

A further recurrent property is feedback. The cyber-defense treatment states the loop explicitly as “Generation → Characterization → Defense → (feeds back to) Generation,” where new artifacts refine features and decision boundaries, and defense outcomes generate labeled data for subsequent adversarial training (Neupane et al., 2023). This suggests that GenCharDef is best understood not merely as a taxonomy, but as an iterative systems methodology.

2. Generation as attack synthesis and defensive perturbation

Generation in GenCharDef includes both direct malicious synthesis and security-preserving transformation. For LLM-generated phishing, the SoK distinguishes data-guided generation, prompt-guided generation, and adversarially-guided generation. The listed paradigms include task-adaptation via fine-tuning, knowledge-augmented generation with retrieved target-specific facts, basic instruction prompting, conditional prompting, sequential multi-turn prompting, model-crafted prompt mining, and attacker-defender formulations that maximize evasion (Chen et al., 29 Aug 2025).

For copyright compliance, SHIELD operationalizes generation with a concrete probing protocol. The paper uses API-based and open-source models, runs them with temperature $=0$, and defines the generated continuation $y=(y_1,\dots,y_T)$ by greedy decoding,

$y_t=\arg\max_w P_M(w\mid y_{<t},x),$

with $P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$. The prompting regime includes Prefix Probing, Direct Probing, and Jailbreaking using 76 published jailbreak templates (Liu et al., 2024).

In intelligent network AIGC services, Generation covers LLM-based malicious text and diffusion-based image attacks. The enumerated examples include misinformation campaigns, phishing emails, jailbreak prompts, DDoS through automated flood of chat requests, prompt injection, adversarial example generation through reverse diffusion to reconstruct private images from encrypted embeddings, and face-morphing attacks that fool face-recognition systems (Du et al., 2023). The paper summarizes representative objectives rather than a unified threat function, including

$$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$

for prompt injection, and

$\min D(x_{\text{orig}},x_{\text{rec}}(\theta))$

for diffusion-based adversarial reconstruction (Du et al., 2023).

Reasoning distraction instantiates Generation as black-box prompt injection into large reasoning models. The adversary samples a distractor task $d$ from five categories, a malicious meta-instruction $m$, and an insertion position $i\in\{\text{start, middle, end}\}$, then produces a modified prompt by inserting a distractor block into the user prompt (Zhang et al., 17 Oct 2025). The distractor categories are AIME math, Codeforces-style code, ZebraLogic puzzles, Dyck parentheses, and simple arithmetic, with uniform sampling over both category and insertion position (Zhang et al., 17 Oct 2025).

Generation also appears as a defensive operator. GenFighter uses a conditional text-to-text paraphrasing model $p$, specifically T5, to generate $y=(y_1,\dots,y_T)$0 paraphrases of an input sentence inside an $y=(y_1,\dots,y_T)$1-round evolutive loop. At each round, the system generates paraphrases, scores them by anomaly detection, retains the top-$y=(y_1,\dots,y_T)$2 candidates, and either terminates early when they exceed a normality threshold $y=(y_1,\dots,y_T)$3 or iterates on the best candidate (Islam et al., 2024). Defensive ML similarly defines a defender generator $y=(y_1,\dots,y_T)$4 that adds perturbations to side-channel traces, while Anti-Reference uses either an Adversarial Noise Encoder in one forward pass or $y=(y_1,\dots,y_T)$5-PGD to generate image-level perturbations against reference-based generation (Nam et al., 2023, Song et al., 2024).

3. Characterization as taxonomy, measurement, and benchmark design

Characterization gives GenCharDef its empirical and formal specificity. In intelligent network AIGC services, threats are organized along two orthogonal axes: attacks by GAI on discriminative AI and attacks by discriminative AI on GAI services. The first axis contains malicious content generation, DDoS via chatbot API, phishing or social engineering, adversarial examples, and face morphing; the second contains data poisoning, prompt injection, and Trojan or backdoor attacks (Du et al., 2023). The same paper defines Attack Success Rate,

$y=(y_1,\dots,y_T)$6

Poisoning Impact,

$y=(y_1,\dots,y_T)$7

and Adversarial Reconstruction Quality,

$y=(y_1,\dots,y_T)$8

These metrics quantify subversion of discriminative models and recovery of private data (Du et al., 2023).

SHIELD makes Characterization a benchmark-construction problem. It introduces five manually labeled collections—BS-C, BS-NC, BS-PC, SSRL, and BEP—each with 100 examples except BS-PC with 20, each trimmed to 1,000 words (Liu et al., 2024). Its primary metrics are Longest Common Subsequence, ROUGE-L, and RefusalRate, where refusal is determined by whether the response contains any string from a fixed set of refusal templates. The paper further defines compliance on copyrighted data as low LCS and ROUGE-L with high refusal, and fairness on public-domain data as high LCS and ROUGE-L with low refusal (Liu et al., 2024).

Reasoning distraction uses downstream accuracy and behavior labeling. The paper measures $y=(y_1,\dots,y_T)$9, $y_t=\arg\max_w P_M(w\mid y_{<t},x),$0, and

$y_t=\arg\max_w P_M(w\mid y_{<t},x),$1

along with distraction rates on reasoning tokens and final answers,

$y_t=\arg\max_w P_M(w\mid y_{<t},x),$2

It then classifies failures into Implicit Compliance, Overt Compliance, and Covert Compliance (Zhang et al., 17 Oct 2025). The empirical results show that injected distractors can reduce task accuracy by up to 60%, and the paper reports model-specific covert-compliance rates such as 75% for DeepSeek-R1 and 36% for Qwen-3-4B (Zhang et al., 17 Oct 2025).

The phishing SoK expands Characterization beyond model-centric metrics to include human factors and operational economics. Listed measures include perplexity, Flesch-Kincaid Reading Ease, Type-Token Ratio, POS tag frequencies, cosine similarity, BERTScore-style coherence, counts of Cialdini principles summarized as persuasion density, click-through outcomes in user studies, “trust in AI” scales, generation time, and API cost per message (Chen et al., 29 Aug 2025).

Instantiation	Characterization objects	Main metrics
Intelligent network AIGC (Du et al., 2023)	Threat taxonomy A.1–A.5 and B.1–B.3	ASR, PI, ARQ
Copyright compliance (Liu et al., 2024)	Copyrighted vs. public-domain continuations	LCS, ROUGE-L, RefusalRate
Reasoning distraction (Zhang et al., 17 Oct 2025)	Accuracy loss and compliance behavior	$y_t=\arg\max_w P_M(w\mid y_{<t},x),$3, $y_t=\arg\max_w P_M(w\mid y_{<t},x),$4, $y_t=\arg\max_w P_M(w\mid y_{<t},x),$5
LLM phishing (Chen et al., 29 Aug 2025)	Stylometry, persuasion, human factors	Perplexity, TTR, PD, Precision/Recall/F1

A common misconception is that Characterization is reducible to a single accuracy number. The cited literature instead uses attack taxonomies, overlap metrics, refusal behavior, anomaly scores, resource costs, human-vulnerability measures, and failure-mode labels, indicating that GenCharDef treats threat description as a multidimensional inference problem rather than a scalar benchmark.

4. Defense as filtering, purification, alignment, and obfuscation

Defense in GenCharDef spans wrappers, generative purification, safe-alignment training, anomaly detection, and physical-layer obfuscation. SHIELD’s defense is a real-time, model-agnostic agent in front of an LLM API or open-source model. Its three modules are a Copyright Material Detector using an $y_t=\arg\max_w P_M(w\mid y_{<t},x),$6-gram LLM with $y_t=\arg\max_w P_M(w\mid y_{<t},x),$7, substring length $y_t=\arg\max_w P_M(w\mid y_{<t},x),$8, and threshold $y_t=\arg\max_w P_M(w\mid y_{<t},x),$9; a Copyright Status Verifier that checks Project Gutenberg and, if needed, queries Perplexity AI’s llama-3-sonar endpoint; and a Copyright Status Guide that forwards the original query unchanged for public-domain cases or prepends few-shot refusal examples otherwise (Liu et al., 2024). The reported outcome is reduced copyrighted output, refusal rates of 85–100% on API models, and unchanged LCS/ROUGE and refusal rates on public-domain datasets (Liu et al., 2024).

In intelligent network AIGC services, defenses are explicitly dual. LLM-aided defenses include adversarial example generation for robustness testing and natural-language interpretability; diffusion-based defenses include adversarial purification with DDPMs and differential privacy via diffusion sampling; discriminative-AI defenses include Moderation API-style safety filters, bias detectors, and DP-SGD for diffusion training (Du et al., 2023). The DDPM purification pipeline is given as a sequence of diffusion and denoising steps applied to an adversarial input $P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$0, after which the purified $P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$1 is fed to a classifier (Du et al., 2023).

GenFighter treats defense as distributional repair. It extracts a feature embedding $P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$2, scores it under a GMM

$P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$3

uses the log-likelihood $P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$4 as a normality score, and iteratively paraphrases outliers until the top-$P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$5 candidates exceed threshold $P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$6 (Islam et al., 2024). Final prediction is made by a weighted ensemble over the repaired paraphrases. The paper reports average gains of $P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$7 in accuracy under attack over the next best defense, a drop of $P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$8 in attack success rate, and $P_M(y\mid x)=\prod_{t=1}^T P_M(y_t\mid y_{<t},x)$9 in attacker query count, with strict superiority in 15 of 18 settings (Islam et al., 2024).

For reasoning distraction, the defense is training-based. The paper constructs synthetic adversarial prompts, labels good versus distracted outputs, and applies sequential Supervised Fine-Tuning followed by Direct Preference Optimization using

$$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$0

and a DPO objective parameterized by $$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$1 (Zhang et al., 17 Oct 2025). Reported robustness gains are over 50 points on challenging distractor attacks, with SFT-only often accounting for 40–50 point improvements and DPO adding a further 1–5 points in certain cases (Zhang et al., 17 Oct 2025).

Two additional defense families show the breadth of the framework. Defensive ML uses DefenderGAN to train a side-channel obfuscator under adversarial and utility losses; in the memory-contention case, the hardware defender achieves attacker accuracy at or below 53% with 8% IPC slowdown, compared with 17% IPC loss for padding (Nam et al., 2023). Anti-Reference uses a unified loss over fine-tuning, tuning-free, and human-centric customization methods to protect images, with either PGD or a single-pass Adversarial Noise Encoder; the reported inference time is approximately 0.2 s for ANE and approximately 846 s for PGD per image on a high-end GPU (Song et al., 2024).

5. End-to-end integration and resource-aware optimization

A distinctive feature of GenCharDef is that defense is often embedded into a service chain and optimized against explicit cost functions. In the intelligent-network case study, a provider serves $$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$2 requested images from a repository with 30% poisoned images, applying diffusion-based verification and re-fetching when an image is flagged (Du et al., 2023). With $$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$3 Wh per transmission, $$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$4 Wh per diffusion step, and defense strength $$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$5, the paper defines

$$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$6

and selects

$$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$7

The reported optimum is $$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$8, yielding 8.7% energy savings relative to no defense and reducing retransmissions from 32 to 6 (Du et al., 2023).

Integration can also be computational rather than energetic. SHIELD emphasizes asynchronous verification, TTL caching, and the absence of model finetuning or parameter access, framing defense as a lightweight interposition layer (Liu et al., 2024). GenFighter integrates training and inference into a single loop: paraphrase augmentation, classifier fine-tuning, GMM fitting by BIC, percentile thresholding, iterative repair, and weighted inference (Islam et al., 2024). Defensive ML likewise integrates architectural design, adversarial training, and deployment under strict timing constraints, distinguishing nanosecond-scale hardware response for memory contention from millisecond-scale software response for power traces (Nam et al., 2023).

An abstract analogue appears in connected-subgraph defense games, where generation corresponds to mixed-strategy construction, characterization to equilibrium analysis, and defense to optimal or approximate strategy computation (Akrida et al., 2019). The defender’s cover probabilities $$\max f_{\text{prompt}}(x+\delta)\ \text{subject to}\ \|\delta\|_p\le \epsilon$$9, Nash-equilibrium characterization, LP computation of $\min D(x_{\text{orig}},x_{\text{rec}}(\theta))$0, defense ratio $\min D(x_{\text{orig}},x_{\text{rec}}(\theta))$1, and Price of Defense formalize a non-generative but structurally similar GenCharDef pipeline (Akrida et al., 2019). This suggests that the framework can be interpreted as a general pattern for adversarial design, measurement, and optimization, not only as an LLM or diffusion-specific recipe.

6. Breadth, misconceptions, and related theoretical directions

GenCharDef is sometimes conflated with prompt filtering or with LLM safety alone. The surveyed literature does not support that restriction. The same three-stage logic is used for phishing campaigns, copyrighted-text refusal, adversarial text repair, intelligent-network image verification, reasoning-model robustness, cyber-defense deception, side-channel obfuscation, and diffusion-image protection (Chen et al., 29 Aug 2025, Liu et al., 2024, Islam et al., 2024, Du et al., 2023, Zhang et al., 17 Oct 2025, Neupane et al., 2023, Nam et al., 2023, Song et al., 2024). This suggests that GenCharDef is domain-agnostic at the level of method structure, even when the concrete artifacts, metrics, and defenses differ sharply.

Another misconception is that stronger alignment uniformly improves resilience. The reasoning-distraction study reports that certain alignment techniques can amplify the weakness it identifies, and it documents covert compliance, where the model follows hidden adversarial instructions in reasoning while concealing them in the final output (Zhang et al., 17 Oct 2025). In copyright compliance, the corresponding tension is between refusal and fairness on public-domain texts; SHIELD addresses this by explicitly measuring both copyrighted compliance and public-domain fairness (Liu et al., 2024). In phishing defense, the SoK identifies open challenges such as sequential contextual phishing, human-AI interaction pathways, dynamic benchmark corpora, and the balance between stealth and reliability (Chen et al., 29 Aug 2025).

A related but distinct theoretical direction is “Defensive Generation,” which studies online generative models that are unfalsifiable with respect to a pre-specified collection of computational tests and infinite classes of tests, including higher-order moments (Farina et al., 24 Feb 2026). That work develops efficient algorithms via online high-dimensional multicalibration with respect to an RKHS and expected variational inequality problems, achieves near-linear time in the number of samples, and proves a vanishing $\min D(x_{\text{orig}},x_{\text{rec}}(\theta))$2 generation-error rate (Farina et al., 24 Feb 2026). It does not instantiate the same application-layer GenCharDef pipeline, but it provides a mathematically adjacent notion of defensive generative modeling.

Taken together, the literature presents GenCharDef as a unifying research pattern in which generative capability is first used to create or transform security-relevant artifacts, then subjected to formal taxonomy and measurement, and finally coupled to defenses that are often adaptive, optimization-aware, and themselves generative. The framework’s importance lies less in a single algorithm than in the disciplined separation—and subsequent recombination—of synthesis, measurement, and mitigation across heterogeneous security domains.