GANGRL-LLM: GAN & LLM for Robust Cyber Defense

• GANGRL-LLM is a semi-supervised framework combining a GAN-based discriminator and an LLM-based generator to improve malicious code recognition.
• It employs reward-guided iterative training with adaptive decay to generate high-quality malicious code even with limited labeled examples.
• Its synthetic code generation boosts IDS training, achieving improved accuracy and detection metrics across various cybersecurity applications.

GANGRL-LLM is a semi-supervised learning framework that integrates Generative Adversarial Networks (GANs) with LLMs to enhance both the generation and detection of malicious code in settings where labeled examples are scarce. The framework is specifically designed for applications in network security, such as SQL injection (SQLi) generation and Intrusion Detection System (IDS) training, and is structured to leverage collaborative adversarial learning between a code generator (LLM) and a discriminator (GAN-based), thus addressing data scarcity and improving system robustness against evolving cyber threats (Ma et al., 25 Aug 2025).

1. Architectural Composition and Objectives

GANGRL-LLM operates with two central components:

• GAN-Based Discriminator: This module incorporates a BERT-like LLM tailored for code and two multi-layer perceptrons (MLPs)—a code word vector distribution simulator and a code type classifier. Its primary function is to distinguish real, manually labeled malicious samples from synthetically generated ones, thereby improving malicious pattern recognition.
• LLM-Based Generator: Utilizing a pre-trained code model (e.g., Qwen2.5Coder), the generator produces malicious code snippets based on prompts, with the generation refined by reward signals derived from the discriminator’s output.

The framework is designed for iterative, collaborative training in which the generator synthesizes candidate malicious code; the discriminator then evaluates these samples to provide reward feedback, thereby guiding both components toward mutually improved performance.

2. GAN Component: Discriminator and Simulator Mechanisms

The discriminator architecture integrates:

• a BERT-like code encoder that represents input samples (real or simulated) into latent vector space,
• an MLP-based word vector simulator to model the distribution of genuine code representations via noise-driven latent vectors,
• a code type classifier distinguishing among $k$ real code classes plus an additional “fake” class.

Loss functions employed include:

• Supervised loss for manually labeled samples:

$$L_{(C)_{sup}} = -\mathbb{E}_{(x, y) \sim p_c} [\log C(y|x)]$$

• Unsupervised loss for simulated/fake samples:

$$L_{(C)_{unsup}} = -\mathbb{E}_{x\sim p_c}[\log(1-C(y=k+1|x))] - \mathbb{E}_{x\sim p_s}[\log C(y=k+1|x)]$$

where $C$ is the classifier probability over classes, $p_c$ is the distribution of labeled data, and $p_s$ is the simulator’s distribution.

• Simulator adversarial and feature matching loss:

$$L_{S} = -\mathbb{E}_{x_s\sim p_s} \log(1-C(y=k+1|x_s)) + \lambda \mathbb{E}[\|\mu_{real} - \mu_{fake}\|_2^2]$$

$\mu_{real}$ and $\mu_{fake}$ are mean representations from the classifier’s intermediate layers; $\lambda$ balances diversity in feature distributions.

This structure enhances the capability of recognizing subtle, high-fidelity malicious code patterns with minimal labeled data.

3. LLM Component: Generator and Reward-Guided Training

The LLM generator generates code sequences $y$ for a given prompt $x$:

$$y \sim P_\theta(y|x) = \prod_{t=1}^{T} P_\theta(y_t | y_{<t}, x)$$

Training leverages both the standard maximum likelihood objective and reward signals from the discriminator:

• Reward signal:

$r(y_{gen}) = D(y=1 | y_{gen})$

where $D(y=1|\cdot)$ is discriminator-assigned probability of code being malicious.

• Generator loss: Combines policy gradient and cross-entropy:

$$\text{Loss}_{\text{gen}} = \text{CrossEntropy} + \lambda \cdot r(y_{gen})$$

An adaptive reward weight $\lambda(t) = \alpha \cdot \theta^{t/T}$ is decayed over epochs, modulating reliance on discriminator feedback during training.

The result is targeted synthesis of higher-quality, more realistic malicious code even in few-shot labeled settings.

4. Collaborative Training Dynamics

The central innovation of GANGRL-LLM is its joint optimization scheme:

• Iterative process:
  • The generator creates batches of malicious code.
  • The discriminator evaluates these samples alongside labeled data, updating via supervised and unsupervised loss components.
  • The generator receives composite loss information (standard plus policy gradient from discriminator) and adjusts output accordingly.
  • Reward weight decay ensures dynamic transition from adversarial guidance to stability-focused learning.

This paradigm prevents mode collapse and overfitting, maintaining diversity and authenticity in both generated samples and detection capabilities.

5. Empirical Results and Benchmarks

Experimental analysis demonstrates several outcomes:

• Quality of generated malcode: Under training with 1000 samples, Qwen2.5Coder trained using GANGRL-LLM produces generation scores of approximately 5.74 (out of 10), compared to ~5.275 for conventionally fine-tuned models.
• Component ablation: Removal of discriminator, simulator, or adaptive reward yields substantial drops in generation and detection performance, affirming each module’s necessity.
• Transferability: Effectiveness persists when adopting other LLM bases, such as Llama 3.2, and across different malicious code domains (e.g., Cross-Site Scripting).
• IDS augmentation: Incorporation of generated samples into IDS training datasets leads to measurable improvements in classic classifiers (CNN, SVM, Decision Trees) across accuracy, precision, and recall metrics.

6. Domain Impact and Applications

GANGRL-LLM delivers several implications for cybersecurity and adaptive defense:

• Alleviation of labeled data scarcity: High-quality synthetic malcode generation aids IDS model training in data-sparse contexts.
• Adaptive and robust detection: Enhanced model robustness against novel and evolving attack patterns via adversarial refinement.
• Realistic simulation and honeypot environments: The generator may be employed to create authentic adversarial scenarios for penetration testing and system defense calibration.
• Broader semi-supervised blueprint: The collaborative paradigm offers a transferable methodology for rare-sample domains beyond cybersecurity, such as fraud detection and anomaly identification.

7. Summary and Future Directions

By fusing a GAN-based discriminator with an LLM-based generator in a unified, semi-supervised adversarial framework, GANGRL-LLM advances both code generation and malicious pattern recognition for network security. The architecture’s reward-guided interaction and simulator-classifier split yield higher code authenticity and detection performance in labeled-data scarce environments. A plausible implication is that similar collaborative adversarial structures may be adapted to other domains where data scarcity and the need for authentic synthetic sample generation are critical. The success in augmenting IDS training and enhancing detection metrics marks GANGRL-LLM as a technically significant contribution to the arsenal of adaptive defense methodologies in cybersecurity.