Attention Adversarial Dual AutoEncoder

• The paper introduces ADAEN, a hybrid framework combining dual autoencoder architectures with an attention layer to prioritize salient features in anomaly detection.
• It integrates adversarial training with ranking-based prioritization and active learning, using GAN-generated augmentations to reduce labeled data requirements.
• Empirical results demonstrate significant improvements in anomaly scoring and ranking across diverse operating systems, highlighting its applicability in real-world security contexts.

The Attention Adversarial Dual AutoEncoder (ADAEN) is a hybrid neural anomaly detection framework introduced for highly imbalanced detection tasks such as advanced persistent threat (APT) identification in security provenance trace data. ADAEN integrates dual autoencoder architectures, an attention mechanism, adversarial learning, ranking-based prioritization, and a data-efficient active learning loop with generative augmentation to achieve superior anomaly prioritization with minimal labeled data requirements (Benabderrahmane et al., 25 Nov 2025).

1. Dual AutoEncoder Architecture

ADAEN consists of two feed-forward autoencoders (AEs), denoted AE₁ and AE₂. AE₁ functions as a generator focusing on capturing core data characteristics, while AE₂ serves as a complementary refiner/discriminator, promoting distinct but compatible latent representations. Formally, let $\mathcal{X} = \{x^{(i)}\}_{i=1}^m$ with $x^{(i)}\in\mathbb{R}^d$. Each AE maps $x\in\mathbb{R}^d$ to a latent code $z\in\mathbb{R}^k$ and reconstructs it:

$$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$

$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$

Each network uses LeakyReLU activations, batch normalization, and dropout. The loss for each AE is its mean squared reconstruction error:

$$\mathcal{L}_{rec_j} = \frac{1}{|\mathcal{X}|} \sum_{x \in \mathcal{X}} \| x - g_{\phi_j}(f_{\theta_j}(x)) \|^2_2$$

Total reconstruction loss is a weighted combination:

$$\mathcal{L}_{rec} = \alpha \mathcal{L}_{rec_1} + (1-\alpha)\mathcal{L}_{rec_2}, \quad \alpha=0.5$$

2. Attention Mechanism

ADAEN incorporates an attention layer between AE₁’s encoder and decoder to enhance focus on salient features or temporal slices of the latent code, $z_1\in\mathbb{R}^{T \times k}$. For each slice $h_i\in\mathbb{R}^k$ and context vector $x^{(i)}\in\mathbb{R}^d$0, compute attention weights:

$x^{(i)}\in\mathbb{R}^d$1

The attended summary $x^{(i)}\in\mathbb{R}^d$2 is:

$x^{(i)}\in\mathbb{R}^d$3

This vector $x^{(i)}\in\mathbb{R}^d$4 is input to the decoder, enabling adaptive emphasis on features relevant for reconstruction, and consequently anomaly discrimination.

3. Adversarial Training Regime

ADAEN introduces an adversarial game where a discriminator $x^{(i)}\in\mathbb{R}^d$5 (parameterized as a 3-layer MLP) aims to separate genuine data from reconstructed outputs of AE₁ and AE₂. The adversarial losses are:

Discriminator:

$x^{(i)}\in\mathbb{R}^d$6

Generator (AE₁, AE₂):

$x^{(i)}\in\mathbb{R}^d$7

The overall objective combines reconstruction and adversarial losses via hyperparameter $x^{(i)}\in\mathbb{R}^d$8 ($x^{(i)}\in\mathbb{R}^d$9):

$x\in\mathbb{R}^d$0

Optimization alternates between minimizing $x\in\mathbb{R}^d$1 (updating $x\in\mathbb{R}^d$2 with AE parameters fixed) and minimizing $x\in\mathbb{R}^d$3 (updating AE parameters with $x\in\mathbb{R}^d$4 fixed).

4. Anomaly Scoring and Ranking

During inference, ADAEN computes an anomaly score for each $x\in\mathbb{R}^d$5:

$x\in\mathbb{R}^d$6

Samples are sorted in descending order by $x\in\mathbb{R}^d$7, producing a ranked anomaly list. Evaluation prioritizes high-fidelity ranking, employing normalized discounted cumulative gain (nDCG):

$x\in\mathbb{R}^d$8

where $x\in\mathbb{R}^d$9 is the anomaly label and $z\in\mathbb{R}^k$0 is the ideal $z\in\mathbb{R}^k$1.

5. Active Learning with Data Augmentation

To address limited labeled anomaly data, ADAEN incorporates an active learning protocol:

1. Train ADAEN on initial labeled set $z\in\mathbb{R}^k$2 (all normals).
2. Score unlabeled pool $z\in\mathbb{R}^k$3 by $z\in\mathbb{R}^k$4.
3. Set threshold $z\in\mathbb{R}^k$5 at the $z\in\mathbb{R}^k$6-th percentile of $z\in\mathbb{R}^k$7.
4. Compute uncertainty $z\in\mathbb{R}^k$8 for $z\in\mathbb{R}^k$9.
5. Select top-$$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$0 points (minimum $$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$1) and query an oracle for ground truth.
6. For points confirmed as normal, augment using a GAN $$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$2:
   • Train GAN on oracle-confirmed normals.
   • Generate synthetic normals $$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$3.
7. Update $$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$4 new normals (real and synthetic).
8. Retrain ADAEN and repeat steps up to $$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$5 rounds.

Pseudocode (as in (Benabderrahmane et al., 25 Nov 2025)):

$$\mathcal{L}_{rec_j} = \frac{1}{|\mathcal{X}|} \sum_{x \in \mathcal{X}} \| x - g_{\phi_j}(f_{\theta_j}(x)) \|^2_2$$5

6. Implementation and Hyperparameters

The ADAEN configuration is defined by several key hyperparameters and training details:

Symbol	Meaning	Default/Example Value
$$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$6	Input dimension	(dataset-dependent)
$$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$7	Latent code size	$$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$8
$$z_j = f_{\theta_j}(x) = \sigma(W_e^{(j)}x + b_e^{(j)}),$$9	AE₁/AE₂ recon. weight	$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$0
$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$1	Adversarial term weight	$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$2
$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$3	Anomaly threshold percentile	$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$4
$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$5	Oracle queries per round	user-defined
$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$6	Max active learning iters	40
$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$7	Activation function	LeakyReLU + batch-norm, dropout
$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$8	Discriminator net arch	3-layer MLP
$$\hat{x}_j = g_{\phi_j}(z_j) = \sigma(W_d^{(j)}z_j + b_d^{(j)}), \quad j=1,2$$9	GAN generator/discriminator	(see (Benabderrahmane et al., 25 Nov 2025))

Optimization uses Adam (lr=$$\mathcal{L}_{rec_j} = \frac{1}{|\mathcal{X}|} \sum_{x \in \mathcal{X}} \| x - g_{\phi_j}(f_{\theta_j}(x)) \|^2_2$$0-$$\mathcal{L}_{rec_j} = \frac{1}{|\mathcal{X}|} \sum_{x \in \mathcal{X}} \| x - g_{\phi_j}(f_{\theta_j}(x)) \|^2_2$$1, $$\mathcal{L}_{rec_j} = \frac{1}{|\mathcal{X}|} \sum_{x \in \mathcal{X}} \| x - g_{\phi_j}(f_{\theta_j}(x)) \|^2_2$$2=0.5, $$\mathcal{L}_{rec_j} = \frac{1}{|\mathcal{X}|} \sum_{x \in \mathcal{X}} \| x - g_{\phi_j}(f_{\theta_j}(x)) \|^2_2$$3=0.999), batch size 128, and early stopping with patience 10 on validation $$\mathcal{L}_{rec_j} = \frac{1}{|\mathcal{X}|} \sum_{x \in \mathcal{X}} \| x - g_{\phi_j}(f_{\theta_j}(x)) \|^2_2$$4.

7. Application Context and Empirical Results

ADAEN targets extreme class-imbalance settings, exemplified by APT detection where attacks comprise as little as 0.004% of events in provenance trace databases (DARPA Transparent Computing). Empirical evaluation spans Android, Linux, BSD, and Windows datasets under two distinct attack scenarios. Adoption of the ranking- and active-learning-enhanced protocol yields significant improvements in detection rates and ranking metrics (nDCG), outperforming previous approaches in prioritizing true anomalies with reduced labeling overhead. A plausible implication is that such an architecture facilitates deployment in real-world security infrastructure with minimal manual annotation requirements (Benabderrahmane et al., 25 Nov 2025).