ARES: Adaptive Anomaly Scoring

• ARES is an anomaly detection framework that adapts reconstruction error scoring by analyzing local latent space neighborhoods.
• It computes a local reconstruction score and density measure (via LOF) to address non-uniform error distributions in high-dimensional data.
• The approach improves detection accuracy with minimal runtime overhead, achieving significant AUC gains across diverse benchmarks.

ARES (Adaptive Reconstruction Error-based Scoring) is a framework for anomaly detection in high-dimensional datasets, such as images and sensor signals. It addresses the limitations of standard autoencoder-based (AE) anomaly scoring by introducing local adaptivity in the estimation of anomaly scores, thereby accounting for non-uniform distributions of reconstruction error among normal inputs. ARES enhances detection accuracy by contextualizing anomaly measurements in latent space neighborhoods and combining adaptive reconstruction scoring with a density estimation term (Goodge et al., 2022).

1. Motivation and Limitations of Conventional Autoencoder Scoring

Standard autoencoder approaches train an encoder–decoder pair on "all-normal" data points $X_{\text{train}} \subset \mathbb{R}^d$, minimizing the mean squared reconstruction error: $$\min_{\theta, \phi} \sum_i \|x_i - (f_\theta \circ g_\phi)(x_i)\|_2^2,$$ and use the squared error at test time as an anomaly score: $R(x) := \|x - (f_\theta \circ g_\phi)(x)\|_2^2.$ These methods assume a homoscedastic error model, meaning the distribution of reconstruction error is uniform across the input or latent space. However, in realistic data, some "hard" normal examples consistently have larger reconstruction errors than "easy" ones. A global threshold $\tau$ for flagging anomalies thus results in excessive false positives (flagging hard normals) and false negatives (missing anomalies that resemble easy normals). This failure of non-adaptive scoring is especially pronounced in cases where the structure of the normal data is multi-modal or spans a wide range of complexities (Goodge et al., 2022).

2. Adaptive Local Anomaly Score: Latent Neighborhoods and Score Composition

ARES adapts the anomaly score by using the local behavior of reconstruction errors in the autoencoder's latent space. After AE training, each training sample $x_i$ is mapped to a latent code $z_i = g_\phi(x_i)$ and has a stored reconstruction error $R_i = \|x_i - \hat{x}_i\|_2^2$. For a test point $x$ with code $z = g_\phi(x)$, ARES constructs a latent-space neighborhood $N_k(z)$, defined as the $k$ nearest latent codes among $\{z_1, ..., z_m\}$ under Euclidean distance.

Within this neighborhood, ARES computes two scores:

• Local reconstruction score: A nonparametric, locally-centered value,

$$r(x) = R(x) - \text{median} \{ R(x_i): i \in N_k(z) \}$$

which measures how atypical the reconstruction error of $x$ is compared to its local neighborhood.

• Local density score: Using the Local Outlier Factor (LOF) in latent space, which normalizes the error based on local data density,

$d(x) = LOF_k(z)$

where LOF is computed as in [Breunig et al. 2000].

The final ARES score is: $s(x) = r(x) + \alpha d(x)$ with the default $\alpha = 0.5$ (no dataset-specific tuning).

3. Algorithmic Workflow and Practical Implementation

Training phase:

1. Train AE on $X_{\text{train}}$ to minimize reconstruction error.
2. For each $x_i$, store $z_i = g_\phi(x_i)$ and $R_i$.

Inference phase (for each test $x$):

1. Encode $x$ to $z$, reconstruct $\hat{x}$, compute $R(x)$.
2. Find $k$-nearest neighbors $N_k(z)$ among $\{z_i\}$.
3. Compute local reconstruction score $r(x)$.
4. Compute local density term $d(x) = LOF_k(z)$.
5. Anomaly score: $s(x) = r(x) + \alpha d(x)$.
6. Flag $x$ as anomalous if $s(x) > \tau$ for some threshold $\tau$, typically selected via validation set or statistical modeling.

This process introduces only minor overhead (mainly a k-NN search in latent space and LOF computation) relative to a plain AE. Approximate neighbor search or tree structures can further improve runtime; the paper reports a ≲3% total runtime increase when using the default settings (Goodge et al., 2022).

4. Empirical Results and Comparative Performance

ARES was evaluated on multiple anomaly detection benchmarks (single-class and multi-class settings), including:

• SNSR (sensor signals), MNIST, FMNIST, OTTO (e-commerce), MI-F and MI-V (CNC milling), and EOPT (storage failures).

Key metric: average Area Under the ROC Curve (AUC).

Selected one-class improvements (AE vs. ARES, absolute AUC gain): | Dataset | AE | ARES | ΔAUC | |----------|---------------|-------------|--------| | SNSR | 98.30 | 98.83 | +0.53 | | MNIST | 96.96 | 97.89 | +0.93 | | OTTO | 85.26 | 87.86 | +2.60 | | MI-F | 71.19 | 89.52 | +18.33 | | MI-V | 90.75 | 93.94 | +3.19 | | EOPT | 59.85 | 68.43 | +8.58 |

In the multi-class normality case, absolute AUC gains are even larger (e.g., MNIST: +13.21). ARES also consistently outperformed other baselines: LOF in input space, Isolation Forest, PCA, One-Class SVM, DAGMM, variational AE, plain AE, and reconstruction-path methods (Goodge et al., 2022).

5. Latent Space Design and Hyperparameter Analysis

ARES’s local adaptivity hinges on two key hyperparameters—neighborhood size $k$ and latent dimension $p$:

• Neighborhood size $k$: Small $k$ offers high locality but is more sensitive to noise and contamination. Larger $k$ yields robustness (up to 5–10% anomaly contamination in training data) but can reduce adaptivity in multimodal or dense latent spaces. Experimental sweeps identified $k = 10$ (default) and $k = 50\!–\!200$ (more robust) as effective settings.
• Latent dimension $p$: Lower $p$ alleviates curse-of-dimensionality issues in neighbor search and clusters semantically similar samples. Experimental sweet-spot: $p \approx 32\!–\!64$ is robust, covering a broad range.

ARES is tolerant of a wide range of configurations, with performance stable except at extremes of $k$ or $p$.

6. Theoretical Implications and Limitations

ARES explicitly relaxes the homoscedasticity assumption of standard AE scoring. By normalizing reconstruction error against the typical local error in the latent space, ARES effectively models heteroscedastic noise structures—where "easy" and "hard" normal inputs have distinct error regimes.

A practical implication is that density estimation in the latent space (with LOF or even k-NN distances) is effective and more robust than input-space density estimation. Normalizing flows for density estimation were tested but underperformed relative to nonparametric approaches.

Computationally, ARES introduces only a small additional burden for k-NN search and LOF; with appropriate approximate nearest neighbor or tree-based methods, scaling is practical for large datasets. No dataset-specific tuning of $\alpha$ or other parameters is required.

7. Significance and Application Contexts

ARES provides a principled and empirically validated improvement over standard AE-based anomaly detection, particularly in domains where normal data is heterogeneous and local context is crucial (e.g., sensor logs with daily cycles, images with varied complexity). It is applicable wherever meaningful local variation in data complexity or regularity would make global error statistics misleading.

By combining local reconstruction error normalization with latent space density-based scoring, ARES achieves statistically significant increases in detection power, robustness to training data contamination, and resilience to variability in data complexity—demonstrated across diverse benchmarks in both synthetic and real-world domains (Goodge et al., 2022).

---

References:

ARES: "ARES: Locally Adaptive Reconstruction-based Anomaly Scoring" (Goodge et al., 2022)