Robust Multi-sphere Learning (RMSL)

• The paper introduces a novel framework that integrates multiple learnable hyper-spheres with a discriminative anomaly classifier to precisely localize malicious behaviors within user sequences.
• It employs a three-stage training process—zero-positive warm-up, multiple instance learning, and adaptive self-training debiasing—to convert sequence-level labels into accurate behavior-level anomaly scores.
• Empirical evaluations on CERT r4.2 and r5.2 benchmarks demonstrate significant improvements in AUC, detection rate, and false-positive reduction over 16 baseline methods.

Robust Multi-sphere Learning (RMSL) is a weakly supervised behavior-level insider threat detection framework that uses multiple learnable hyper-spheres to model diverse normal behavior patterns and combines that geometric view with a discriminative anomaly classifier. It is designed for the setting in which training labels are available only at the sequence level, while the target output is a behavior-level anomaly score for each action inside a user-behavior sequence. The framework is trained progressively through a zero-positive one-class warm-up, multiple instance learning (MIL), and adaptive behavior-level self-training debiasing, with the stated goal of improving behavior-level anomaly localization under sequence-level supervision (Wang et al., 15 Aug 2025).

1. Problem setting and formal task

RMSL is formulated for behavior-level insider threat detection in enterprise logs. In this setting, logs record user actions over time, including logon/logoff, email, file access, device use, and HTTP requests, and these actions are aggregated into behavior sequences. The objective is not merely to classify an entire sequence as anomalous or normal, but to assign an anomaly score to each individual behavior so that specific malicious actions can be localized. The labeled training set is written as

$$\mathcal{D}_L = \{S^{(i)}, Y^{(i)}\}_{i=1}^{|\mathcal{D}_L|},$$

where

$S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$

is a sequence of behaviors and

$Y^{(i)} \in \{0,1\}$

is a sequence-level label. At test time, behavior-level ground truth is available,

$$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$

with

$$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$

A sequence is anomalous if it contains at least one anomalous behavior (Wang et al., 15 Aug 2025).

The framework is motivated by three difficulties emphasized in the underlying work. First, behavior-level anomaly annotations are extremely expensive because malicious behaviors are rare and stealthy, and a sequence can contain hundreds or thousands of behaviors. Second, normal and abnormal behaviors are ambiguous: purely unsupervised or one-class methods learn “normality,” but normal behavior is diverse, incomplete in training data, and often overlaps with subtle malicious behavior. Third, unsupervised methods can incur high false positives and miss rates, because unseen benign behaviors may be flagged as anomalous while malicious users may imitate normal patterns. RMSL addresses this by introducing sequence-level weak labels, which are cheaper than full behavior-level annotation but still inject explicit anomaly-related supervision.

The resulting training regime is a weakly-supervised insider threat detection setting. During training, only sequence labels are used; during evaluation, the method is assessed at the behavior level. This distinction is central to the framework: RMSL is not a fully supervised behavior classifier, and it is not purely one-class anomaly detection. It is a weakly supervised model that attempts to convert sequence-level supervision into behavior-level discriminative structure.

2. Representation learning, hyper-spheres, and anomaly scoring

RMSL consists of three components: a sequence encoder, multiple hyper-spheres based normal prototypes, and an anomaly classifier. Each discrete behavior code $e_l$ is first embedded,

$\mathbf{e}_l = \operatorname{Embedding}(e_l),$

and a two-layer bidirectional GRU produces contextual behavior representations,

$$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$

where $\mathbf{x}_l \in \mathbb{R}^d$ is the contextual representation of behavior $e_l$ (Wang et al., 15 Aug 2025).

The geometric core of the method is a set of $S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$0 learnable hyper-sphere centers

$S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$1

For a behavior representation $S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$2, the distance to the $S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$3-th sphere center is

$S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$4

and the nearest sphere index is

$S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$5

The hyper-sphere deviation score is then

$S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$6

This score is small when the behavior lies near one normal prototype and large when it deviates from all known normal modes.

RMSL supplements this geometric score with a discriminative anomaly classifier $S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$7. A self-attention layer refines $S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$8 into $S^{(i)} = \{e_l^{(i)}\}_{l=1}^{N_S^{(i)}}$9, and a fully connected layer produces an anomaly probability,

$Y^{(i)} \in \{0,1\}$0

where $Y^{(i)} \in \{0,1\}$1 and $Y^{(i)} \in \{0,1\}$2. The final behavior-level anomaly score combines the two views:

$Y^{(i)} \in \{0,1\}$3

with $Y^{(i)} \in \{0,1\}$4 as the dual scoring balance factor. When $Y^{(i)} \in \{0,1\}$5, inference is classification-only; when $Y^{(i)} \in \{0,1\}$6, inference is sphere-deviation-only. Intermediate values combine class separability with deviation from normal prototypes.

The use of multiple hyper-spheres rather than a single sphere is a defining feature. Prior one-class approaches such as Deep SVDD and OC4Seq compress normal data into a single minimal-volume hyper-sphere. RMSL instead assumes that normal behavior is multi-modal: different users, times, contexts, and actions may produce distinct normal patterns. The multi-sphere design is therefore intended to avoid overcompressing heterogeneous normality and to reduce false positives on valid but rare normal behaviors.

3. Three-stage training procedure

The training algorithm is progressive. Stage 1 is called multiple hyper-spheres based zero positive warm-up and uses only normal sequences. Its purpose is to produce an anomaly-supervision-free starting point before weak supervision is introduced. For a normal sequence $Y^{(i)} \in \{0,1\}$7, RMSL applies a multi-center loss that pulls each behavior toward its nearest sphere and a separability loss that prevents sphere collapse by encouraging the second-nearest center to be farther than the nearest center. The total warm-up objective is

$Y^{(i)} \in \{0,1\}$8

The paper states that directly beginning MIL can be unstable because, early in training, the highest-scoring behaviors in anomalous sequences may not actually be anomalous, creating accumulating optimization error. The one-class warm-up is therefore used as a stable initialization (Wang et al., 15 Aug 2025).

Stage 2 introduces weak sequence-level supervision through MIL. The key MIL assumption is that an anomalous sequence contains at least one anomalous behavior, so the top-scoring instances in an anomalous sequence should score higher than those in a normal sequence. If $Y^{(i)} \in \{0,1\}$9 denotes the selected top-anomaly-score behaviors in sequence $$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$0, the predicted sequence score is

$$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$1

and the MIL loss is

$$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$2

This stage refines the encoder, the classifier, and the sphere structure using sequence labels, and the paper explicitly states that it improves the model’s ability to distinguish anomalous from normal behavior beyond pure one-class learning.

Stage 3 addresses MIL’s selection bias through adaptive behavior-level self-training debiasing. RMSL uses MC Dropout to estimate uncertainty over behavior-level anomaly scores. For a behavior $$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$3, with dropout-perturbed parameters $$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$4, $$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$5, the score mean and variance are estimated as

$$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$6

and

$$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$7

Lower variance indicates higher confidence. Behaviors in an anomalous sequence are partitioned into high-confidence, medium-confidence, and low-confidence subsets according to these variances. High-confidence samples receive hard pseudo-labels using threshold $$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$8, medium-confidence samples use a mixture of hard pseudo-labels and soft pseudo-labels from an EMA teacher, and low-confidence samples are effectively left unlabeled. The stage-3 loss is

$$\mathcal{D}_{test}=\{S^{(i)}, \mathbf{Y}^{(i)}\}_{i=1}^{|\mathcal{D}_{test}|},$$9

This debiasing stage is intended to use more behaviors than MIL alone while controlling pseudo-label noise.

At inference time, the model embeds and encodes a sequence, computes $$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$0 and $$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$1, combines them into $$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$2, and predicts anomaly when the score exceeds $$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$3. The output is therefore behavior-level anomaly scoring, not only sequence classification.

4. Empirical evaluation and reported performance

RMSL is evaluated on two public insider threat benchmarks, CERT r4.2 and CERT r5.2. In both datasets, logs from multiple sources are merged chronologically per user, sessions are segmented by login/logout, and each session is treated as one behavior sequence. The first year is used for training and validation, and the remaining six months are used for testing. Training uses sequence-level weak labels only, while testing uses behavior-level labels (Wang et al., 15 Aug 2025).

The data are extremely imbalanced. For CERT r4.2, the reported counts are 469,478 normal sequences, 1,134 abnormal sequences, a sequence-level imbalance ratio of 414, 32,762,906 normal behaviors, 7,316 abnormal behaviors, and a behavior-level imbalance ratio of 4,478. For CERT r5.2, the counts are 1,004,791 normal sequences, 1,843 abnormal sequences, a sequence-level imbalance ratio of 545, 79,846,358 normal behaviors, 10,306 abnormal behaviors, and a behavior-level imbalance ratio of 7,748. RMSL is compared against 16 baselines: DeepLog, TIRESIAS, RNN, GRU, Transformer, RWKV, DIEN, BST, FMLP, m-RNN, m-GRU, m-LSTM, m-Transformer, m-FMLP, ITDBERT, and OC4Seq. Evaluation uses behavior-level AUC, DR, FPR, DR@5%, DR@10%, and DR@15%, with

$$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$4

On CERT r4.2, RMSL reports AUC $$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$5, DR $$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$6, FPR $$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$7, DR@5% $$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$8, DR@10% $$\mathbf{Y}^{(i)}=\{y_l^{(i)}\}_{l=1}^{N_S^{(i)}}, \qquad y_l^{(i)}\in\{0,1\}.$$9, and DR@15% $e_l$0. The best baseline AUC is reported as $e_l$1 from m-FMLP, yielding an absolute AUC improvement of $e_l$2 and a relative improvement of $e_l$3. The paper also reports DR improvement of $e_l$4, FPR reduction of $e_l$5, DR@5% improvement of $e_l$6, DR@10% improvement of $e_l$7, and DR@15% improvement of $e_l$8. On CERT r5.2, RMSL reports AUC $e_l$9, DR $\mathbf{e}_l = \operatorname{Embedding}(e_l),$0, FPR $\mathbf{e}_l = \operatorname{Embedding}(e_l),$1, DR@5% $\mathbf{e}_l = \operatorname{Embedding}(e_l),$2, DR@10% $\mathbf{e}_l = \operatorname{Embedding}(e_l),$3, and DR@15% $\mathbf{e}_l = \operatorname{Embedding}(e_l),$4. The best baseline AUC is reported as $\mathbf{e}_l = \operatorname{Embedding}(e_l),$5 from OC4Seq, with absolute AUC improvement $\mathbf{e}_l = \operatorname{Embedding}(e_l),$6 and relative improvement $\mathbf{e}_l = \operatorname{Embedding}(e_l),$7, together with DR improvement of $\mathbf{e}_l = \operatorname{Embedding}(e_l),$8, FPR reduction of $\mathbf{e}_l = \operatorname{Embedding}(e_l),$9, DR@5% improvement of $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$0, DR@10% improvement of $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$1, and DR@15% improvement of $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$2.

The ablation study evaluates four variants: stage 1 only, stage 2 only, stage 1 + 2, and stage 1 + 2 + 3. The reported conclusions are that stage 1 + 2 significantly outperforms stage 1 alone, stage 1 + 2 + 3 slightly but consistently outperforms stage 1 + 2, and stage 1 + 2 is better than stage 2 alone. Exact ablation values are not given in the text because they are presented in a figure. Hyperparameter analysis varies the number of spheres $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$3 from 1 to 100 in steps of 10; AUC first increases and then slightly decreases on both datasets, with best performance at $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$4. Analysis of the dual score balance $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$5 reports an optimum of $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$6 on CERT r4.2 and $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$7 on CERT r5.2, while the implementation details report $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$8 for r4.2 and $$\mathbf{X} = (\mathbf{x}_1,\mathbf{x}_2,\dots,\mathbf{x}_{N_S}) = \operatorname{GRU}(\mathbf{e}_1,\mathbf{e}_2,\dots,\mathbf{e}_{N_S}),$$9 for r5.2. The paper explicitly notes these values, so the discrepancy is part of the published description.

The paper also reports t-SNE visualizations comparing the zero-positive one-class embeddings to the embeddings after weak supervision. In the zero-positive setting, normal and anomalous embeddings overlap substantially; after weak supervision, normal behaviors cluster around sphere centers and are more clearly separated from anomalies. This supports the claim that weak sequence labels improve inter-class separability.

5. Implementation profile, strengths, and limitations

The reported implementation uses AdamW with weight decay $\mathbf{x}_l \in \mathbb{R}^d$0. The learning rates are $\mathbf{x}_l \in \mathbb{R}^d$1 for stage 1, $\mathbf{x}_l \in \mathbb{R}^d$2 for stage 2, and $\mathbf{x}_l \in \mathbb{R}^d$3 for stage 3. The batch size is 128, consisting of 64 normal sequences and 64 abnormal sequences. The embedding dimension is 128, the number of spheres is 40, $\mathbf{x}_l \in \mathbb{R}^d$4, training lasts 10 epochs, and early stopping is used. Hyper-sphere centers are randomly initialized. The implementation is reported in Python 3.8 and PyTorch 1.13, and experiments are run on an A6000 GPU with 48 GB memory (Wang et al., 15 Aug 2025).

The framework’s stated strengths follow directly from its construction. It addresses label scarcity by using sequence-level labels rather than behavior-level labels; it models heterogeneous normality through multiple spheres rather than a single sphere; it combines geometry and discrimination through the dual score $\mathbf{x}_l \in \mathbb{R}^d$5; its progressive training improves optimization stability; and its confidence-aware self-training attempts to debias MIL by using more of each anomalous sequence. The experimental section reports state-of-the-art behavior-level detection on CERT r4.2 and r5.2.

The method also depends on several assumptions and carries explicit limitations. The paper notes that weak label quality matters: inaccurate sequence-level labels can degrade performance. Additional assumptions embedded in the method are that anomalous sequences contain at least one anomalous behavior, that top-scoring MIL instances are informative, that MC Dropout is sufficiently useful for confidence estimation, and that normal behaviors can be represented by a finite set of hyper-spherical prototypes. Potential failure modes noted in the description include mislabeled normal or anomalous sequences, anomalies that lie close to a normal prototype, inappropriate choice of the number of spheres $\mathbf{x}_l \in \mathbb{R}^d$6, and poor calibration of pseudo-label thresholds. These are plausible constraints of the training design rather than separate theoretical guarantees.

6. Terminological scope and related usages

A recurrent source of confusion is the earlier method RMMSL, short for Robust Multiple Manifolds Structure Learning. RMMSL studies multiple low intrinsic dimensional manifolds, estimates local tangent spaces by weighted low-rank matrix factorization, and performs robust manifold clustering using curvature-aware similarity. It is closely related to what someone might loosely mean by “robust multi-sphere learning” because its synthetic examples include intersecting spheres, but it is not a method explicitly named Robust Multi-sphere Learning, and it is strictly manifold-based rather than sphere-fitting-based (Gong et al., 2012).

A broader but distinct usage of “multi-sphere” appears in subseasonal-to-seasonal forecasting. TianXing-S2S is described as a multi-sphere coupled probabilistic model in which “multi-sphere” denotes atmosphere–ocean–land–cryosphere/interface-flux interactions. That work is relevant to the general idea of sphere-specific representation learning and cross-sphere coupling, but it does not introduce a framework explicitly called RMSL and belongs to Earth-system forecasting rather than insider threat detection (Mu et al., 14 Dec 2025).

In the precise contemporary sense established by the 2025 insider-threat paper, RMSL denotes the weakly supervised framework that uses multiple learnable hyper-spheres, MIL, and adaptive self-training debiasing for behavior-level insider threat detection. The acronym therefore names a specific anomaly-detection architecture rather than a generic class of manifold, sphere-fitting, or Earth-system multi-sphere methods (Wang et al., 15 Aug 2025).