BN-WVAD: BatchNorm Weakly Supervised VAD

• The paper introduces a BatchNorm-based framework that repurposes normalization statistics as a model of normality for weakly supervised video anomaly detection, achieving state-of-the-art performance.
• It employs a novel Divergence of Feature from Mean (DFM) criterion combined with a Mean-based Pull-Push loss to robustly separate anomalous from normal snippets despite temporal ambiguity and label noise.
• The method integrates a dual selection strategy (SLS and BLS) along with pre-trained I3D features and Transformer enhancement to effectively localize anomalies in untrimmed videos.

Searching arXiv for BN-WVAD and closely related weakly supervised video anomaly detection papers. BN-WVAD most commonly denotes a BatchNorm-based weakly supervised video anomaly detection framework in which Batch Normalization statistics are repurposed as an implicit model of normality, and anomalous snippets are identified by their Divergence of Feature from Mean (DFM). In this formulation, training uses only video-level labels for untrimmed videos, so temporal anomaly localization must be inferred despite temporal ambiguity and label noise. The framework combines BatchNorm-derived outlier scoring, a batch-aware snippet selection mechanism, and a classifier supervised only on normal snippets, yielding state-of-the-art results on UCF-Crime, XD-Violence, and ShanghaiTech (Zhou et al., 2023).

1. Problem setting and conceptual basis

In weakly supervised video anomaly detection, each training video $V_i$ has only a video-level label: $Y_i = 0$ for a normal video and $Y_i = 1$ for an abnormal video. Videos are segmented into snippets, but snippet-level anomaly annotations are absent. The central difficulty is therefore temporal ambiguity: an abnormal-labeled video contains abnormal events somewhere, but the abnormal snippets are unknown. This induces label noise, because any heuristic that selects candidate abnormal snippets from abnormal videos will inevitably include some normal snippets and exclude some abnormal ones (Zhou et al., 2023).

Earlier WVAD methods are described as largely MIL-based. They typically select top-$k$ snippets per abnormal video according to a hand-designed abnormality criterion such as feature magnitude or attention, then train a classifier so those selected snippets score higher than snippets from normal videos. BN-WVAD is motivated by three limitations identified for that regime: unreliable abnormality criteria, fixed per-video top-$k$ selection that ignores heterogeneous abnormality ratios across videos, and classifier sensitivity to misselected snippets.

The key statistical insight is that BatchNorm can function as a normality model rather than merely an optimization primitive. Hidden features are organized as $\mathbf{X}^h \in \mathbb{R}^{B \times T \times C}$, where $B$ is the batch size, $T$ the number of snippets per video, and $C$ the feature dimension. Because mini-batches are constructed with half normal and half abnormal videos, and because most snippets in real surveillance datasets are normal even inside abnormal videos, the majority of snippets in a batch are normal. The batch mean therefore tends to be dominated by normal content, so abnormal snippets can be interpreted as outliers relative to BatchNorm statistics. The paper explicitly links this view to the Central Limit Theorem and treats the BatchNorm mean and variance, or their EMA running estimates, as a statistical reference of normality (Zhou et al., 2023).

A common misconception is that BN-WVAD is simply a conventional WVAD architecture with BatchNorm inserted for training stability. In the named 2023 method, BatchNorm is structurally central because its running mean and variance define the abnormality criterion itself. The method’s novelty is therefore not just normalization, but the use of normalization statistics as an implicit memory of normality.

2. Architecture and representation pipeline

BN-WVAD follows a standard high-level WVAD pipeline of pre-extracted features, temporal enhancement, and snippet-wise anomaly scoring, but modifies both abnormality estimation and supervision. The feature extractor uses pre-trained I3D (Kinetics) to obtain snippet-level visual features. For XD-Violence, some variants additionally use audio embeddings from VGGish, concatenated with visual features (Zhou et al., 2023).

A Transformer-based enhancer, described as the same type used in UR-DMU, processes the snippet features and outputs enhanced features $\mathbf{X}^e$ of dimension 512. These are passed to a temporal classifier head composed of two temporal $Y_i = 0$0-D convolutions with kernel size $Y_i = 0$1:

• first Conv1d: $Y_i = 0$2, followed by BatchNorm and ReLU, producing hidden features $Y_i = 0$3;
• second Conv1d: $Y_i = 0$4, followed by BatchNorm and ReLU, producing hidden features $Y_i = 0$5.

An anomaly classifier $Y_i = 0$6 then produces snippet-wise anomaly predictions from the BatchNorm-plus-ReLU hidden representation. The BatchNorm layers are standard rather than modified, with momentum $Y_i = 0$7. What distinguishes the framework is that the running mean and variance of these layers are later reused to define DFM (Zhou et al., 2023).

Videos are uniformly split and linearly interpolated to 200 snippets per video, giving a fixed temporal resolution for training and inference. At inference time, the method applies multi-crop aggregation by cropping a video into multiple temporal segments and averaging predictions across crops; the number of crops is 10 for UCF-Crime and 5 for XD-Violence.

This architecture is deliberately compact in its classifier head. A plausible implication is that the method concentrates representational complexity in the feature enhancer and uses the BatchNorm layers to impose a statistically interpretable notion of normality, rather than relying on a deep discriminative head trained with noisy pseudo-positive labels.

3. DFM, MPP, and the normal-only training regime

The core abnormality measure in BN-WVAD is the Divergence of Feature from Mean (DFM). Given hidden features $Y_i = 0$8, BatchNorm computes a batch mean and per-dimension variance,

$Y_i = 0$9

with covariance approximated by a diagonal matrix $Y_i = 1$0. DFM is defined as the Mahalanobis distance from a snippet feature to the BatchNorm mean,

$Y_i = 1$1

In practice, the method uses the running mean and variance rather than instantaneous batch statistics, which the paper states improves consistency between training and inference and reduces sensitivity to batch composition noise (Zhou et al., 2023).

Mahalanobis distance is used because feature dimensions can have different variances. The ablations report Mahalanobis $Y_i = 1$2 Euclidean $Y_i = 1$3 cosine similarity, indicating that variance-aware scaling is important for the abnormality criterion. DFM is computed at both hidden levels $Y_i = 1$4 and $Y_i = 1$5, and the two DFM values are summed when forming the DFM component of the final anomaly score.

BN-WVAD redefines supervision around two losses. The first is a normal-only regression loss,

$Y_i = 1$6

which is applied only to normal videos and encourages all snippet predictions in those videos to remain near zero. No corresponding classifier loss is imposed on abnormal videos. This is a central departure from conventional MIL pipelines, because the classifier is never trained on potentially mislabeled positive snippets.

The second is a Mean-based Pull-Push (MPP) loss, which uses the BatchNorm mean as an anchor. Let $Y_i = 1$7 denote the $Y_i = 1$8 highest-DFM snippets from abnormal videos, and $Y_i = 1$9 the $k$0 highest-DFM snippets from normal videos. MPP is

$k$1

with margin $k$2. The overall objective is

$k$3

with $k$4 and $k$5.

This design separates normality modeling from classifier supervision. The classifier learns only “what normal looks like,” whereas anomaly structure is imposed on the feature space through DFM and MPP. The paper reports that adding an explicit abnormal loss $k$6 based on pseudo labels significantly hurts performance, especially on XD-Violence, which is presented as evidence that direct positive supervision is too noisy in weakly supervised anomaly localization (Zhou et al., 2023).

4. Sample-Batch Selection and final anomaly scoring

A second major component is the Sample-Batch Selection (SBS) strategy. The paper argues that conventional sample-level selection (SLS)—selecting top-$k$7 or top-$k$8 snippets per abnormal video—fails when abnormality ratios differ sharply across videos. A fixed quota can under-select anomalies in videos with many abnormal snippets and over-select noise in videos with few.

To address this, BN-WVAD adds batch-level selection (BLS). Instead of ranking snippets independently within each abnormal video, all abnormal snippets in a mini-batch are pooled together, and the top $k$9 snippets are selected across the batch according to DFM, where $k$0 is the number of abnormal videos in the batch. This lets videos with higher abnormality ratios contribute more candidate abnormal snippets (Zhou et al., 2023).

SBS takes the union of SLS and BLS in abnormal videos:

1. SLS in abnormal videos: select the top $k$1 snippets per video by DFM.
2. BLS in abnormal videos: select the top $k$2 snippets across the batch by DFM.
3. SBS abnormal set: union of these selections.
4. Normal-video selection: apply only SLS in normal videos, matching the number of selected normal snippets to the number selected from abnormal videos for MPP pairing.

The paper characterizes this union as capturing both highly abnormal snippets from high-ratio videos and difficult lower-score anomalies from low-ratio videos. The dataset-specific hyperparameters are:

• UCF-Crime: $k$3, $k$4
• XD-Violence: $k$5, $k$6

Final snippet-level anomaly scores combine classifier output and DFM multiplicatively. If

$k$7

and $k$8 is the DFM value, then

$k$9

When two hidden levels are used, their DFM scores are summed before multiplication. This multiplicative fusion is important to the method’s behavior: DFM provides an outlier-based anomaly prior, while the classifier acts as a learned normality suppressor. The ablations report that the combination outperforms either component alone.

5. Empirical performance and ablation evidence

The reported experimental results position BN-WVAD as state of the art on multiple weakly supervised VAD benchmarks (Zhou et al., 2023).

Dataset	Metric	BN-WVAD result
UCF-Crime	AUC	87.24%
XD-Violence (video-only, I3D)	AP	84.93%
XD-Violence (audio+video)	AP	85.26%
ShanghaiTech	AUC	97.61%

On UCF-Crime, BN-WVAD achieves 87.24% AUC, compared with 86.97% for UR-DMU as reported in that comparison and with prior systems such as CU-Net, SAS, S3R, and RTFM described as lying in the 84–86% range. On XD-Violence, BN-WVAD reaches 84.93% AP in the video-only setting and 85.26% AP in the audio-visual setting, exceeding 83.40% AP for MACIL-SD and 81.77% AP for UR-DMU in the cited comparison. On ShanghaiTech, BN-WVAD obtains 97.61% AUC, slightly above earlier results around 97.3–97.5%.

The ablations are central to understanding why the method works. With only $\mathbf{X}^h \in \mathbb{R}^{B \times T \times C}$0 and Dropout, performance on UCF-Crime is reported as 65.21% AUC. Replacing Dropout with BatchNorm raises this to 82.97% AUC, which the paper interprets as evidence that BatchNorm statistics alone already encode useful normality information. Adding DFM and MPP increases performance further to 86.44% AUC, and adding BLS, yielding the full SBS strategy, reaches the final 87.24% AUC.

Several additional findings sharpen the method’s profile. Replacing feature magnitude with DFM improves performance both in BN-WVAD and when plugged into RTFM, suggesting that DFM is a more discriminative abnormality criterion. Using SBS outperforms either SLS only or BLS only. The MPP loss improves feature separation in t-SNE visualizations, while adding an abnormal loss $\mathbf{X}^h \in \mathbb{R}^{B \times T \times C}$1 hurts performance significantly, described as especially severe on XD-Violence. The BatchNorm momentum performs best around $\mathbf{X}^h \in \mathbb{R}^{B \times T \times C}$2; both smaller values such as 0.01 and larger values such as 0.5 or 1 degrade performance, and $\mathbf{X}^h \in \mathbb{R}^{B \times T \times C}$3 is said to cause a large drop because it overfits to per-batch statistics. Balanced batches are also important: the supplementary analysis reports that balanced batches with 64 normal + 64 abnormal videos perform best, whereas too small, too large, or unbalanced batches degrade results.

Taken together, these results support a specific interpretation: BN-WVAD is not merely a stronger classifier head, but a framework in which BatchNorm-derived statistics, selection policy, and supervision regime are tightly coupled.

6. Scope, limitations, and terminological disambiguation

The framework assumes that anomalies appear as outliers in feature space and that mini-batches are sufficiently large and balanced for BatchNorm statistics to be reliable. The paper identifies several limitations: dependence on stable BatchNorm statistics, reduced gains on smaller datasets such as ShanghaiTech where BatchNorm can overfit, sensitivity to the SBS hyperparameters $\mathbf{X}^h \in \mathbb{R}^{B \times T \times C}$4 and $\mathbf{X}^h \in \mathbb{R}^{B \times T \times C}$5, and the possibility that BLS alone will miss rare or subtle anomalies. It also notes possible extensions, including alternative normalization schemes such as GroupNorm or LayerNorm, more sophisticated batch construction, dynamic adaptation of $\mathbf{X}^h \in \mathbb{R}^{B \times T \times C}$6 and $\mathbf{X}^h \in \mathbb{R}^{B \times T \times C}$7, improved audio-visual fusion, iterative temporal refinement, and cross-dataset use of learned BatchNorm statistics.

The acronym also has a separate, potentially confusing usage in speech processing. In the voice activity detection literature, WVAD denotes “waveform-based VAD”, a fully convolutional encoder-decoder that operates directly on raw time-domain waveforms for frame-level speech/non-speech prediction; the same work also defines WEVAD as a multi-branched ensemble of attribute-specific encoders (Yu et al., 2020). That paper does not explicitly define BN-WVAD as a named method. The associated technical explanation states that, from the context and common naming conventions, BN-WVAD is very likely to denote a waveform-based VAD architecture augmented with batch normalization. This suggests a distinct speech-oriented interpretation of the acronym, separate from the named 2023 video anomaly detection framework.

For the research literature, the dominant meaning of BN-WVAD is therefore the 2023 weakly supervised video anomaly detection model built around BatchNorm statistics, DFM, MPP, and SBS. Its defining contribution is the reinterpretation of BatchNorm from a training utility into a statistical reference of normality that can support robust anomaly localization under weak supervision.