VAAR: Visual-Audio Anomaly Recognition Dataset

• VAAR is a medium-scale, real-world benchmark comprising 3,000 clips with synchronized audio-visual data across ten distinct anomaly classes.
• Its methodology leverages event-centered segmentation and rigorous manual quality checks to ensure precise audio-visual synchronization.
• Benchmark results with AVAR-Net showcase robust cross-modal learning, improving state-of-the-art performance in spatiotemporal anomaly detection.

Visual-Audio Anomaly Recognition (VAAR) Dataset is a medium-scale, real-world benchmark for synchronized audio-visual anomaly detection, designed to facilitate research in multimodal surveillance, transportation, and public safety systems. The dataset consists of 3,000 curated video clips, each accompanied by temporally aligned high-quality audio, and annotated across ten diverse anomaly classes that reflect critical real-world scenarios. This dataset addresses major limitations of earlier works by providing extensive multimodal coverage, robust synchronization, and a broad event taxonomy tailored for cross-modal learning and robust spatiotemporal reasoning.

1. Dataset Specifications and Composition

The VAAR dataset comprises 3,000 high-definition video clips, sampled from online platforms (e.g., YouTube, TikTok, news sources). Each clip presents a temporally synchronized audio-visual segment, lasting from 5 seconds up to 2 minutes. Video sources are weighted: approximately 80% are from surveillance footage, the remainder from movies and user-generated recordings. Temporal segmentation is performed around event boundaries to maximize alignment between visual actions and audio signals.

Anomaly categories are:

• Abuse
• Baby cry
• Crash
• Brawling
• Explosion
• Intruder
• Normal
• Pain
• Police siren
• Vandalism

Each instance is quality-checked for synchronization and annotated, allowing for both binary and multi-class anomaly detection tasks. The inclusion of subtle acoustic anomalies (e.g., baby cry or police siren) ensures that audio cues are critical for correct classification, extending the discriminative power beyond traditional vision-only surveillance datasets.

2. Audio-Visual Synchronization and Event Diversity

Synchronization is achieved through event-centered segmentation and manual quality checks, guaranteeing that audio data matches the depicted visual phenomena. This methodology ensures that joint representations (actions and sound) are tightly coupled, permitting models to exploit temporal correspondences for more robust event detection—especially under conditions where visual data is obscured, noisy, or affected by poor lighting.

The anomaly taxonomy spans violent actions, catastrophic events (e.g., crash, explosion), illicit activities (vandalism, intruder), as well as events marked solely by audio signals (pain, baby cry, siren). The presence of a "normal" class permits control for baseline behavior, supporting both thresholded binary anomaly recognition and fine-grained classification.

3. Benchmark Utility and Comparison

VAAR advances anomaly recognition benchmarks by supplying a richer set of synchronized multimodal data compared to prior datasets, which were commonly limited by single modality (video or audio) or binary class settings. In comparison to datasets summarized in "Multimedia Datasets for Anomaly Detection: A Review" (Kumari et al., 2021), VAAR features:

• Multiple distinct anomaly types beyond violence/emotion
• Balanced scene complexity, including dense crowds, public spaces, and diverse environmental settings
• High-fidelity audio streams corresponding to real events, not synthesized or heavily acted anomalies This diversity enables evaluation and training of anomaly recognition systems for real-world challenges in smart city contexts and heterogeneous surveillance environments.

4. Application Domains

VAAR enables research and deployment in settings requiring multimodal robustness:

• Public safety and intelligent surveillance: detection of disorderly or illicit behavior even under occlusion or poor visibility
• Transportation systems: robust identification of crashes, explosions, and other traffic-related anomalies integrating both video and environment sounds
• Healthcare monitoring: rapid response to events like distress or falls through combined acoustic-visual cues
• Industrial and edge security: lightweight deployment scenarios where low-latency multimodal event recognition is critical

The dataset has also been validated for generalization across domains such as traffic anomaly detection (see MAVAD (Leporowski et al., 2023)), and shows potential for extension to scenarios where video is degraded but audio remains informative.

5. Representative Algorithms and Frameworks

The VAAR dataset was introduced and benchmarked within the AVAR-Net framework (Ali et al., 15 Oct 2025), a lightweight fusion network that models cross-modal relationships for anomaly recognition. AVAR-Net integrates:

• Spatial visual features via MobileViT (local encodings, transformer-based global context)
• Temporal audio features via Wav2Vec2 (raw waveform convolution, self-attention, transformer layers)
• Early cross-modal fusion via concatenation of feature maps: $F = \mathrm{Concat}(V, A)$
• Sequential modeling via Multi-Stage Temporal Convolutional Network (MTCN), employing dilated convolutions over fused features: $Y(t) = \sum_{i=0}^{k-1} w_i x(t - d\cdot i)$ with $d = 2^l$ (layer-dependent dilation rate)

Attention mechanisms based on 1D-CNN and self-attention modules further refine the selection of discriminative segments before final classification.

Empirically, AVAR-Net achieves 89.29% accuracy on VAAR and 88.56% average precision on the XD-Violence dataset, outperforming contemporary state-of-the-art approaches by 2.8% in some metrics. Ablation studies confirm that the early fusion mechanism and sequence length of 10 frames yield the highest performance.

6. Impact, Limitations, and Future Directions

The release of VAAR addresses several previously identified gaps in anomaly datasets (Kumari et al., 2021):

• Enhanced modality diversity
• Rich class granularity supporting fine-grained recognition
• Real-world synchronization, overcoming prior reliance on acted or synthetic data

However, future progress can be made by extending the dataset to continuous, untrimmed scenarios to support concept drift adaptation, increasing coverage of natural, unacted anomalies, and supplying finer metadata (ambient context, audio sampling rates). Algorithmically, adaptive fusion techniques, attention-guided weighting between modalities, and temporal event retrieval methods—such as those in ALAN (Wu et al., 2023) or uncertainty-driven fusion in AVadCLIP (Wu et al., 6 Apr 2025)—can be systematically benchmarked using VAAR.

VAAR thus provides a rigorous base for research and practical deployment in audio-visual anomaly recognition, well aligned with advances in cross-modal learning, spatiotemporal event detection, and real-time intelligence for public safety and surveillance systems.