Learning Deep Representations of Appearance and Motion for Anomalous Event Detection (1510.01553v1)

Abstract: We present a novel unsupervised deep learning framework for anomalous event detection in complex video scenes. While most existing works merely use hand-crafted appearance and motion features, we propose Appearance and Motion DeepNet (AMDN) which utilizes deep neural networks to automatically learn feature representations. To exploit the complementary information of both appearance and motion patterns, we introduce a novel double fusion framework, combining both the benefits of traditional early fusion and late fusion strategies. Specifically, stacked denoising autoencoders are proposed to separately learn both appearance and motion features as well as a joint representation (early fusion). Based on the learned representations, multiple one-class SVM models are used to predict the anomaly scores of each input, which are then integrated with a late fusion strategy for final anomaly detection. We evaluate the proposed method on two publicly available video surveillance datasets, showing competitive performance with respect to state of the art approaches.

• The paper presents AMDN, a novel framework utilizing deep autoencoders to fuse appearance and motion cues for effective unsupervised video anomaly detection.
• It employs a double fusion strategy that integrates pixel-level early fusion with decision-level late fusion to capture comprehensive video features.
• Evaluation on UCSD Ped datasets shows competitive performance with notable AUC and EER metrics, highlighting the framework’s practical surveillance potential.

Learning Deep Representations of Appearance and Motion for Anomalous Event Detection

This paper presents a method for unsupervised anomalous event detection in video streams using a novel framework called Appearance and Motion DeepNet (AMDN). This framework seeks to overcome the limitations inherent in traditional techniques which rely on hand-crafted feature extraction for video surveillance applications. AMDN utilizes deep learning, specifically stacked denoising autoencoders (SDAEs), to learn feature representations directly from data, enhancing the ability to detect anomalies in complex video scenes.

Overview of AMDN Framework

The authors propose a double fusion strategy within the AMDN framework, combining both early and late fusion techniques. Initially, separate pipelines using SDAEs are employed to learn appearance and motion features from video data. These representations are then integrated at the pixel level (early fusion) to form a joint representation which captures the interplay between static and dynamic elements in the video.

In order to perform anomaly detection, the system applies multiple one-class SVM models to these learned representations. The anomaly scores derived from these models are combined using a late fusion strategy to enhance detection accuracy. The approach is evaluated on two public datasets, demonstrating competitive performance against existing state-of-the-art methods.

Methodological Insights and Numerical Results

The AMDN system employs a structured approach to feature learning and anomaly detection:

• Separate Pipeline Learning: Appearance and motion representations are learned using individual SDAE networks. The joint representation integrates these insights at a higher abstraction level.
• Fusion Strategies: Double fusion is achieved by combining pixel-level early fusion and decision-level late fusion techniques, which allows for comprehensive modeling of both appearance and motion cues.

Numerically, the approach achieves notable results:

• In the UCSD Ped1 dataset, AMDN demonstrated an Equal Error Rate (EER) of 16% for frame-level detection and 40.1% for pixel-level detection. The Area Under Curve (AUC) values were 92.1% (frame) and 67.2% (pixel), respectively.
• For the UCSD Ped2 dataset, AMDN achieved an EER of 17% with an AUC of 90.8%, underscoring the robustness of the proposed double fusion model in varying scenarios.

Implications and Future Directions

The AMDN framework makes a compelling case for the effectiveness of unsupervised deep learning in video anomaly detection by demonstrating high accuracy without the need for handcrafted features. This method not only caters to the intricacies of complex video scenes but also offers scalability with automated feature learning.

The results propose significant implications for practical video surveillance, providing a path toward systems capable of real-time anomaly detection with reduced manual intervention. Theoretically, this work adds depth to the exploration of unsupervised deep learning methodologies and their application in computer vision tasks.

Future exploration could entail expanding upon the deep learning architectures employed, considering alternate multimodal fusion techniques, and leveraging multi-task learning for heterogeneous anomaly detection in diverse video environments. Such developments could further refine the capability and precision of automated surveillance systems, catering to complex and crowded scenarios with varying anomaly definitions.