Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection (2003.06780v1)

Abstract: Video anomaly detection is of critical practical importance to a variety of real applications because it allows human attention to be focused on events that are likely to be of interest, in spite of an otherwise overwhelming volume of video. We show that applying self-trained deep ordinal regression to video anomaly detection overcomes two key limitations of existing methods, namely, 1) being highly dependent on manually labeled normal training data; and 2) sub-optimal feature learning. By formulating a surrogate two-class ordinal regression task we devise an end-to-end trainable video anomaly detection approach that enables joint representation learning and anomaly scoring without manually labeled normal/abnormal data. Experiments on eight real-world video scenes show that our proposed method outperforms state-of-the-art methods that require no labeled training data by a substantial margin, and enables easy and accurate localization of the identified anomalies. Furthermore, we demonstrate that our method offers effective human-in-the-loop anomaly detection which can be critical in applications where anomalies are rare and the false-negative cost is high.

• The paper introduces an end-to-end deep ordinal regression model that learns anomaly scores without the need for manual labeling.
• It formulates anomaly detection as a two-class ordinal regression task, iteratively refining pseudo labels using state-of-the-art unsupervised methods.
• The approach yields significant AUC improvements across real-world datasets and supports human-in-the-loop refinement for precise anomaly localization.

Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection

The paper, "Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection," addresses the problem of video anomaly detection, emphasizing the necessity of methods that do not require labeled data, which is often expensive to obtain. The authors propose a novel approach by leveraging a self-trained deep ordinal regression framework to detect anomalies within video sequences. This method aims to overcome the two primary limitations of the existing unsupervised techniques: dependence on manually labeled normal data and sub-optimal feature learning.

The authors introduce an end-to-end trainable model that merges feature representation learning with anomaly scoring, effectively eliminating the need for manual labeling of normal and abnormal video frames. The approach is formulated as a two-class ordinal regression task that utilizes a self-training mechanism. This design allows for the automatic learning and optimization of feature representations tailored specifically for anomaly detection tasks in video data.

The method involves three key components: initial anomaly detection, end-to-end anomaly score learning, and iterative learning via self-training. Initially, anomaly scores for video frames are generated using a combination of state-of-the-art unsupervised methods, such as Sp and iForest. These combined scores initialize two sets of pseudo-labeled data — normal and anomalous frames. The end-to-end anomaly score learner employs a neural network that integrates a ResNet-50-based feature extractor and a fully connected network for scoring. The iterative learning approach further refines the model by updating pseudo labels based on newly optimized anomaly scores and retraining the model iteratively to enhance its performance.

The paper demonstrates the effectiveness of the proposed approach through extensive experiments on three real-world datasets: UCSD, Subway, and UMN. The proposed method consistently outperforms existing unsupervised methods across various scenarios, showing significant improvements in AUC scores. Moreover, the model's design supports human-in-the-loop anomaly detection, allowing for interactive refinement of anomaly scores based on expert feedback. The end-to-end framework also enables the generation of saliency maps, facilitating precise localization of anomalies in video frames.

In exploring the broader implications of this research, the paper suggests that extending the model to include features such as motion could enhance its versatility in detecting various anomaly types. Given the increasing volume of video data across numerous fields, this model provides a scalable solution to anomaly detection without the burden of manual data labeling, thus broadening its applicability in areas like surveillance, industrial monitoring, and automated systems.

The proposed method marks a significant advancement in unsupervised anomaly detection frameworks, combining effective unsupervised learning strategies with deep learning's capacity for feature optimization. However, future investigations might focus on the model's adaptability to different kinds of anomalies beyond appearance-based deviance, possibly integrating advanced features such as temporal dynamics and interaction-based cues. As artificial intelligence continues to evolve, research in robust and adaptive unsupervised learning methods will remain crucial to leveraging large, unlabeled datasets across various domains.