Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge (1709.09121v1)

Abstract: This paper addresses the problem of joint detection and recounting of abnormal events in videos. Recounting of abnormal events, i.e., explaining why they are judged to be abnormal, is an unexplored but critical task in video surveillance, because it helps human observers quickly judge if they are false alarms or not. To describe the events in the human-understandable form for event recounting, learning generic knowledge about visual concepts (e.g., object and action) is crucial. Although convolutional neural networks (CNNs) have achieved promising results in learning such concepts, it remains an open question as to how to effectively use CNNs for abnormal event detection, mainly due to the environment-dependent nature of the anomaly detection. In this paper, we tackle this problem by integrating a generic CNN model and environment-dependent anomaly detectors. Our approach first learns CNN with multiple visual tasks to exploit semantic information that is useful for detecting and recounting abnormal events. By appropriately plugging the model into anomaly detectors, we can detect and recount abnormal events while taking advantage of the discriminative power of CNNs. Our approach outperforms the state-of-the-art on Avenue and UCSD Ped2 benchmarks for abnormal event detection and also produces promising results of abnormal event recounting.

• The paper introduces a unified framework that simultaneously detects and recounts abnormal events, reducing false alarms in surveillance systems.
• It leverages a multi-task Fast R-CNN trained on large-scale datasets to extract rich semantic features for accurate anomaly detection.
• Empirical results demonstrate significant performance gains, achieving an AUC of 89.2% on UCSD Ped2 compared to state-of-the-art methods.

Overview of the Paper: Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge

The paper "Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge" presents a novel approach to the problem of abnormal event detection and recounting in video surveillance. The researchers propose a framework that integrates generic knowledge of visual concepts with environment-specific knowledge, leveraging convolutional neural networks (CNNs) for both tasks. This integration addresses a noteworthy challenge in utilizing CNNs for anomaly detection, given the varying definitions of normalcy across different environments.

Key Contributions

The paper introduces several significant contributions to the field:

1. Integration of Detection and Recounting: The framework proposed by the authors facilitates the simultaneous detection and recounting of abnormal events. Recounting refers to the system's ability to explain why detected events are classified as abnormal, which is crucial for distinguishing false alarms from genuine alerts in surveillance systems.
2. Generic Knowledge Acquisition: The method involves training a multi-task Fast R-CNN model on large-scale supervised datasets to acquire generic knowledge about visual concepts, including objects, actions, and attributes. This model captures semantic information that enhances both detection and recounting tasks.
3. Environment-specific Anomaly Detectors: The generic CNN model is complemented by environment-dependent anomaly detectors that learn normal behavior from training data. These detectors are applied to the semantic features and classification scores derived from the CNN model to identify anomalies in test samples.
4. Empirical Validation: The authors demonstrate the superiority of their method compared to state-of-the-art techniques on standard benchmarks, specifically the Avenue and UCSD Ped2 datasets. Notably, their approach achieves remarkable performance improvements, with an AUC of 89.2% on the UCSD Ped2 dataset.

Methodology

The proposed framework consists of several components:

• Generic Model Training: A multi-task Fast R-CNN is learned using labeled image data sets like Microsoft COCO and Visual Genome. This model is tasked with classifying objects, actions, and attributes, offering a robust feature representation that is relevant to detecting and recounting abnormal events.
• Detection Process: For each frame in the video, object proposals are generated, and semantic features along with classification scores are extracted using the multi-task Fast R-CNN model. Anomaly detectors then classify these features to yield anomaly scores, identifying abnormal events.
• Recounting Process: The recounting procedure involves predicting the categories of detected events and computing anomaly scores for these predictions, using kernel density estimation to model the distribution of classification scores.

Numerical Results and Evaluation

The paper highlights strong numerical results, outperforming previous methods on the Avenue and UCSD Ped2 benchmarks in both frame-level and pixel-level detection metrics. Specifically, the approach significantly advances AUC metrics, demonstrating its effectiveness in identifying and recounting abnormal events.

Implications and Future Directions

The integration of generic CNN-based knowledge in event recounting paves the way for more nuanced and context-aware surveillance systems. The ability to not only detect anomalies but also explain them enriches the interpretability of anomaly detection models, which is critical for real-world applications.

Future research could explore incorporating additional types of knowledge, such as interactions between objects, into the framework. Additionally, extending the approach to handle video data from moving cameras, or leveraging motion information through techniques like two-stream CNNs or 3D-CNNs, could address some limitations in capturing dynamic abnormalities.

Overall, the paper delivers a comprehensive and methodically sound contribution to video surveillance, enhancing both practical surveillance applications and theoretical understanding of anomaly detection systems.