Eye in the Sky: Real-time Drone Surveillance System (DSS) for Violent Individuals Identification using ScatterNet Hybrid Deep Learning Network (1806.00746v1)

Abstract: Drone systems have been deployed by various law enforcement agencies to monitor hostiles, spy on foreign drug cartels, conduct border control operations, etc. This paper introduces a real-time drone surveillance system to identify violent individuals in public areas. The system first uses the Feature Pyramid Network to detect humans from aerial images. The image region with the human is used by the proposed ScatterNet Hybrid Deep Learning (SHDL) network for human pose estimation. The orientations between the limbs of the estimated pose are next used to identify the violent individuals. The proposed deep network can learn meaningful representations quickly using ScatterNet and structural priors with relatively fewer labeled examples. The system detects the violent individuals in real-time by processing the drone images in the cloud. This research also introduces the aerial violent individual dataset used for training the deep network which hopefully may encourage researchers interested in using deep learning for aerial surveillance. The pose estimation and violent individuals identification performance is compared with the state-of-the-art techniques.

• The paper presents a SHDL-based approach that improves human pose estimation under aerial conditions by extracting invariant features.
• It employs a Feature Pyramid Network and cloud offloading to achieve real-time detection of violent activities in public spaces.
• The study introduces the AVI dataset of 2000 annotated images, enhancing evaluations and advancing aerial surveillance research.

Real-time Drone Surveillance System for Violent Individuals Identification

This paper discusses the development and implementation of a real-time drone surveillance system designed to identify violent individuals in public spaces. The system relies on a Feature Pyramid Network (FPN) for human detection and a ScatterNet Hybrid Deep Learning (SHDL) network for pose estimation to identify violence-related activities effectively. This approach addresses key challenges associated with drone surveillance, including the variations in human size, orientation, and positioning due to aerial perspective, as well as issues such as illumination changes and image blurring.

The proposed system is notable for its utilization of SHDL, which incorporates a hand-crafted ScatterNet front-end and a coarse-to-fine regression network back-end. The ScatterNet accelerates learning by extracting invariant features, which facilitates efficient training with fewer labeled examples—a significant consideration given the complexity and cost of acquiring annotated datasets. This capability is advantageous for tasks such as aerial surveillance where annotated datasets are scarce and expensive to produce.

Key Contributions and Findings

• Human Pose Estimation: The proposed system effectively utilizes the SHDL network for human pose estimation, showing a notable improvement over previous methods. The network benefits from the ScatterNet's capacity to generate translation, rotation, and scale-invariant features, enabling it to better handle the diverse challenges inherent in aerial imagery. The integration of structural priors further accelerates learning, allowing for more efficient training and better performance compared to existing methodologies.
• Real-time Identification: By offloading the computation-intensive tasks to the cloud, the system achieves real-time identification of violent activities. This approach addresses the limitations of local drone processing power and provides flexibility in resource scaling as needed for robust performance.
• Aerial Violent Individual (AVI) Dataset: The paper introduces the AVI dataset, which comprises 2000 annotated images capturing a variety of violent activities such as punching, stabbing, and shooting. This dataset provides a valuable resource for researchers focused on advancing techniques in aerial surveillance using deep learning.

Performance Evaluation

The SHDL network's performance surpasses existing pose estimation techniques, such as CoordinateNet and SpatialNet, achieving an accuracy of 87.6% for a 5-pixel threshold when identifying key body points. Furthermore, the proposed system accomplished high classification accuracies for various violent activities within the AVI dataset, demonstrating significant improvements over prior approaches. Even with increasing numbers of individuals in the frame, the system maintains commendable accuracy, highlighting its robustness.

Implications and Future Work

The implications of this research are significant for the domains of law enforcement and public safety. As the need for efficient large-scale surveillance grows, such technology can aid in preemptively identifying threats and promptly responding to violent incidents. However, the expansion of surveillance capabilities also necessitates careful consideration of privacy and ethical concerns.

Future developments could explore integrating more advanced AI techniques, enhancing the scalability and accuracy of the system. Additionally, the expansion of the AVI dataset with more diverse and complex scenarios could enable further refinement of the model's capability to handle extensive real-world variations. This research lays a foundation for more sophisticated and responsive drone-based surveillance systems that could be pivotal in safeguarding public spaces.