Student Classroom Behavior Detection based on Improved YOLOv7 (2306.03318v2)

Abstract: Accurately detecting student behavior in classroom videos can aid in analyzing their classroom performance and improving teaching effectiveness. However, the current accuracy rate in behavior detection is low. To address this challenge, we propose the Student Classroom Behavior Detection method, based on improved YOLOv7. First, we created the Student Classroom Behavior dataset (SCB-Dataset), which includes 18.4k labels and 4.2k images, covering three behaviors: hand raising, reading, and writing. To improve detection accuracy in crowded scenes, we integrated the biformer attention module and Wise-IoU into the YOLOv7 network. Finally, experiments were conducted on the SCB-Dataset, and the model achieved an [email protected] of 79%, resulting in a 1.8% improvement over previous results. The SCB-Dataset and code are available for download at: https://github.com/Whiffe/SCB-dataset.

• The paper introduces a novel approach by modifying YOLOv7 with Bi-Level Routing Attention and Wise-IoU, boosting [email protected] to 79%.
• The study uses a custom SCB-Dataset with 4.2k images and 18.4k annotations covering actions like hand raising, reading, and writing.
• The enhancements improve detection in complex, crowded classroom environments, enabling real-time feedback and advanced pedagogical analysis.

Analysis of Student Classroom Behavior Detection Based on Improved YOLOv7

In this paper, the authors present a novel approach to detecting student behaviors in classroom settings by leveraging an enhanced version of the YOLOv7 object detection network. The challenge addressed is the low accuracy of traditional behavior detection in classroom videos, which can hinder assessments of classroom performance and teaching effectiveness. The paper's contribution lies in introducing the Student Classroom Behavior Dataset (SCB-Dataset) coupled with modifications to the YOLOv7 architecture, yielding improved detection accuracy.

The SCB-Dataset, specifically constructed for this research, includes 18.4k labels across 4.2k images, capturing crucial student behaviors, namely hand raising, reading, and writing. The paper highlights the complexity this dataset presents due to varied environments, perspective angles, and dense scenes in classrooms. It serves as a significant step forward, providing a substantial volume of annotated data, which has been lacking in educational behavior detection research.

The proposed methodology revolves around augmenting the original YOLOv7 model with the integration of Bi-Level Routing Attention (BRA) and Wise Intersection over Union (Wise-IoU). These enhancements aim to address specific deficiencies in object detection performance, specifically misidentifying actions and errors in bounding boxes within crowded and occluded scenes.

Methodological Improvements

1. Bi-Level Routing Attention (BRA): The BRA module introduces dynamic sparse attention, which enables selective focus on relevant regions of the input, thus enhancing detection accuracy. This is particularly useful in complex classroom environments where distinguishing between similar behaviors can be challenging.
2. Wise-IoU Loss Functions: The paper experiments with different versions of the Wise-IoU loss function (v1, v2, and v3) designed to dynamically modulate learning focus based on the outlier status of anchor boxes. This adjustment counters the drawbacks of traditional IoU methods, especially in handling low-quality examples common in classroom datasets.

Experimental Results

The comprehensive experimental evaluation demonstrates an [email protected] improvement to 79%, marking a 1.8% enhancement compared to the unmodified YOLOv7. Notably, the precision and [email protected]:0.95 were also improved significantly, indicating the effectiveness of the modifications. The experiments utilized a potent computational setup, ensuring the proposed system's feasibility in real-time applications.

Implications and Future Directions

This research provides an important tool for educational institutions looking to harness technology for improved teaching quality and student engagement insights. The improved detection accuracy can facilitate real-time feedback mechanisms and bolster analytical capabilities in educational settings. Future investigations may explore extending the dataset to incorporate additional behavioral categories and refining the model to ensure broader applicability across diverse educational contexts.

The approach is grounded in both theoretical and practical advancements, emphasizing the importance of customized datasets and tailored model modifications for domain-specific object detection challenges. The methods and results delineated in this paper will likely inform subsequent research in educational AI applications, warranting further exploration into adaptive attention mechanisms and dynamic loss functions.