SCB-Dataset3: Classroom Behavior Detection

• SCB-Dataset3 is a large-scale visual benchmark with fine-grained, six-class behavior annotations for detecting student actions in K–12 and university classrooms.
• It integrates diverse image sources and an iterative train–detect–correct annotation process to overcome real-world challenges like occlusion and class imbalance.
• Robust YOLO-based performance baselines and high mAP scores, particularly for hand-raising and phone use, validate its effectiveness for automated classroom behavior analysis.

SCB-Dataset3 is a publicly available, large-scale visual benchmark specifically constructed for the detection of student behaviors in classroom environments. Designed by Yang Fan, Wang Tao et al., its principal aim is to provide a realistic, well-annotated corpus suitable for evaluating object-detection-based recognition methods of pedagogically relevant behaviors across educational stages, spanning kindergarten to university classroom settings. The dataset addresses key shortcomings in prior work, such as limited real-world classroom diversity, insufficient class granularity, and the methodological limitations of purely action-recognition or pose-estimation-based pipelines. Through comprehensive annotation protocols and strong performance baselines, SCB-Dataset3 establishes itself as a foundational resource for empirical development and assessment in automated classroom behavior analysis (Yang et al., 2023).

1. Motivation, Scope, and Research Goals

The development of SCB-Dataset3 is motivated by the need to obtain objective, scalable, and fine-grained measurements of classroom student behavior. Manual observation is labor-intensive, especially in crowded scenes, and real-time inference of engagement or problematic behaviors is impractical without automation. While video-based action recognition and pose-estimation methods can capture temporal context, they are bottlenecked by annotation demands and frequent failure in high-density, visually ambiguous scenes. Object-detection methods—specifically those derived from the YOLO family—provide a principled framework for recognizing discrete, anatomically interpretable behaviors at the per-student bounding-box level.

By supplying a large and diverse set of annotated classroom images (n=5,686; m=45,578 boxes), carefully defined six-way behavior taxonomy, and rigorous object-level ground truth, SCB-Dataset3 directly addresses these gaps and enables both baseline and frontier computer vision methods to be evaluated under realistic constraints (Yang et al., 2023).

2. Dataset Composition and Class Taxonomy

SCB-Dataset3 consists of 5,686 images accompanied by 45,578 object-level bounding-box annotations. It merges two core data sources: SCB3-S (“school”, K–12) and SCB3-U (“university”), distinguished by grade level and data collection protocols. The six annotated behaviors, each corresponding to a distinct class label, are:

• Hand-raising: Arm extended upward to draw attention.
• Reading: Head inclined with eyes on a book or worksheet.
• Writing: Hand active with pen/pencil in contact with paper.
• Using a phone: Hands manipulating a mobile device, positioned in front of face or on the desk.
• Bowing the head: Head lowered/resting on desk or strongly downcast.
• Leaning over the table: Torso or shoulders angled forward past the chair edge.

Class distributions exhibit substantial variation across educational stages, e.g., hand-raising is frequent in SCB3-S but nearly absent in SCB3-U, where “using phone” is predominant. Precise bounding-boxes are drawn for each student instance manifesting one of these behaviors.

3. Data Collection and Annotation Protocols

Images in SCB3-S are sourced from over 1,000 online educational video recordings (platforms: bjyhjy, 1s1k, youke, etc.), with 3–15 frames sampled at fixed intervals per video. Manual annotation is performed in an iterative train–detect–correct loop: initial hand-labeled subsets are used to train a YOLOv7 model, preliminary predictions on new data are visually inspected and corrected, and refined labels are reincorporated for further training. This workflow increases annotation throughput while maintaining quality.

For SCB3-U, where authentic classroom footage is less common, one frame per second is selected, with labeling performed on every second frame (2-second interval). Frame interpolation is used to project annotations onto the intermediate, unlabeled frames. Annotation protocols enforce consistent class definition and bounding-box tightness; though formal inter-annotator metrics (e.g., kappa) are not reported, the iterative correction process enforces quality control (Yang et al., 2023).

4. Statistical Breakdown and Splitting Strategy

Aggregate statistics for SCB-Dataset3 and its subcomponents are as follows:

Subset	Images	Hand-raising	Reading	Writing	Using phone	Bowing head	Leaning over
SCB3-S	–	11,207	10,841	3,762	–	–	–
SCB3-U	–	6	7,826	2,984	6,976	947	1,029

Split protocol is 80% training and 20% validation; no separate test set is defined. Validation metrics serve as the de facto held-out assessment. Notably, class imbalance is pronounced in the university-level subset, with hand-raising virtually absent and “using phone” forming the dominant category.

5. Evaluation Methodology and Baseline Performance

Experiments leverage the YOLO detection framework, evaluating YOLOv5 (n, s, m, l, x), YOLOv7 (std, x, w6), and YOLOv8 (n, s, m, l, x) architectures. Training is configured on Ubuntu 20.04 with PyTorch 1.11, CUDA 11.3, and OpenCV 4.1.2 using an RTX 3080 GPU. All inputs are resized to 640×640; batch size is 8; epochs = 100 with COCO-pretrained initialization; standard data augmentation (scaling, flip, color jitter) is used per Ultralytics YOLO implementations.

The primary metric is mean Average Precision at an IoU threshold of 0.5 ([email protected]), with the following standard definitions: $$\mathrm{Precision} = \frac{TP}{TP + FP},\quad \mathrm{Recall} = \frac{TP}{TP + FN}$$

$$AP_c = \int_0^1 P_c(r) dr, \quad mAP = \frac{1}{C} \sum_{c=1}^C AP_c$$

Headline results for SCB3-S (K–12 subset) are as follows:

Model	[email protected]
YOLOv5n	71.1
YOLOv5s	72.9
YOLOv5m	74.7
YOLOv5l	76.0
YOLOv5x	74.9
YOLOv7	77.2
YOLOv7x	80.3
YOLOv7w6	80.0
YOLOv8n	72.4
YOLOv8s	74.5
YOLOv8m	77.6
YOLOv8l	76.8
YOLOv8x	76.4

Per-class [email protected] for YOLOv7x on SCB3-S:

• Hand-raising: 87.7%
• Reading: 79.9%
• Writing: 73.2%

For SCB3-U, frame interpolation leads to substantially improved mAP (e.g., for YOLOv7x: all-classes [email protected] improves from 71.4% before interpolation to 94.4% after; hand-raising class improves from 0.0% to 99.5%) (Yang et al., 2023).

6. Qualitative Results, Limitations, and Open Problems

Successful recognitions typically occur for well-separated students with prototypical postures. Failure modes include dense seating-induced occlusion, pose ambiguity (notably reading vs. writing when head and hand positions are similar), and partial side/back views. The iterative train–detect–correct cycle and interpolation ameliorate, but do not fully resolve, these issues.

Open challenges remain:

• Severe class imbalance (particularly insufficient university hand-raising samples, overrepresentation of “using phone”),
• High intra-class visual diversity across ages/environments,
• Inter-class similarity (especially reading vs. writing).

Future research avenues include expanding university-level coverage, leveraging temporal context via spatio-temporal architectures, class-balancing via re-weighted loss, and multi-task models for joint behavior and identity prediction to enforce timeline consistency.

7. Dataset Access and Integration Guidelines

SCB-Dataset3 is publicly distributed via https://github.com/Whiffe/SCB-dataset and includes all necessary annotation files in YOLO format. Integration protocol involves:

1. Cloning the repository,
2. Installing dependencies (Python 3.8, PyTorch 1.11, CUDA 11.3),
3. Positioning image and label directories as specified,
4. Editing the provided YAML data files for desired splits,
5. Invoking training (e.g., python train.py --img 640 --batch 8 --epochs 100 --data scb3.yaml --cfg yolov7x.yaml --weights yolov7x.pt).

This resource—combining realistic, real-world diversity, comprehensive labels, and validated YOLO baselines—constitutes a robust empirical substrate for research on detection and analysis of classroom student behaviors (Yang et al., 2023).