---
title: 'SCB-ST-Dataset4: Classroom Behavior Corpus'
url: https://www.emergentmind.com/topics/scb-st-dataset4
type: topic
---

# SCB-ST-Dataset4: Classroom Behavior Corpus

SCB-ST-Dataset4 is a large-scale, spatio-temporal video corpus specifically constructed for the automatic detection and analysis of student classroom behaviors using deep learning. It provides a high-fidelity foundation for benchmarking temporal action detection and multi-model fusion approaches in educational settings. The dataset is centered on three core behaviors—hand-raising, reading, and writing—and is accompanied by detailed annotations, benchmarking protocols, and practical resources for immediate usage in machine learning pipelines [2310.16267].

## 1. Dataset Scope and Statistics

SCB-ST-Dataset4 comprises 757,265 unique video frames and 25,810 labeled spatio-temporal clips distributed across three classroom behaviors: hand-raising (class 0), reading (class 1), and writing (class 2). Each clip is a 5-second window at 30 fps, corresponding to 150 contiguous frames, resulting in approximately 3.87 million total frames when considering replication across clips.

**Annotation Distribution:**

| Behavior      | #Annotations |
|:--------------|:------------|
| Hand-raising  | 11,207      |
| Reading       | 10,841      |
| Writing       | 3,762       |

The dataset is a culmination in a series of increasing-scale corpora, with SCB-ST-Dataset4 manifestly outscaling its predecessors both in sample size and diversity:

| Dataset            | #Classes | #Images  | #Annotations |
|:-------------------|:--------:|:--------:|:-------------|
| SCB-Dataset1       | 1        | 4,001    | 11,248       |
| SCB-Dataset2       | 3        | 4,266    | 18,499       |
| SCB-Dataset3-U     | 6        | 671      | 19,768       |
| **SCB-ST-Dataset4** | 3      | 757,265  | 25,810       |

Every labeled instance seeds a 150-frame video clip; where contiguous frames are unavailable, the key frame is temporally padded by replication. The class distribution among these clips adheres to the annotation counts specified above.

## 2. Construction and Annotation Protocol

SCB-ST-Dataset4 was created via a fully automated, two-stage pipeline leveraging both human annotation and machine-generated pseudo-labels, with the explicit aim of avoiding additional manual bounding-box labeling during the spatio-temporal extension phase. The construction proceeds as follows:

**Stage A: Image-level Detector Training and Label Expansion**
1. Approximately 1,000 full-length classroom videos (∼40 minutes each) were drawn from designated sources ("bjyhjy", "1s1k", "youke", "qlteacher", "youke-smile.shec").
2. 3–15 frames per video were uniformly sampled and manually labeled for one of the three target behaviors.
3. YOLOv7 was trained on these key annotated frames.
4. The trained detector generated labels on remaining frames, followed by rapid human-in-the-loop correction and fine-tuning.
5. The detector was retrained on the expanded annotation set.

**Stage B: Spatio-Temporal Clip Generation**
- For each annotated frame at time $t$ in video $v$, a consecutive 150-frame clip spanning $[t-60, t+89]$ was extracted, or padded by replicating the key frame when insufficient pre/post frames existed.
- The bounding boxes from the single annotated frame were propagated identically to all frames in the clip, with no additional manual intervention.

**Pseudocode Summary:**
```
for each (v, t, boxes) in L:
   if video v length ≥ t+89 and t ≥ 60:
       clip_frames = read_frames(v, t-60 … t+89)
   else:
       frame_f = read_frame(v, t)
       clip_frames = [frame_f]*60 + [frame_f]*90
   clip_annotations = [boxes] * 150
   Clips.append((clip_frames, clip_annotations))
```
No new manual labeling occurs after initial image-level annotations; all spatio-temporal annotation is obtained by box propagation.

## 3. Behavior Similarity Index

To quantify the visual ambiguity between target behaviors, the dataset introduces the Behavior Similarity Index (BSI). For two classes $i$ and $j$:
- $N_i$: total samples in class $i$
- $N_j$: total samples in class $j$
- $S_{i,j}$: total number of frames where class $i$ and $j$ boxes overlap.

The pairwise BSI is defined as:
$$
\mathrm{BSI}(i,j) = \left( \frac{S_{i,j}}{N_i}, \frac{S_{i,j}}{N_j} \right)
$$
$\mathrm{BSI}(i,j).0$ indicates the fraction of $i$'s samples overlapping $j$, and $\mathrm{BSI}(i,j).1$ vice versa. High BSI values (approaching (1,1)) indicate near-indistinguishability. In practice, BSI for reading vs. writing ranged from (4–5%, 12–19%) across evaluated YOLO models, indicating a meaningful degree of visual overlap [2310.16267].

## 4. Baseline Benchmarks and Performance

SCB-ST-Dataset4 serves as a rigorous testbed for both image-based and spatio-temporal models. Object detection baselines—YOLOv5 (small/medium), YOLOv7 (standard/x), YOLOv8 (small/medium)—and the dual-stream SlowFast architecture were evaluated.

**Aggregate and Per-class Results:**

| Model    | mAP@50 (%) | Hand-raise (AP@50) | Reading (AP@50) | Writing (AP@50) |
|:---------|:----------:|:-------------------|:---------------|:---------------|
| YOLOv5s  |    72.9    |       —            |      —         |      —         |
| YOLOv5m  |    74.7    |      85.4          |    75.2        |    63.3        |
| YOLOv7   |    77.2    |       —            |      —         |      —         |
| YOLOv7x  |    80.3    |      86.8          |    77.4        |    70.0        |
| YOLOv8s  |    74.5    |       —            |      —         |      —         |
| YOLOv8m  |    77.6    |      84.6          |    76.2        |    66.8        |
| **SlowFast** | **82.3** | 96.9           | 89.8           | 57.8           |

SlowFast achieves the best overall performance, with $>80\%$ mAP in under 10 epochs, outperforming purely spatial detectors (YOLOv7x attains $\sim 80\%$ mAP after 100 epochs). Class-wise, the highest precision is attained for hand-raising, with the lowest for writing, reflecting the inherent visual ambiguity in these behaviors.

## 5. Multi-Model Fusion for Student Profiling

To support enriched analytics, a multi-modal fusion strategy augments behavior detection with identity, pose, and affective information using the following parallelized streams:

- **Continuous (30 fps):**
  - YOLOv7 (CrowdHuman-fine-tuned) for crowd/person boxes.
  - DeepSort for multi-object tracking and persistent IDs.
  - SlowFast for per-frame behavior classification.
- **Non-Continuous (1 fps):**
  - YOLOv7 for student/teacher role distinction.
  - SynergyNet for head-pose estimation.
  - A facial expression model to render one of seven emotion classes.

All outputs are time-synchronized with per-person IDs and stored in a unified format:
\{student\_ID, timestamp, behavior, pose, identity, expression\}

This fusion framework enables complex downstream analytics, such as concentration scoring and behavioral profiling. The YOLOv7 CrowdHuman-fine-tuned model achieves 85.6% mAP@50 on crowd detection [2310.16267].

## 6. Data Access and Usage

SCB-ST-Dataset4 is publicly available at [https://github.com/Whiffe/SCB-dataset](https://github.com/Whiffe/SCB-dataset). The repository contains:
- `clips/`: video segments (e.g., .mp4 or .avi), each 150 frames.
- `annotations/`: per-clip annotations in text or JSON.
- `train/`, `val/`, `test/`: splits for standard machine learning workflows.

**Example: Download and Basic Usage**
```bash
git clone https://github.com/Whiffe/SCB-dataset.git
cd SCB-dataset/SCB-ST-Dataset4
ls
```

**PyTorch Dataset Loader Snippet**
```python
class SCBST4(Dataset):
    def __init__(self, root, split='train', transform=None):
        self.root     = root
        self.transform= transform
        split_file   = os.path.join(root, f"{split}.txt")
        self.clips   = [line.strip() for line in open(split_file)]
    def __len__(self):
        return len(self.clips)
    def __getitem__(self, idx):
        clip_name = self.clips[idx]
        imgs = []
        for i in range(150):
            img_path = os.path.join(self.root, 'clips', clip_name, f"{i:03d}.jpg")
            img = Image.open(img_path).convert('RGB')
            if self.transform:
                img = self.transform(img)
            imgs.append(img)
        ann_path = os.path.join(self.root, 'annotations', clip_name + '.txt')
        boxes, labels = load_yolo_txt(ann_path)
        return torch.stack(imgs), boxes, labels
```
Recommended settings for SlowFast training: batch size 2, learning rate 0.1, 43 epochs, input size $640 \times 640$. Pretraining on Kinetics or AVA is advised for optimal convergence. For YOLO models, a typical configuration involves 100 training epochs with pretrained weights [2310.16267].

## 7. Research Impact and Benchmark Use

SCB-ST-Dataset4 addresses the challenge of large-scale, annotated, spatio-temporal classroom behavior data in education AI, enabling rigorous analysis and development of multi-modal detection systems without the costly overhead of manual video labeling. It establishes new standards for benchmarking behavioral detection, introduces the Behavior Similarity Index for quantifying class overlap, and supports multi-perspective analytics via model fusion. Its scale, annotation protocol, and accessibility position it as a foundational resource for future research in automated educational assessment and computer vision in classroom environments [2310.16267].

Source: https://www.emergentmind.com/topics/scb-st-dataset4