---
title: Weights Task Dataset (WTD) for CPS Analysis
url: https://www.emergentmind.com/topics/weights-task-dataset-wtd
type: topic
---

# Weights Task Dataset (WTD) for CPS Analysis

Searching arXiv for the Weights Task Dataset and prior WTD-based CPS work.
Searching arXiv for Bradford et al. WTD CPS detection work.
The **Weights Task Dataset (WTD)** is a small-group, face-to-face, goal-directed collaborative learning corpus centered on a structured problem-solving activity in which **ten triads** determine the weights of a set of blocks using a balance scale. In recent work, it has served as a benchmark for **Collaborative Problem-Solving (CPS)** detection because it combines group audio/speech interactions, **gold-standard (“Oracle”) segmentation and transcripts**, **Google ASR transcripts** with segmentation based on **Google’s Voice Activity Detector**, and CPS annotations grounded in the framework of **Sun et al. (2020)** [2507.04454]. Within that usage, WTD is not an open-ended discussion dataset; it is a situated collaborative corpus designed around a shared physical task and utterance-level CPS analysis.

## 1. Corpus scope and collaborative setting

WTD, as described in the recent automation study, consists of **ten triads**, each engaged in a structured collaborative problem-solving task that requires participants to infer the weights of blocks by using a balance scale. This makes the corpus a dataset of **triadic collaboration** rather than dyadic tutoring, monologic explanation, or free conversation. Its interactional content is therefore shaped by coordination around a concrete external artifact and a common goal, which is central to the type of CPS behaviors annotated in the corpus [2507.04454].

The same study treats WTD as its **sole experimental testbed**, which underscores the dataset’s role as a reference corpus for CPS detection rather than as one example among many. The dataset includes **group audio/speech interactions**, manually annotated Oracle segmentation and transcripts, automatically generated Google ASR transcripts, and CPS annotations. At the same time, several corpus-level details are not reported there: the paper does **not** provide participant demographics, age, educational level, session durations, total audio duration, or a native train/validation/test partition for the dataset.

## 2. Annotation framework and label structure

WTD’s annotation scheme is grounded in the CPS framework of **Sun et al. (2020)** and contains **19 distinct markers of collaborative behavior**. In the cited study, those marker-level annotations are collapsed into three higher-level CPS facets: **Constructing Shared Knowledge** (`Const`), **Negotiation and Coordination** (`Neg`), and **Maintaining Team Function** (`Maintain`) [2507.04454].

The prediction problem is therefore formulated not at the original 19-marker granularity but as a **three-label multi-label classification** problem. Each utterance or segment may express more than one facet simultaneously. The paper gives a concrete example: if an utterance contains subcategories such as *Suggests appropriate ideas* and *Confirms understanding*, then that utterance is labeled `Const = 1`, `Neg = 0`, and `Maintain = 0`, because those subcategories belong to the Constructing Shared Knowledge facet.

This aggregation has methodological consequences. It simplifies the output space and permits facet-level AUROC reporting, but it also removes some of the original annotation granularity. A plausible implication is that WTD supports both fine-grained discourse annotation and coarser CPS-facet prediction, but the cited study evaluates only the latter.

## 3. Segmentation regimes and units of analysis

The unit of analysis in WTD is the **utterance/segment**. Under Oracle conditions, this is a **manually segmented utterance**. Under automated conditions, it is an automatically segmented speech segment produced by **Google’s VAD-based pipeline**. This distinction is central because the paper’s main question is whether changes in transcription and segmentation alter downstream CPS detection or only the form in which the same interaction is represented [2507.04454].

| Segmentation regime | Utterances | `Const` / `Neg` / `Maintain` |
|---|---:|---:|
| Oracle segmentation | 2482 | 906 / 866 / 391 |
| Google segmentation | 1824 | 664 / 642 / 328 |

The shift from **2482** Oracle utterances to **1824** Google-segmented utterances corresponds to a reduction of  
\[
\frac{2482 - 1824}{2482} \approx 0.265
\]
or **26.5%**. The paper attributes this coarsening largely to segment merging. It reports **518 instances** in which multiple Oracle segments were merged into one Google segment, including **18 cases** in which **more than two Oracle segments** were merged.

A crucial methodological step is the transfer of CPS labels onto automated segments. The Google segments **inherit the labels of all of the Oracle segments that fall under the time span of the Google segment**. This means that a single automatically segmented unit may accumulate several CPS labels from distinct underlying utterances. The result is greater multi-label density, reduced turn-level precision, and less direct access to “who said what and when.”

## 4. Feature extraction, models, and evaluation protocol

The study uses **multimodal utterance-level features** from WTD. Linguistic features are extracted with **BERT-base-small** and prosodic/acoustic features with **openSMILE**; the two feature types are then **concatenated** into a single multimodal representation [2507.04454]. The paper does not specify the exact BERT embedding extraction strategy, the exact openSMILE feature set used in practice, whether features are normalized, or whether speaker identity or diarization features are included.

CPS detection is evaluated with traditional supervised classifiers rather than deep sequence models. The models considered are **Random Forest** and **AdaBoost**, and deep learning models are explicitly not used because the dataset was considered too small to support them without overfitting. Hyperparameters are tuned with **Hyperopt** for **500** search iterations, using **average AUROC** as the optimization target. Tuning is performed only on the **Oracle segmentation + Oracle transcription** condition, and the selected model is then reused across the remaining conditions.

The best model is a **Random Forest** with `criterion = entropy`, `max features = None`, and `number of estimators = 148`. Evaluation uses **leave-one-group-out cross-validation**, which is appropriate because WTD contains ten triads and each fold can leave out one group. Reported metrics are **AUROC**, plus **average precision** and **average recall** across the three CPS facets.

## 5. Reported performance under manual and automatic pipelines

The central empirical comparison is across four pipeline conditions: Oracle segmentation with Oracle transcription, Oracle segmentation with Google transcription, Google segmentation with Oracle transcription, and Google segmentation with Google transcription [2507.04454].

| Segmentation | Transcription | Average AUROC | Avg. Precision | Avg. Recall |
|---|---|---:|---:|---:|
| Oracle | Oracle | **0.744** | **0.704** | 0.297 |
| Oracle | Google | 0.698 | 0.528 | 0.246 |
| Google | Oracle | 0.740 | 0.658 | 0.339 |
| Google | Google | 0.679 | 0.601 | **0.342** |

The **best overall** condition is **Oracle segmentation + Oracle transcription**, with average AUROC **0.744**. The **closest second** is **Google segmentation + Oracle transcription**, at **0.740**. The **worst overall** condition is the fully automated pipeline, **Google segmentation + Google transcription**, with average AUROC **0.679**.

A central finding is that **automated transcription hurts performance more clearly than automated segmentation**. With Oracle segmentation fixed, replacing Oracle transcripts by Google transcripts lowers average AUROC from **0.744** to **0.698**, precision from **0.704** to **0.528**, and recall from **0.297** to **0.246**. By contrast, with Oracle transcripts fixed, replacing Oracle segmentation by Google segmentation changes AUROC only slightly, from **0.744** to **0.740**, while precision drops from **0.704** to **0.658** and recall rises from **0.297** to **0.339**.

The paper interprets this as a trade-off between headline predictive performance and interactional granularity. Automated segmentation can preserve aggregate AUROC while producing fewer, broader units that merge multiple contributions. It also reports facet-level AUROC values showing that `Maintain` is consistently lower than `Const` and `Neg`; this suggests that **Maintaining Team Function** is the hardest facet under the tested feature and model regime.

## 6. Methodological implications and limitations

The paper argues that WTD supports the development of practical CPS monitoring tools because a fully automated pipeline remains feasible even when both transcription and segmentation are automated. At the same time, it emphasizes that educationally useful systems need to preserve not only whether a CPS behavior occurred, but also **what behavior occurred, who produced it, and when**. In that respect, segmentation quality is not a secondary preprocessing detail; it directly shapes interpretability and pedagogical actionability [2507.04454].

Several limitations are made explicit. First, the evaluation is **single-dataset**: only WTD is used, which limits claims about generalization to other collaborative settings, age groups, or task formats. Second, the task itself is **controlled and structured**; the balance-scale activity ensures consistency, but may not represent broader classroom collaboration. Third, the study evaluates a **specific automated segmentation system**, namely Google’s VAD-based segmentation, which is said to **under-segment**; other pipelines might over-segment or behave differently. Fourth, the label-transfer rule is based on **temporal overlap**, so alternative mapping rules could change the results. Fifth, the **26.5% reduction in utterance count** under automated segmentation entails a real loss of granularity even when aggregate AUROC remains comparatively stable.

These limitations imply that WTD is especially strong as a benchmark for **automation sensitivity** in a tightly defined collaborative task. A plausible implication is that its greatest value lies in controlled comparisons of Oracle versus automated speech-processing components, rather than in broad claims about CPS detection in all classroom discourse.

## 7. Relation to prior WTD work and terminological ambiguity

The cited study positions itself relative to earlier WTD-based CPS research, especially **Bradford et al. (2023)**, which showed that CPS markers could be detected using multimodal features in naturalistic small-group settings, as well as WTD dataset papers by **Khebour et al.** and related discourse-annotation and segmentation-aid work by **Terpstra et al. (2023)**. Its distinctive contribution is not a new WTD release, but a four-condition benchmark for testing how **automatic transcription** and **automatic segmentation** affect CPS detection on the same corpus [2507.04454].

The acronym **WTD** should also be distinguished from unrelated uses of “weights task” language in other areas. In the broader arXiv literature, similarly named or informally described “weights task” resources refer to very different objects, including **body weight estimation from pressure images** in “MassNet” [2303.10136], **dataset-size recovery from LoRA weights** in “LoRA-WiSE” [2406.19395], and **longitudinal weight prediction from diet diaries** in “DietDiary” [2408.05445]. Those works concern physiological weight or model-parameter inference, not the collaborative problem-solving corpus described here. This contrast is useful because it clarifies that, in its established educational-computing usage, the **Weights Task Dataset** denotes a CPS-annotated triadic interaction dataset built around a balance-scale block-weight task, not a generic benchmark for learning from numerical or neural-network weights.

Source: https://www.emergentmind.com/topics/weights-task-dataset-wtd