Candy Speech Detection: Text and Audio

• Candy Speech Detection is the automated identification of positive online discourse in texts and candy-related keywords in audio.
• It employs span-based token classification for text and keyword spotting with CNN/RNN models for speech to enhance detection accuracy.
• Evaluations on German YouTube comments and speech commands show that span supervision and multilingual, emoji-aware tokenization significantly boost performance.

Candy speech detection denotes the automated identification of positive, supportive online communication, particularly in social-media comments. In the GermEval 2025 setting, “candy speech” refers to comments or spans that express supportive, affiliative, or otherwise positive interpersonal content in German YouTube discussions, and the central technical problem is to detect such content either at the comment level or as character-aligned spans within comments (Thelen et al., 9 Sep 2025). In a distinct keyword-spotting usage, the same label is applied to a speech-recognition pipeline that listens for candy-related words such as “candy,” “chocolate,” or “lollipop,” reusing architectures from command recognition rather than social-media NLP (Rai et al., 2023).

1. Definition and terminological scope

The GermEval formulation defines candy speech as positive, supportive online comments and studies its automatic detection in a 46 000-comment German YouTube corpus. The task is split into binary comment classification and categorized span-based detection, with the latter requiring exact character boundaries and candy-type labels (Thelen et al., 9 Sep 2025).

A separate usage appears in a keyword-spotting adaptation derived from a speech-recognition study. There, “Candy Speech Detection” means detecting candy-related spoken keywords by applying Mel Frequency Cepstral Coefficients (MFCCs), HMMGMM, CNN, and RNN-based models to short audio clips (Rai et al., 2023).

Usage in the literature	Object of detection	Primary modality
GermEval candy speech	Positive, supportive online comments or spans	Text
Keyword-spotting adaptation	Candy-related words such as “candy” or “chocolate”	Speech

This terminological bifurcation is important. In current arXiv-facing NLP usage, the term is anchored by the GermEval shared task on German YouTube comments. The speech-based usage is best understood as an adaptation of a keyword-spotting pipeline rather than the canonical meaning of candy speech detection.

2. Corpus construction and annotation design

The GermEval 2025 Shared Task data, built upon Clausen et al. 2025, comprises 46 286 unique user comments sourced from the threads of sixteen German-language YouTube videos. Eleven videos supplied 37 057 comments for training, while five held out 9 229 comments for final testing. The comments exhibit typical “internet style” characteristics, including informal orthography, emojis, and occasional code-switching, and average 1.47 candy-speech spans each. Only 301 comments, approximately 1.9%, contain span overlaps (Thelen et al., 9 Sep 2025).

Preprocessing removed 3 829 exact duplicates. At the same time, 99 comments with identical text but conflicting labels were retained to preserve annotation variability. Each annotation is represented as a triplet $$(\mathrm{start\_char}, \mathrm{end\_char}, \mathrm{type})$$, and the annotation scheme distinguishes ten candy-speech types: positive feedback, compliment, affection declaration, encouragement, gratitude, agreement, ambiguous, implicit, group membership, and sympathy (Thelen et al., 9 Sep 2025).

This design places the task closer to fine-grained information extraction than to coarse sentiment classification. The low rate of overlapping spans suggests that single-label BIO tagging is operationally viable for most instances, although the paper notes that overlap remains a structural limitation for the submitted systems. The absence of explicit inter-annotator agreement statistics in the reported study constrains direct assessment of annotation reliability, even though the paper emphasizes the rarity of overlaps.

3. Modeling formulations and learning objectives

Three pretrained transformer backbones were investigated: GBERT-Large, Qwen3 Embedding-8B, and XLM-RoBERTa-Large. GBERT-Large is described as a monolingual German BERT variant pretrained on approximately 12 GB of German text using masked language modeling and next-sentence prediction. Qwen3 Embedding-8B is described as an 8 billion-parameter embedding model derived from the Qwen3 generation family and trained to produce high-quality multilingual text embeddings. XLM-RoBERTa-Large is described as a 550-million-parameter multilingual encoder trained on 2 TB of CommonCrawl data across 100 languages under the RoBERTa objective (Thelen et al., 9 Sep 2025).

For binary comment classification, two pipelines were evaluated. The first uses embeddings from a frozen Qwen33-Embedding-8B model and feeds them to an SVM with RBF kernel. The second fine-tunes GBERT-Large with an MLP head on the [CLS] token. For span detection, GBERT-Large and XLM-RoBERTa-Large are extended with a linear layer over the last hidden state of each token and trained under a token-wise BIO scheme. Because each of the ten candy types has separate B- and I-labels plus an O-label, the span model predicts 21 classes in total (Thelen et al., 9 Sep 2025).

The span-level objective is the token-level cross-entropy

$$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$

where $T$ is the number of tokens in the batch, $y_{t,\ell}\in\{0,1\}$ is the gold indicator for token $t$ and label $\ell$, and $\hat p_{t,\ell}$ is the corresponding softmax probability. For binary classification, the study uses the standard two-class cross-entropy

$$L_{\mathrm{ce}}=-\frac{1}{N}\sum_{i=1}^{N}\Bigl[y_i\log\hat p_i+(1-y_i)\log(1-\hat p_i)\Bigr].$$

A central empirical theme is that span-level supervision provides a richer learning signal than direct binary training. In practical terms, the span-trained models are converted to binary predictions by marking any comment with at least one predicted span as positive. This creates a direct bridge between structured extraction and document-level classification.

4. Tokenization, optimization, and implementation

All text models rely on subword tokenization. GBERT-Large uses WordPiece, XLM-RoBERTa-Large employs a multilingual BPE with 250 k merges, and Qwen3 Embedding applies SentencePiece. The paper emphasizes that both Qwen3 and XLM-RoBERTa vocabularies include common emojis and special symbols as standalone tokens, making them “emoji-aware” by default. No further vocabulary modifications are reported (Thelen et al., 9 Sep 2025).

Fine-tuning was implemented in PyTorch using HuggingFace’s Transformers library and executed on NVIDIA GPUs provided by M. Reißel and V. Sander. For Subtask 1 with GBERT fine-tuning, the setup used batch size 32, 5 epochs, AdamW with weight decay 0.01, linear warmup to $5\times10^{-5}$ over the first 30% of steps followed by linear decay to zero, gradient clipping with L2 norm 1.0, and early stopping on positive-class F1 using a 10% validation split with patience of 64 evaluations every 44 steps. For the Qwen3 + SVM pipeline, grid search ranged over $C\in[1,100]$ and $$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$0, with optimal hyperparameters $$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$1 and $$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$2. For Subtask 2 span-level fine-tuning, both GBERT and XLM-RoBERTa used batch size 32, 20 epochs, peak learning rate $$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$3, evaluation every 40 steps, and early stopping patience of 87 evaluations; warmup steps were 200 for GBERT and 500 for XLM-RoBERTa (Thelen et al., 9 Sep 2025).

The authors speculate that multilingual capabilities, span-based training, and emoji-aware tokenizers improved detection performance. A plausible implication is that tokenization is not a peripheral implementation choice in this task, because supportive language in internet-style comments often co-occurs with emotive symbols and nonstandard orthography.

5. Evaluation protocol and reported performance

The binary task is evaluated with positive F1, defined for precision $$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$4 and recall $$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$5 as

$$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$6

The span task is evaluated with strict span-based F1, under which a predicted span is correct only if both its character boundaries and its candy-type label exactly match a gold span (Thelen et al., 9 Sep 2025).

In Subtask 1, Qwen3 Embedding + SVM achieved a mean positive F1 of 0.901 with $$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$7 on validation and 0.875 on test, ranking eighth. Fine-tuned GBERT-Large reached 0.887 with $$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$8 on validation. When span-trained models were converted to binary decisions, GBERT and XLM-RoBERTa achieved 0.903 and 0.913 with $$L_{\mathrm{span}}=-\frac{1}{T}\sum_{t=1}^{T}\sum_{\ell=1}^{21} y_{t,\ell}\log\bigl(\hat p_{t,\ell}\bigr),$$9 on validation, respectively. XLM-RoBERTa’s span-derived binary predictions ranked first in the shared task, with binary positive F1 reported as 0.8906 in the abstract and 0.891 in the detailed results (Thelen et al., 9 Sep 2025).

In Subtask 2, XLM-RoBERTa-Large with the basic postprocessing pipeline achieved the best strict span F1 on test, 0.6307, corresponding to first place. GBERT-Large with extended postprocessing scored 0.623, corresponding to second place. On validation, XLM-RoBERTa reached 0.747 with basic postprocessing and 0.742 with extended postprocessing, whereas GBERT scored 0.731 with basic and 0.739 with extended postprocessing (Thelen et al., 9 Sep 2025).

The reported ablation comparing span and binary training shows an improvement of roughly $T$0 F1, from 0.887 to 0.913, for binary detection when the model is trained at the span level rather than directly as a binary classifier. The paper does not provide formal statistical significance tests, but it does note that the standard deviations across folds are approximately 0.002–0.006. This suggests that the observed gain is consistent under the reported validation protocol, while still leaving inferential claims formally untested.

6. Limitations, extensions, and the speech-domain reinterpretation

Several limitations are explicitly identified. The submitted span models cannot handle overlapping spans, a design choice adopted to simplify the shared-task submission, even though overlaps account for only 1.9% of the data. The authors note potential biases associated with the YouTube comment domain and the eleven training videos, and they recommend replication on other platforms such as Twitter and Reddit before deployment. Suggested future directions include fine-tuning larger models such as Qwen3 Embedding, building ensembles, incorporating conversational context such as preceding comments or video transcripts, and extending span detection to multi-label or overlapping cases. The paper also notes potential applications in real-time feedback loops for generative LLM control, including monitoring sycophancy or unwarranted positivity (Thelen et al., 9 Sep 2025).

A distinct research line uses the phrase “Candy Speech Detection” for speech keyword spotting. In that setting, the recipe begins with MFCC extraction: pre-emphasis with $T$1 and $T$2, 25 ms windows with 10 ms stride, a Hamming window, an $T$3-point DFT such as $T$4, Mel-filterbank processing, logarithmic energies, and a DCT yielding cepstral coefficients, with the cited setup using $T$5 coefficients. Optional steps include cepstral liftering, delta and $T$6 features, CMVN, and augmentations such as time-shifting, additive background noise, speed or pitch perturbation, and SpecAugment (Rai et al., 2023).

The corresponding model families include HMMGMM with six phonetic substates per keyword and two-component Gaussian mixtures, an M5-style 1-D CNN on 8 kHz raw waveforms, and RNNs over 12-dimensional MFCC sequences, including unidirectional LSTM, bidirectional LSTM, and attention-equipped Bi-LSTM. On Google Speech Commands v2, comprising 105,829 one-second clips across 35 single-word classes with an 80/10/10 train/validation/test split, the reported test accuracies are approximately 55% for HMMGMM, 92.5% for the CNN, 91.4% for Uni-LSTM, 92.0% for Bi-LSTM, and 93.9% for Bi-LSTM + Attention. The adaptation to candy-related keywords recommends defining a candy keyword set, collecting at least 1,000 utterances per keyword, and using Bi-LSTM + Attention when moderate compute is available (Rai et al., 2023).

This speech-domain reinterpretation should not be conflated with the GermEval task. The former is an adaptation of command recognition to candy-related acoustic tokens; the latter is a span-sensitive NLP problem over supportive online discourse. The shared terminology is therefore incidental rather than methodological, and the two settings occupy different positions in the broader taxonomy of detection tasks.