StutterCut: Graph-Based Stutter Segmentation

• StutterCut is a semi-supervised framework that segments dysfluencies by formulating speech as a weighted graph and partitioning it with normalized cuts.
• It extracts overlapping speech embeddings from pre-trained models and refines similarity measures through classifier-derived uncertainty.
• The method achieves state-of-the-art results on datasets like FluencyBank++ and VCTK-TTS, reducing the need for detailed frame-level annotations.

StutterCut is a semi-supervised framework for the segmentation of dysfluencies in continuous speech, formulating the task as a graph partitioning problem guided by classifier-derived uncertainty. It enables precise temporal localization of stuttering events without requiring frame-level labels or transcriptions, and demonstrates state-of-the-art performance on both real and synthetic datasets by integrating graph-based normalized cuts with weakly supervised deep learning priors and uncertainty quantification (Ghosh et al., 4 Aug 2025).

1. Graph-Based Semi-Supervised Dysfluency Segmentation

StutterCut addresses the segmentation of dysfluent regions by modeling overlapping audio windows as nodes in a weighted, fully connected, undirected graph. Each node corresponds to a speech window embedding, and edges encode similarity between windows. The principal objective is to partition this graph into fluent and dysfluent regions in the absence of strong (frame-level) labels by minimizing a normalized cut (N-Cut) criterion.

Unlike traditional graph cuts, which tend to isolate small (rare) clusters due to unnormalized objectives, the N-Cut balances the cumulative cut value with the association strength to each partition, promoting cluster balance and mitigating over-fragmentation—properties essential for rare event detection such as stutter onset. Given similarity matrix $W$ and degree matrix $D$, the normalized cut objective is:

$J(y) = \frac{y^\top(D - W)y}{y^\top D y}$

where binary labeling $y \in \{1,-b\}^N$ (with $b = |S_1|/|S_2|$) enforces balanced partitioning. The minimization reduces to solving the generalized eigenproblem:

$D^{-1/2}(D-W)D^{-1/2}z = \lambda z$

The Fiedler vector $z_1$ (associated with $\lambda_2$) is thresholded (by sign or mean) to obtain two clusters, interpreted as fluent and dysfluent segments, respectively.

2. Windowing, Embedding Extraction, and Initial Similarity

Each utterance is decomposed into $N$ overlapping windows of length $l$ and stride $D$0 ($D$1 s, $D$2 s). For each window $D$3, a $D$4-dimensional embedding $D$5 is extracted from a pre-trained speech foundation model such as Whisper, WavLM, or Wav2Vec2, which capture acoustic features relevant to dysfluency events (e.g., prolongations, silent blocks). The initial edge weights form the similarity matrix $D$6 using cosine similarity:

$D$7

This unsupervised similarity forms the basis for subsequent classifier-guided refinement.

3. Pseudo-Oracle Classifier and Uncertainty-Guided Graph Refinement

To leverage weak supervision, StutterCut utilizes a multi-label classifier $D$8 (“Whisterₘₗ”) trained on utterance-level presence/absence tags for each dysfluency type. Inputs are window embeddings; outputs are per-class probabilities. The classifier is not exposed to any temporal boundary annotations, relying solely on turbo-aggregated focal loss at the utterance level.

For each node, $D$9 produces a vector $J(y) = \frac{y^\top(D - W)y}{y^\top D y}$0 of class probabilities: $J(y) = \frac{y^\top(D - W)y}{y^\top D y}$1 (fluent), $J(y) = \frac{y^\top(D - W)y}{y^\top D y}$2 ($J(y) = \frac{y^\top(D - W)y}{y^\top D y}$3 denotes each dysfluency class). A classifier-derived similarity $J(y) = \frac{y^\top(D - W)y}{y^\top D y}$4 is constructed:

$J(y) = \frac{y^\top(D - W)y}{y^\top D y}$5

where $J(y) = \frac{y^\top(D - W)y}{y^\top D y}$6. This approach increases affinity between nodes labeled as likely the same class.

To quantify epistemic uncertainty in classifier outputs, Monte Carlo dropout is employed at inference: each window embedding is passed through $J(y) = \frac{y^\top(D - W)y}{y^\top D y}$7 $J(y) = \frac{y^\top(D - W)y}{y^\top D y}$8 times, generating $J(y) = \frac{y^\top(D - W)y}{y^\top D y}$9 probability vectors. Predictive entropy $y \in \{1,-b\}^N$0 for window $y \in \{1,-b\}^N$1 is computed as:

$y \in \{1,-b\}^N$2

with $y \in \{1,-b\}^N$3 and $y \in \{1,-b\}^N$4. A confidence mask $y \in \{1,-b\}^N$5 is defined.

Final edge weighting fuses raw embedding and classifier priors using the confidence mask:

$y \in \{1,-b\}^N$6

where $y \in \{1,-b\}^N$7 and $y \in \{1,-b\}^N$8 is elementwise multiplication. Entries with $y \in \{1,-b\}^N$9 ($b = |S_1|/|S_2|$0) are set to negligible value. This mechanism ensures that edge weights are enhanced by classifier-derived knowledge where confident, and default to unsupervised similarity where classifier uncertainty is high.

4. Segmentation and Postprocessing

Graph partitioning proceeds by solving the N-Cut eigenproblem for $b = |S_1|/|S_2|$1. The Fiedler vector $b = |S_1|/|S_2|$2 generates cluster assignments $b = |S_1|/|S_2|$3 by thresholding ($b = |S_1|/|S_2|$4). Cluster-to-dysfluency mapping is determined by majority vote: if most nodes in a cluster score highest for a dysfluency class under $b = |S_1|/|S_2|$5, that cluster is assigned as dysfluent.

Boundaries are derived by merging consecutive windows labeled as dysfluent, forming segments from the earliest to latest detected window times. Short gaps less than or equal to $b = |S_1|/|S_2|$6 ms and isolated segments of similar duration are post-processed—merged or dropped—according to validation-optimized $b = |S_1|/|S_2|$7.

5. FluencyBank++ Dataset Construction

StutterCut introduced a refined evaluation corpus, FluencyBank++, by augmenting the weakly labeled FluencyBank dataset with high-quality temporal boundaries for four dysfluency types: interjection, repetition, block, and prolongation. The protocol involved:

1. Utterance-Level Correction: Extending each clip to 5 s; expert revision of weak tags.
2. Frame-Level Boundary Marking: Independent annotation of dysfluency intervals by two experts.
3. Consensus Refinement: A panel of three speech pathologists resolved disagreements, with a frame marked as dysfluent if ≥2 agree. Inter-annotator agreement (Fleiss’ κ) improved from 0.47 to 0.71.

Statistics for FluencyBank++:

Dysfluency Type	# Clips	Interval Duration (s)
Interjection	1,130	0.12–1.88
Repetition	921	0.20–4.99
Block	530	0.23–4.20
Prolongation	436	0.41–3.95

Evaluation uses time-F1 (t-F1), time-recall (t-recall), and onset error (absolute deviation of predicted onset from ground-truth), with macro-averaging per speaker and class.

6. Evaluation and Comparative Analysis

Performance benchmarks on FluencyBank++ and the synthetic VCTK-TTS dataset demonstrate that StutterCut outperforms prior methods in segment-level metrics and onset detection accuracy. Key comparisons are summarized below:

Method	t-F1 (%) FB++ / VCTK	t-recall (%) FB++ / VCTK
YOLO-Stutter	56.8 / 58.8	87.1 / 100.0
Harvill et al.	21.6 / 25.3	47.1 / 67.0
Whister_MC	43.7 / 36.8	55.5 / 77.0
Whister_ML	61.1 / 70.6	88.5 / 86.7
StutterCut (SCut)	69.3 / 87.2	89.2 / 92.6

Ablation studies demonstrate the necessity of uncertainty gating (mask $b = |S_1|/|S_2|$8): replacing $b = |S_1|/|S_2|$9 with a mean-probability mask reduces t-F1 by ~5%; omitting $D^{-1/2}(D-W)D^{-1/2}z = \lambda z$0 lowers t-F1 to 63.4%; pure N-Cut yields a further ∼10% reduction in t-F1 and ∼5% in t-recall. StutterCut achieved the lowest onset error (0.3 s for FluencyBank++, 0.2 s for VCTK-TTS), indicating superior boundary localization accuracy.

7. Significance and Implications

StutterCut demonstrates the effectiveness of casting frame-local dysfluency segmentation as a graph partitioning task conditioned only by weak supervision and classifier uncertainty. The approach removes the requirement for transcriptions or frame-level supervision, generalizing across acoustic embedding types. The methodology establishes a new state-of-the-art for stutter boundary detection on both authentic and synthetic speech corpora. A plausible implication is the applicability of StutterCut's uncertainty-gated graph cut paradigm in other domains involving rare event localization from weak or ambiguous supervision (Ghosh et al., 4 Aug 2025).