TokEye: Fast Spectrogram Event Extraction

• TokEye is a fast, automated spectrogram event-extraction framework that segments coherent and transient structures from noisy tokamak signals.
• It employs a two-phase design with offline pseudo-label generation and a surrogate U-Net for rapid inference, reducing processing time to about 0.5 seconds per shot.
• The framework enhances plasma diagnostics by effectively removing broadband noise and applying multichannel self-supervised denoising to support near-real-time monitoring and analysis.

TokEye is a fast, automated spectrogram event-extraction framework for noisy, multichannel fluctuation measurements in tokamaks and related time-frequency settings. It is presented as a general-purpose, largely label-free, “signals-first” method that converts raw diagnostic time series into segmented coherent and transient structures while suppressing broadband background and stochastic contamination. Its central contribution is a two-phase design: an offline pseudo-label generation pipeline built from broadband baseline removal, multichannel self-supervised denoising, thresholding, and refinement; and a deployment-time surrogate U-Net that compresses that workflow into approximately $0.5$ seconds of GPU inference per full shot (Chen et al., 23 Feb 2026).

1. Conceptual basis and signal taxonomy

TokEye is motivated by the fusion “data deluge”: modern and future tokamaks produce enormous volumes of diagnostic time series, while the phenomena of interest are often buried in nonstationary backgrounds and stochastic noise. The framework therefore starts from measured fluctuation structure in time-frequency space rather than from a narrow inventory of pre-labeled plasma events. This “signals-first” stance is used to avoid overcommitting to a limited ontology of known modes and to reduce dependence on small manually labeled datasets or simulation-generated labels (Chen et al., 23 Feb 2026).

The paper organizes spectrogram content into coherent, quasi-coherent, transient, broad, and stochastic classes, then groups them operationally into coherent measurements, broadband measurements, and noise. Coherent modes are narrowband structures with a well-defined frequency at each instant, even if they chirp in time; examples include tearing modes, neoclassical tearing modes, GAMs, Alfvén eigenmodes, and kinks. Quasi-coherent modes occupy an intermediate regime, neither purely periodic nor fully broadband or stochastic, but still exhibiting localized resonant structure. Transient events are short-lived broadband impulses such as ELMs, pellet injections, sawtooth crashes, disruptions, or neutral-particle spikes, which appear as vertical or spike-like structures in spectrograms. Broad modes refer to low-frequency, nonperiodic, or turbulent/background structures often associated with $1/f^\chi$-type spectral decay. Stochastic noise denotes random, roughly mean-centered noise, not necessarily Gaussian, and may include Gaussian, Laplacian, or uniform components.

This taxonomy directly determines the processing order. The authors argue that broadband structures must be separated first because they can overpower faint coherent modes, especially on integrated diagnostics such as interferometers or ECE. Only after removing this broad spectral “color” does denoising of coherent content become effective. A plausible implication is that TokEye is best understood not as a downstream plasma-mode classifier, but as an upstream event-extraction front-end for later physics analysis, database construction, or control-oriented logic.

2. Spectrogram construction and broadband baseline separation

The workflow begins with raw multichannel time-series signals. All diagnostics used in the study are resampled to $500$ kHz using 8th-order Chebyshev Type I decimation with phase preservation. Spectrograms are then computed with a Hann window of length $N=1024$ and overlap $87.5\%$, producing a standard STFT representation over $0$–$250$ kHz, which the paper identifies as a useful upper bound for much tokamak fluctuation physics, including AEs (Chen et al., 23 Feb 2026).

The core time-frequency quantities are defined as

$Z(t,f) = \operatorname{STFT}\{x(t)\},$

$P(t,f) = |Z(t,f)|^2,$

and

$$R + jI = \operatorname{Re}\{Z\} + j\operatorname{Im}\{Z\}.$$

The observed spectrogram is modeled as

$1/f^\chi$0

where $1/f^\chi$1 denotes coherent observations, $1/f^\chi$2 denotes broadband observations, and $1/f^\chi$3 denotes stochastic noise.

Broadband baseline removal is formulated as an asymmetric penalized optimization: $1/f^\chi$4 with asymmetric weight

$1/f^\chi$5

using $1/f^\chi$6 and $1/f^\chi$7. Positive residuals, which may correspond to candidate peaks or events, are thus made less influential during baseline fitting so that the estimator captures the broadband background rather than narrow modes.

Before baseline estimation, the paper applies pre-emphasis

$1/f^\chi$8

with $1/f^\chi$9, corresponding to an assumed first-order inverse power law. This partially flattens the low-frequency tail and stabilizes baseline estimation. The paper notes an explicit practical failure mode: edge effects on some sensors can persist to about $500$0 kHz in a $500$1 kHz spectrum. That caveat is important because it constrains interpretation of the lowest-frequency regime.

3. Multichannel self-supervised denoising and event extraction

After broadband subtraction, TokEye removes stochastic noise through multichannel self-supervised denoising. The conceptual starting point is the classical cross-power spectrum

$500$2

with complex expansion

$500$3

Under expectation, incoherent random phases cancel while coherent components remain. The paper argues, however, that this is effectively a blind linear filter: if a transient or localized event appears only in a minority of segments, averaging suppresses it, and classical multichannel decompositions can scale poorly with channel number while assuming stronger linear correlations than may hold across diagnostics (Chen et al., 23 Feb 2026).

TokEye replaces that linear averaging with a nonlinear predictor that reconstructs one channel from the others through the conditional expectation

$500$4

The input tensor concatenates the real and imaginary STFT components from $500$5 channels: $500$6 and a U-Net predicts the real and imaginary parts of a target channel: $500$7 Training uses an MAE objective,

$500$8

chosen for robustness.

The paper explicitly preserves complex STFT structure rather than denoising magnitude alone, stating that magnitude-only or real-valued denoising causes worse results because important complex information is lost. It also rejects blind-spot single-image denoisers for this setting because signal and noise correlations extend across the full time-frequency plane and many relevant structures are too sparse to be recovered from local neighborhoods alone. To avoid overfitting residual noise during self-supervised training, the authors use a total variation-based stopping criterion, although no explicit TV formula is given.

Candidate events are extracted from the denoised spectrogram by global thresholding through a cumulative-distribution knee-point method. Values below the mean are first set to the mean, after which the cumulative distribution function is computed and normalized. A line connecting the CDF endpoints is drawn after interpolation onto equalized axes, and the threshold is selected at the point of maximum distance from that line. The paper describes this as parameter-free except for interpolation detail. It also states a caveat: a spectrogram with no signal at all would still yield an arbitrary knee threshold, and the authors avoid this case because all examples considered contain at least one mode.

4. Pseudo-label generation, refinement, and surrogate model

The initial self-supervised masks can miss structures that appear only in a single channel and are therefore not recoverable through inter-channel reconstruction. TokEye addresses these false negatives and edge failures with a second U-Net trained as a refinement or label-correction model on single-channel measurements. This stage uses 5-fold randomized splits, a symmetric binary cross-entropy objective, and image-style augmentations including flips and elastic deformation. High-entropy predictions from an ensemble average are added back into the predicted signal to expand the captured signal region. The resulting procedure produces roughly $500$9 pseudo-labeled spectrogram fragments (Chen et al., 23 Feb 2026).

These pseudo-labels train the final TokEye surrogate: a fast U-Net that maps a single-channel spectrogram directly to a two-channel segmentation output corresponding to coherent and transient portions of the spectrum. Input normalization is done per diagnostic using cumulative means and standard deviations, followed by clipping to the $N=1024$0 and $N=1024$1 percentiles with robust scaling. To reduce sensitivity to STFT parameter choices, the surrogate is trained with multi-period, multi-scale augmentation: labels are bicubically resampled, and spectrograms are generated from random window sizes in $N=1024$2 and hop sizes in $N=1024$3. The paper states that without this, segmentation is highly sensitive to window length and overlap.

The same modified U-Net backbone is used three times: for multichannel self-supervised denoising, single-channel label refinement, and final surrogate segmentation. It is based on Ronneberger et al., with transpose convolutions replaced by upsample-plus-double-convolution blocks to avoid checkerboard artifacts. Training uses bfloat16 precision. Data handling uses FAITH/FusionAIHub and HDF5 datasets.

In operational form, the end-to-end procedure is fixed and explicit: resample to $N=1024$4 kHz, compute complex STFTs, form power spectrograms, optionally apply pre-emphasis, estimate and subtract the broadband baseline, denoise through multichannel self-supervised reconstruction, threshold through CDF knee selection, refine masks with the single-channel U-Net, aggregate pseudo-labeled fragments, train the surrogate, and at inference time skip the expensive offline stages and run the surrogate directly. Post-processing uses a model threshold, often shown as $N=1024$5, then extracts connected event regions and computes quantities such as $N=1024$6, $N=1024$7, $N=1024$8, $N=1024$9, and integrated amplitude. Transients can optionally be gated out to isolate coherent structures for downstream mode-number fitting, database generation, or control logic.

5. Diagnostics, empirical performance, and demonstrated use cases

TokEye is developed across heterogeneous fluctuation diagnostics: DIII-D fast magnetics, electron cyclotron emission, CO$87.5\%$0 interferometers, and beam emission spectroscopy. These were chosen because they exhibit markedly different spectrogram characteristics. Magnetics tend to show cleaner coherent bands; ECE can be noisy but resolves core-localized temperature fluctuations and MHD or AE activity; CO$87.5\%$1 signals often have strong low-frequency broadband bias and integrated noise; BES probes a different fluctuation regime. The method is intended to work across these sensors rather than only on a single diagnostic family (Chen et al., 23 Feb 2026).

The paper’s main quantitative generalization results are concentrated on TJ-II and the non-fusion DCLDE 2011 benchmark.

Evaluation setting	Result	Interpretation in the paper
TJ-II ECE without retraining	recall $87.5\%$2	cross-machine generalization
DCLDE 2011, Delphinus capensis	recall $87.5\%$3	zero-shot use
DCLDE 2011, Delphinus delphis	recall $87.5\%$4	zero-shot use

For TJ-II ECE spectrograms, the surrogate attains recall $87.5\%$5 on expert-labeled data without retraining. The paper also reports a systematic issue: the model tends to place a cutoff around $87.5\%$6 kHz, which the authors attribute to a manual bias and say can be corrected in post-processing, analogous to removing scan lines. They further state that the model identifies some structures not annotated by experts and correctly marks regions with no mode activity.

On the DCLDE 2011 odontocete whistle contour dataset, TokEye achieves recall $87.5\%$7 on Delphinus capensis and $87.5\%$8 on Delphinus delphis in zero-shot use. Precision is lower, but the paper argues that this is partly an artifact of annotation style: human labels are often one-pixel contours whereas TokEye returns thicker segmented regions. The authors explicitly avoid skeletonization because one-pixel ridges are not desirable for tokamak signals, whose widths carry physical meaning.

The DIII-D results are mostly qualitative but informative. In magnetic data, TokEye produces segmented event regions that are converted into database entries with minimum and maximum frequencies, minimum and maximum times, and amplitudes in dB. In CO$87.5\%$9 data, it detects both low-frequency and Alfvén-like or high-frequency structures despite strong low-frequency background and can resolve fine chirps. In ECE, the paper highlights two DIII-D tearing-mode control shots: shot $0$0 with tearing instability and shot $0$1 with ECCD suppression. In the tearing-suppressed shot, sustained high-frequency Alfvén-like modes remain present longer; in the tearing shot, those high-frequency structures terminate near the onset of tearing. The paper also notes core-edge differences in ECE that are said to be consistent with Shapley-based influence analysis from another study. This suggests TokEye can support physics interpretation in addition to event extraction.

6. Runtime, scope, and limitations

TokEye’s practical appeal lies in runtime compression. Starting from the HDF5 format used by the authors, generating a database of $0$2 training-ready examples takes about $0$3 hours on one Nvidia A100 GPU plus a 64-core Intel CPU. Training the final surrogate with full 5-fold validation takes about $0$4 hours. At deployment, inference takes about $0$5 seconds per full shot on GPU and about $0$6–$0$7 seconds per shot on CPU depending on parallelism. The paper attributes this latency reduction to replacing broadband removal, multichannel denoising, thresholding, and refinement with a single surrogate U-Net (Chen et al., 23 Feb 2026).

The paper therefore describes TokEye as suitable for “real-time mode identification,” “active plasma monitoring,” and “inter-shot analysis,” and as enabling advanced plasma control. It also states that TokEye can support large-scale automated database generation. A common misunderstanding would be to read this as demonstrated hard real-time control in the sub-millisecond or few-millisecond regime. The paper does not make that claim. Given the reported $0$8-second latency, the safest interpretation is near-real-time monitoring, event detection, and inter-shot or slower supervisory control workflows rather than PCS-integrated hard real-time actuation.

Several limitations are explicit. The baseline-removal stage can suffer low-frequency edge effects. The self-supervised denoiser is not theoretically perfect because the system does not have infinite independent noisy realizations; residual noise may eventually be learned if training continues too long. Channel prediction can miss structures unique to a single channel, which is why the refinement network is needed. The thresholding method assumes at least one true signal is present. The TJ-II surrogate shows a bias around $0$9 kHz. CPU inference on variable-size inputs is substantially slower than GPU execution. The current system extracts one “view” at a time rather than jointly modeling inter-diagnostic correlations in the final surrogate. The paper also identifies phase information and turbulence characterization as future additions.

In concise technical terms, TokEye is not primarily a plasma-mode classifier. It is an automated spectrogram event-extraction front-end whose novelty lies in combining a physically motivated signals-first taxonomy, broadband baseline separation before denoising, nonlinear multichannel self-supervised denoising on complex STFTs, parameter-light thresholding by CDF knee detection, automated pseudo-label database generation, and a fast surrogate U-Net. Within that scope, it is presented as a diagnostic-agnostic mechanism for isolating coherent and transient structures from noisy fluctuation measurements, with an implementation released at https://github.com/PlasmaControl/TokEye (Chen et al., 23 Feb 2026).