Two-Stage Channel-Aware Transformer for Seizure Prediction

• The paper introduces a two-stage set transformer network that aggregates short temporal EEG segments and channel data to enhance seizure prediction accuracy while reducing sensor count.
• The model leverages permutation-invariant design to treat EEG segments and channels as unordered sets, avoiding ordering assumptions common in CNNs and LSTMs.
• Empirical results on the CHB-MIT dataset demonstrate improved sensitivity and a reduction from 18 to an average of 2.8 channels, underscoring its potential for wearable applications.

Searching arXiv for the primary paper and closely related work on Set Transformer and EEG seizure prediction. Searching arXiv for "(Zheng et al., 21 Jul 2025)" and "EEG-based Epileptic Prediction via a Two-stage Channel-aware Set Transformer Network". The Two-Stage Channel-Aware Set Transformer Network is an EEG-based epileptic seizure prediction architecture that models both temporal EEG structure and inter-channel structure as permutation-invariant sets rather than ordered sequences. It was introduced for wearable seizure prediction under the constraint that practical systems should use fewer EEG sensors while preserving predictive utility. The model operates in two stages: a Temporal Set Transformer aggregates multiple short EEG segments within each channel into a channel-level representation, and a Channel-aware Set Transformer aggregates those channel representations into a seizure-risk prediction while also producing channel attention scores for patient-specific channel selection. On CHB-MIT, the reported mean sensitivity is 76.4% with a false predicting rate (FPR) of 0.09/hour before channel selection, and 80.1% with an FPR of 0.11/hour after channel selection; dominant channels emerged in 20 out of 22 patients, with the average number of channels reduced from 18 to 2.8 (Zheng et al., 21 Jul 2025).

1. Problem setting and design rationale

The network is situated in EEG-based epileptic prediction, where the central engineering constraint is that wearable devices become less feasible as the number of electrodes increases. The motivating argument is explicit: more channels imply more sensors and wiring, bulkier and less comfortable hardware, higher power consumption, higher cost, and lower feasibility for all-day use. The method therefore targets seizure prediction with fewer EEG channel sensors while retaining predictive quality (Zheng et al., 21 Jul 2025).

Three methodological limitations are identified. First, many prior methods use all available channels, which is unfavorable for wearable deployment. Second, conventional CNN, LSTM, and Transformer pipelines typically impose ordered-sequence or grid assumptions on EEG inputs, whereas the paper argues that EEG channels do not have a natural order for the modeling task and that some channels may dominate prediction. Third, commonly used evaluation protocols based on even division can place adjacent windows from the same continuous interictal recording into both training and test sets, which may inflate performance estimates.

Within this framing, the “two-stage” designation refers not to a proposal-refinement pipeline, but to a staged set aggregation procedure. The first stage aggregates temporal evidence within each channel. The second stage aggregates across channels and exposes channel importance through attention. A common misconception is that the model is simply a standard Transformer applied twice. The paper instead positions it as a Set Transformer pipeline without positional encoding, motivated by permutation invariance over both channels and temporal chunks (Zheng et al., 21 Jul 2025).

2. Permutation-invariant representation and temporal aggregation

Each EEG sample is represented as a collection of channel-wise feature vectors derived from band-power features computed on short temporal segments. The input window length is 38 seconds, and features are extracted every 2 seconds, yielding

$T = 19$

segments per input window. For each 2-second segment, the feature extraction uses eight sub-bands: $\theta$ (4–8 Hz), $\alpha$ (8–13 Hz), $\beta$ (13–30 Hz), $\gamma_1$ (30–50 Hz), $\gamma_2$ (50–70 Hz), $\gamma_3$ (70–90 Hz), $\gamma_4$ (90–110 Hz), and $\gamma_5$ (110–128 Hz) (Zheng et al., 21 Jul 2025).

The per-channel feature vector consists of 44 dimensions:

• 8 absolute spectral power features,
• 8 relative spectral power features,
• 28 spectral power ratios.

The corresponding definitions are

$$P_{absolute}(i)=\log\sum_{w\in \text{band } i}PSD(w),\quad i=1,2,\ldots,8$$

$\theta$0

and

$\theta$1

The temporal stage applies the Multihead Attention Block (MAB) from Set Transformer to the set of temporal segments within each channel. Following Lee et al., the paper defines

$\theta$2

and

$\theta$3

with

$\theta$4

where $\theta$5 and $\theta$6 (Lee et al., 2018). In the Set Transformer form used here,

$\theta$7

and

$\theta$8

where

$\theta$9

For channel $\alpha$0, the temporal aggregation is written as

$\alpha$1

where $\alpha$2 is trainable, $\alpha$3, $\alpha$4, and the output is a compact channel representation in $\alpha$5 (Zheng et al., 21 Jul 2025).

The significance of this design is that the model treats temporal evidence as a set of informative patterns rather than as a rigidly ordered sequence. This suggests that the architecture is optimized for aggregation of discriminative temporal content rather than for preserving exact positional chronology.

3. Channel-aware aggregation and dominant-channel selection

The second stage consumes the set of channel embeddings and produces the final prediction. It is defined as

$\alpha$6

where $\alpha$7 is trainable, $\alpha$8, $\alpha$9, and $\beta$0. This output is then passed to a fully connected layer for seizure prediction (Zheng et al., 21 Jul 2025).

The model is “channel-aware” because the second-stage attention is not used only for classification; it is also accumulated to estimate stable channel importance. The paper notes that per-batch channel attention fluctuates strongly, so it introduces an accumulation layer. The source notation for the accumulation formula is explicitly described as garbled, but the stated intent is clear: if $\beta$1 denotes the attention distribution for one input, then accumulated attention $\beta$2 is obtained by aggregating channel attention over samples and batches and then applying Softmax. The channels with the largest accumulated attentions are designated dominant channels (Zheng et al., 21 Jul 2025).

This channel-selection mechanism is patient-specific. In the even division experiments, dominant channels emerged in 20 of 22 patients; the average number of selected channels was 2.8; some patients required only 1 channel; most required 2–4 channels; one patient needed 5 channels; and 2 patients failed channel selection. The stated interpretation is that if prediction is consistently driven by a few channels, those channels preserve most of the discriminative information, whereas lack of convergence indicates failure of channel selection for that patient.

A frequent misunderstanding is to treat the attention output as a generic interpretability map. In this network, channel attention has an operational role: it is used to trigger a second round of training and inference using only the selected channels. The architecture therefore couples prediction and sensor reduction in a single framework rather than treating feature selection as a separate preprocessing step (Zheng et al., 21 Jul 2025).

4. Data protocol, seizure-independent division, and evaluation criteria

The experiments use the CHB-MIT scalp EEG dataset from pediatric subjects with intractable epilepsy. The dataset handling is specified as follows: 23 subjects in total, patient 23 excluded due to only 0.27 h of interictal EEG, 22 patients used for even division experiments, and 7 patients used for seizure-independent division. Adjacent seizures with interval less than 1 hour were merged, and seizures with preictal length significantly less than 30 minutes were omitted (Zheng et al., 21 Jul 2025).

The 18 common channels are: FP1-F7, F7-T7, T7-P7, P7-O1, P3-O1, C3-P3, F3-C3, FP1-F3, FZ-CZ, CZ-PZ, P4-O2, C4-P4, F4-C4, FP2-F4, FP2-F8, F8-T8, T8-P8, and P8-O2. They are indexed from 0 to 17.

The seizure prediction protocol uses a seizure prediction horizon (SPH) of 3 minutes and a seizure occurrence period (SOP) of 30 minutes. A seizure is counted as correctly predicted if it occurs after the SPH and within the SOP following a positive alarm. The interictal exclusion rule removes only 1 hour before and after seizures, rather than the more common 4 hours; the stated motivation is to reduce the chance of alarms being raised too early and to make the system more clinically useful (Zheng et al., 21 Jul 2025).

The evaluation controversy centers on data splitting. Under even division, long interictal recordings are concatenated and divided into parts, which can place adjacent interictal segments into both train and test sets. The seizure-independent division is proposed as a stricter alternative: the interictal sequence before a seizure is treated as an inseparable unit; that unit is assigned only to one side of the split; adjacent windows from the same interictal run do not cross train/test boundaries; and only patients with sufficiently many seizure-independent interictal periods are eligible. The paper follows Zhang et al. and includes only patients with daily seizure frequency between 2 and 10, which leaves 7 eligible patients (Zheng et al., 21 Jul 2025).

The primary evaluation metrics are

$\beta$3

and

$\beta$4

Here, $\beta$5 denotes correctly predicted seizures, $\beta$6 missed seizures, $\beta$7 false alarms during interictal periods, and $\beta$8 the total interictal duration.

5. Empirical results, ablations, and computational profile

The main quantitative results are summarized below.

Setting	Sensitivity	FPR
Even division, before channel selection	76.4%	0.09/h
Even division, after channel selection	80.1%	0.11/h
Seizure-independent division, before channel selection	72.6%	0.08/h
Seizure-independent division, after channel selection	72.6%	0.10/h

On the even division, the paper additionally reports that 8 patients achieved 100% sensitivity before selection, and after selection 9 patients had all seizures correctly predicted. Patient 16 and Patient 22 failed channel convergence in the even division, while Patient 04 and Patient 07 had low sensitivity after selection. The practical systems-level result emphasized by the authors is the reduction from 18 channels to an average of 2.8 channels (Zheng et al., 21 Jul 2025).

The seizure-independent division results are used to argue that traditional even division may overestimate generalization when adjacent EEG data leaks across splits. The lower mean sensitivity under the stricter split is interpreted as evidence for this concern. The paper therefore concludes that seizure-independent division should be used for patients with abundant EEG recordings.

The comparison set includes Truong et al. 2018, Zhang et al. 2023, Shu et al. 2024, Birjandtalab et al. 2017, and Affes et al. 2022, along with ablations over CNN versus Set Transformer in the temporal stage, LSTM versus Set Transformer in the temporal stage, and different channel selection modules. The reported comparative interpretation is that, despite using only 1 hour excluded around seizures and fewer channels, the proposed method is competitive with or better than prior work using more permissive data handling (Zheng et al., 21 Jul 2025).

The implementation profile is compact. The model has 37.4K parameters and inference FLOPs of 8.23M, with memory reporting at batch size 16. Runtime is reported as 23.5 ms preprocessing per 2-second segment, 10.0 ms network inference, and 33.5 ms total per incoming EEG segment. No data augmentation is used in the main pipeline. Post-processing is not per-window thresholding alone: an alarm is activated if positive predictions continue for a certain duration without interjected negatives, about 4 minutes for most patients and 1–2 minutes for some, followed by a 30-minute refractory period. The authors note that direct execution on a wearable is likely infeasible and instead suggest a remote computation framework in which the wearable device collects EEG and a remote server performs prediction (Zheng et al., 21 Jul 2025).

6. Interpretation, limitations, and relation to adjacent transformer-based work

The central contribution of the network is the unification of three objectives: permutation-invariant temporal aggregation, permutation-invariant channel aggregation, and patient-specific channel selection. The first two are architectural; the third is operational and directly connected to wearable deployment. This explains why the reported increase in sensitivity after channel selection on the even division is presented together with a mild increase in FPR: channel reduction is not framed as lossless compression, but as a trade-off that can improve sensitivity while slightly increasing false alarms (Zheng et al., 21 Jul 2025).

Several limitations are explicit in the reported results. Channel selection does not converge for all patients. Performance under seizure-independent division is lower than under even division. Only 7 patients qualify for the stricter split because sufficiently many seizure-independent interictal periods are required. The hardware discussion also makes clear that the method is not presented as an on-device wearable model in its current form, but as a candidate for wearable sensing combined with remote inference.

A common misconception is that any “channel-aware” transformer in the literature denotes the same mechanism. That is not the case. In LiDAR-based 3D detection, CT3D uses a two-stage detector with proposal-to-point encoding and a channel-wise decoding module for proposal refinement (Sheng et al., 2021). By contrast, the EEG model uses Set Transformer MABs across temporal segments and channels, and its “channel-aware” property refers to accumulated channel attention for dominant-channel selection rather than channel-wise decoding. The shared terminology reflects a broader trend toward attention mechanisms that preserve structured feature contributions, but the underlying tasks, representations, and architectural semantics are different.

Taken together, the paper’s main claim is not only that seizure prediction can be performed with fewer channels, but that evaluation methodology materially affects perceived performance. The proposed seizure-independent division is therefore as important to the paper’s technical identity as the two-stage network itself. A plausible implication is that future work on wearable seizure prediction will need to treat channel selection, deployment constraints, and leakage-resistant validation as a single integrated design problem rather than as separable concerns (Zheng et al., 21 Jul 2025).