Transferability of the EEG-based speech decoder to new participants

Determine whether the EEG-based speech decoding system trained via CLIP alignment between EEG and pre-trained audio embeddings using 175 hours of overt speech data from a single healthy participant can be transferred to other participants, and quantify the amount of participant-specific data required to enable such transferability through fine-tuning or adaptation.

Background

The study reports unprecedented zero-shot classification performance for EEG-based speech decoding by training an EEG encoder to align with pre-trained audio encoders using 175 hours of data from one participant. Despite these results, the dataset involves only a single individual, and the authors explicitly note uncertainty about generalizing the system to others.

They suggest that models pre-trained on large datasets often transfer with modest fine-tuning and hypothesize the same might hold for speech decoding, emphasizing the need to measure how much data is required for adaptation across individuals.

References

As such, it is unclear whether this system can be transferred to other participants.

— Scaling Law in Neural Data: Non-Invasive Speech Decoding with 175 Hours of EEG Data  (2407.07595 - Sato et al., 2024) in Section 6 Limitations

We tested this only with the 26-subject pretraining pool, so we cannot say whether the adapter would earn its place when fewer subjects are available for pretraining.

— Multi-Subject Pretraining Enables Short-Calibration Personalization for Closed-Corpus Surface EMG Speech Decoding  (2609.21288 - Le et al., 18 Sep 2026) in Section 4.4, “No Detectable Benefit of the Subject-Specific MLP”

We did not determine whether this dependence would persist with a larger pretraining cohort, nor did we search for an optimization schedule that stabilizes training from random initialization; the schedule was held fixed across conditions by design, so the comparison shows what the released checkpoint contributes under this schedule rather than an upper bound on what random initialization could achieve if tuned separately.

— Multi-Subject Pretraining Enables Short-Calibration Personalization for Closed-Corpus Surface EMG Speech Decoding  (2609.21288 - Le et al., 18 Sep 2026) in Section 5.5, “Limitations”

We suspect joint training on Subject 0 and Subjects 1--32 will improve generalisation on the latter subjects further, but leave this and other avenues of exploration to competition participants.

— The 2026 PNPL Competition: Word Classification and Efficient Cross-Subject Generalisation in LibriBrain100  (2609.03231 - Mantegna et al., 3 Sep 2026) in Section “Baselines, code, and material provided”

Our evaluation is limited to six participants across two listening datasets, with a separate decoder trained for each participant; generalization to new individuals and to other tasks remains untested, and extending NOEMA across participants is a natural next step.

— Decoding Naturalistic Speech from fMRI with Temporal Modeling and Multimodal Alignment  (2610.22049 - Caris et al., 15 Oct 2026) in Section Conclusion, subsection “Limitations and future work”

First, our evaluation is conducted on single-subject datasets, and cross-subject generalization remains an open challenge due to variability in neural representations.

— Do Speech BCIs Need Larger Models? Rethinking Neural Decoding beyond Scaling  (2610.23175 - Feng et al., 16 Oct 2026) in Section 5, “Limitations and future directions”