Transferability of the EEG-based speech decoder to new participants

Determine whether the EEG-based speech decoding system trained via CLIP alignment between EEG and pre-trained audio embeddings using 175 hours of overt speech data from a single healthy participant can be transferred to other participants, and quantify the amount of participant-specific data required to enable such transferability through fine-tuning or adaptation.

Background

The paper reports unprecedented zero-shot classification performance for EEG-based speech decoding by training an EEG encoder to align with pre-trained audio encoders using 175 hours of data from one participant. Despite these results, the dataset involves only a single individual, and the authors explicitly note uncertainty about generalizing the system to others.

They suggest that models pre-trained on large datasets often transfer with modest fine-tuning and hypothesize the same might hold for speech decoding, emphasizing the need to measure how much data is required for adaptation across individuals.