ChineseEEG-2: Multimodal EEG for Chinese Language

• ChineseEEG-2 is a high-density EEG dataset for Chinese language processing that extends earlier silent reading studies by incorporating reading aloud and passive listening modalities.
• It employs synchronized recordings with token-level timing, explicit semantic embeddings, and EEG-BIDS compliant data formats to ensure precise multimodal alignment.
• The dataset supports cross-modal neural decoding and brain–LLM alignment research, serving as a benchmark for transfer learning and reproducible semantic analysis.

Searching arXiv for ChineseEEG-2 and closely related papers to ground the article. {"query":"ChineseEEG-2 EEG dataset multimodal semantic alignment neural decoding reading listening arXiv", "max_results": 10} ChineseEEG-2 is a high-density EEG dataset for multimodal semantic alignment and neural decoding during Chinese language processing, designed to benchmark neural decoding models under real-world language tasks (Chen et al., 6 Aug 2025). Building on the previous ChineseEEG dataset, which focused on silent reading, it adds two active modalities—Reading Aloud (RA) and Passive Listening (PL)—while retaining a shared Chinese corpus and explicit temporal alignment between brain activity, text, and speech (Chen et al., 6 Aug 2025). The resource includes EEG signals, precise audio, aligned semantic embeddings from pre-trained LLMs, and task labels, and is organized in an EEG-BIDS-compliant format intended for cross-modal neural decoding, brain–LLM alignment, transfer learning, and benchmark development (Chen et al., 6 Aug 2025).

1. Origin, scope, and relation to earlier Chinese EEG language resources

ChineseEEG-2 was introduced as a dataset for reading and listening modalities in Chinese, with the explicit goal of supporting semantic alignment and neural decoding at benchmark scale (Chen et al., 6 Aug 2025). In the dataset description, the Reading Aloud task comprises 4 participants (2 males, 2 females) and approximately 10.8 hours of EEG, while the Passive Listening task comprises 8 participants (4 males, 4 females) and approximately 21.6 hours, for a grand total of approximately 32.4 hours of continuous EEG (Chen et al., 6 Aug 2025). The dataset is presented as the first large-scale, token-aligned Chinese EEG resource for reading $\rightarrow$ speaking $\rightarrow$ listening modalities, complete with triggers, high-density scalp coverage, synchronized audio, and precomputed embeddings (Chen et al., 6 Aug 2025).

Its immediate antecedent is the earlier ChineseEEG corpus focused on silent reading. An exploratory study using a subset of that corpus, “EEG2TEXT-CN,” described high-density 128-channel EEG sampled at 256 Hz from native Chinese readers silently viewing text one character at a time, and used six participants reading Chapters 1–18 of Garnett Dream for sentence-level EEG–text alignment and zero-shot sentence generation (Lu et al., 1 Jun 2025). That work explicitly framed several design directions for a future ChineseEEG-2 resource, including multilingual expansion, richer annotations, larger subject pools, and additional evaluation metrics (Lu et al., 1 Jun 2025). This suggests that ChineseEEG-2 can be understood not merely as a new recording campaign, but as a scaling and modality-expansion step from silent-reading alignment toward joint speaking–listening–reading studies.

2. Experimental paradigms and corpus design

ChineseEEG-2 contains two principal paradigms: Reading Aloud and Passive Listening (Chen et al., 6 Aug 2025). In Reading Aloud, the stimuli are Chinese translations of The Little Prince and Garnett Dream. The former contributes 27 chapters and 21,829 characters, while the latter contributes 9 sections and 24,762 characters for one reader (Chen et al., 6 Aug 2025). Text was presented as three scrolling lines on a $1920 \times 1080$ display, with the middle line highlighted in red; participants read at a fixed pace of 0.25 s per character, pressed the spacebar after each line to proceed, paused 20 s between trials, and had a 1 h break between novels (Chen et al., 6 Aug 2025). Each subject completed 4 runs split over two sessions.

In Passive Listening, the stimuli were noise-reduced recordings from two Reading Aloud subjects, one male and one female, selected by acoustic-quality metrics (Chen et al., 6 Aug 2025). Participants listened binaurally via headphones while fixating on a central cross. They completed 4 runs per subject, except for one subject who completed 5 due to an interruption; the listened segments were matched to Reading Aloud chapters (Chen et al., 6 Aug 2025). Attention was monitored through comprehension questionnaires after each segment, and post-session free recall was recorded and transcribed via Whisper-large-v3 (Chen et al., 6 Aug 2025).

The shared corpus across Reading Aloud and Passive Listening is central to the dataset’s alignment logic. Because the same linguistic material is used across modalities, EEG traces from production and perception can be related to the same textual and audio segments. A plausible implication is that ChineseEEG-2 enables controlled comparisons of neural dynamics across speech production and speech perception without changing the underlying semantic content.

3. Signal acquisition, synchronization, and data packaging

ChineseEEG-2 was acquired with an Electrical Geodesics, Inc. HydroCel Geodesic Sensor Net providing 128 channels and full-scalp coverage (Chen et al., 6 Aug 2025). Electrode impedance was maintained below 50 k$\Omega$ using saline-soaked sponges. The Reading Aloud recordings were sampled at 250 Hz, whereas Passive Listening recordings were sampled at 1000 Hz before later harmonization in preprocessing (Chen et al., 6 Aug 2025). The initial reference followed EGI protocol, and average re-reference was applied after preprocessing.

Audio was recorded with a “high-fidelity” EGI-compatible microphone, although the exact model was not specified (Chen et al., 6 Aug 2025). Synchronization relied on hardware trigger lines generated from PsychoPy and fed into the EEG amplifier. Marker types included BEGN/STOP for session start and stop, CHxx for chapter onset, ROWS/ROWE for text-line start and end, and PRES/PREE for preface start and end (Chen et al., 6 Aug 2025). The dataset description states that character-rate timing at 0.25 s per character was used for millisecond-level “character-to-EEG” refinement.

The resource is distributed in BIDS form. Raw files are exported to .mff and converted to BrainVision (.eeg/.vhdr/.vmrk) under BIDS, while derivatives include filtered, ICA-cleaned, re-referenced data in .fif or .set formats (Chen et al., 6 Aug 2025). Text and audio embeddings are stored under /derivatives/embeddings/text/*.npy and audio/*.npy, and event annotations are stored as events.tsv files. Access is provided through the DOI https://doi.org/10.57760/sciencedb.CHNNeuro.00001 and the GitHub repository https://github.com/ncclab-sustech/ChineseEEG-2 (Chen et al., 6 Aug 2025).

Component	Specification
EEG system	EGI HydroCel Geodesic Sensor Net, 128 channels
Sampling	250 Hz (RA), 1000 Hz (PL), then downsampled to 250 Hz in preprocessing
Formats	BrainVision raw under BIDS; derivatives in .fif or .set
Markers	BEGN/STOP, CHxx, ROWS/ROWE, PRES/PREE
Embeddings	BERT-base Chinese text embeddings; wav2vec2 audio embeddings

4. Preprocessing and semantic alignment methodology

The preprocessing pipeline for ChineseEEG-2 is specified in MNE-Python v1.6.0 and MNE-BIDS v0.14 (Chen et al., 6 Aug 2025). Non-task intervals such as practice trials and silent buffers are trimmed first. All EEG data are then downsampled to 250 Hz, including Passive Listening recordings originally acquired at 1000 Hz. A 50 Hz notch filter and a 1–40 Hz bandpass are applied, with the conventional band definitions $\delta: 0.5\!-\!4$ Hz, $\theta: 4\!-\!8$ Hz, $\alpha: 8\!-\!12$ Hz, and $\beta: 12\!-\!30$ Hz (Chen et al., 6 Aug 2025). Bad channels are detected and interpolated using spherical-spline interpolation; ICA is run with FastICA and 40 components for semi-automated removal of ocular and cardiac artifacts; baseline correction and average re-reference follow; the outputs are saved in BIDS-Derivatives format (Chen et al., 6 Aug 2025).

Semantic alignment is based on paired embeddings and explicit temporal anchoring rather than a mandated end-to-end decoder (Chen et al., 6 Aug 2025). Text embeddings are extracted from BERT-base Chinese using transformers v4.36.2, and audio embeddings are extracted from ehcalabres/wav2vec2-lg-xlsr-en-speech-emotion-recognition (Chen et al., 6 Aug 2025). Text embeddings are generated at token level for each Chinese character or word span and saved as .npy arrays per segment; audio embeddings are generated at frame level and aligned to the same segments via trigger timestamps (Chen et al., 6 Aug 2025).

The alignment procedure has three stated stages. First, hardware triggers time-lock EEG to the onset and offset of each text line or audio file. Second, for reading, the fixed 0.25 s per character provides a per-character timestamp grid. Third, for listening, CHxx segment boundaries pin audio to EEG, and within-segment mapping is achieved by matching audio sample indices to the EEG clock (Chen et al., 6 Aug 2025). The paper gives illustrative objectives often used in prior work, including a CCA objective,

$$\max_{\mathbf{w}_e,\mathbf{w}_s}\;\mathrm{corr}\bigl(\mathbf{w}_e^T X_{EEG},\,\mathbf{w}_s^T X_{SEM}\bigr),$$

and an alignment loss,

$$\mathcal{L}_{align} \;=\;\bigl\|f_{EEG}(X_{EEG}) \;-\; f_{SEM}(X_{SEM})\bigr\|_2^2,$$

but explicitly notes that ChineseEEG-2 supplies aligned data rather than prescribing a single alignment model (Chen et al., 6 Aug 2025). This distinction is important: the dataset is a benchmark substrate, not a fixed algorithmic pipeline.

5. Validation, benchmark orientation, and intended use

The ChineseEEG-2 paper emphasizes dataset creation and validation rather than a single headline decoding score (Chen et al., 6 Aug 2025). The reported validation metrics are Inter-Subject Correlation (ISC) per band and source-level cross-task correlation. Genuine ISC is reported to be much greater than a shuffled baseline, with significance assessed by one-sample $\rightarrow$0-test at $\rightarrow$1, and source-level cross-task correlation in language ROIs versus control regions is evaluated with Steiger’s $\rightarrow$2 test at $\rightarrow$3 (Chen et al., 6 Aug 2025). These measures are used to establish that the recordings contain reliable task-related neural structure and cross-modal language-related effects.

The benchmark role of the dataset is defined broadly. The paper states that ChineseEEG-2 is designed to support future evaluations of linear models such as Ridge and Lasso, Canonical Correlation or PLS regression, deep architectures including LSTM and Transformer-based EEG encoders, and multimodal fusion and contrastive training strategies (Chen et al., 6 Aug 2025). Intended use cases include cross-modal neural decoding, for example training on Reading Aloud and testing on Passive Listening or silent reading; brain–LLM alignment by mapping EEG signals into LLM embedding spaces for real-time semantic decoding; transfer learning by pre-training EEG encoders on multi-task data and fine-tuning for BCI text or command decoding; and benchmark development comparing linear, deep, and contrastive methods in a Chinese language setting (Chen et al., 6 Aug 2025).

A notable property of the resource is that it packages not only raw and cleaned EEG but also precomputed semantic representations. This lowers the barrier to comparative studies whose primary novelty lies in the neural encoder, cross-modal objective, or transfer protocol rather than in repeated feature extraction from text and audio. A plausible implication is that ChineseEEG-2 is particularly well suited for reproducible representation-alignment experiments.

6. Position within Chinese EEG language-decoding research

ChineseEEG-2 occupies a different methodological niche from earlier Chinese EEG language-decoding studies. In “Speech Recognition With No Speech Or With Noisy Speech Beyond English,” continuous noisy speech recognition was demonstrated using CTC models on a limited Chinese vocabulary and on a limited joint English and Chinese vocabulary, with 12 healthy adult subjects, a 32-channel EEG cap, handcrafted EEG feature sets and raw EEG, and Character Error Rate as the evaluation metric (Krishna et al., 2019). That work established feasibility for end-to-end EEG-based recognition under constrained vocabularies and showed that CER increased sharply as the number of unique characters grew (Krishna et al., 2019). By contrast, ChineseEEG-2 is a dataset resource rather than a single recognition model, and it is organized around multimodal semantic alignment across reading, speaking, and listening rather than limited-vocabulary CTC decoding.

A second relevant strand concerns language-specific neural structure. “Decoding Neural Correlation of Language-Specific Imagined Speech using EEG Signals” investigated English and Chinese imagined-speech tasks in 12 right-handed adults and reported that, in the Chinese task, theta-band relative power spectral density was significantly elevated over inferior parietal and occipital regions, including electrodes $\rightarrow$4 (Lee et al., 2022). That study did not report classifiers or decoding accuracies, but it identified spectral and spatial contrasts relevant to Chinese-language imagination (Lee et al., 2022). Relative to such hypothesis-driven analyses, ChineseEEG-2 provides a much broader multimodal benchmark with synchronized language stimuli and embedding annotations.

The nearest methodological bridge to ChineseEEG-2 is the silent-reading alignment literature represented by EEG2TEXT-CN. That study used a subset of ChineseEEG consisting of six participants, approximately 9,000 sentence–EEG pairs for training and validation, and 300 held-out test sentences, with each sentence containing approximately 9–10 Chinese characters and each character highlighted for 350 ms (Lu et al., 1 Jun 2025). Its NICE-EEG–MiniLM architecture aligned per-character EEG and text embeddings with masked pretraining and InfoNCE contrastive learning, and zero-shot sentence generation on a held-out subject achieved a best BLEU-1 score of 6.38% (Lu et al., 1 Jun 2025). ChineseEEG-2 extends the ecosystem in a different direction: rather than only silent reading, it enables reading aloud and passive listening with shared corpora and token-level timing, thereby broadening the space of cross-modal alignment and transfer experiments.

7. Conceptual significance and open directions

ChineseEEG-2 is defined by the conjunction of three properties: high-density EEG, multimodal language tasks, and explicit semantic alignment scaffolding (Chen et al., 6 Aug 2025). Because it pairs neural recordings with text and audio representations under matched linguistic content, it supports research questions that are difficult to address with single-modality corpora, including whether production- and perception-related EEG can be mapped into shared semantic spaces and whether encoders trained on one task transfer to another (Chen et al., 6 Aug 2025). The dataset description explicitly identifies applications in neural semantic decoding and brain–LLM alignment.

Several future directions have already been articulated in adjacent work. EEG2TEXT-CN proposed multilingual expansion to English, Mandarin, and Cantonese; higher sampling rates or overlapping character windows; richer annotations such as eye-tracking, self-paced reading times, semantic role labeling, and subcharacter stroke order; a larger subject pool with cross-session recordings; and additional evaluation metrics such as BERTScore, cosine similarity in embedding space, and human fluency judgments (Lu et al., 1 Jun 2025). These are not features of the released ChineseEEG-2 dataset, but they identify a clear trajectory for subsequent benchmark design. This suggests that ChineseEEG-2 should be seen as a foundational resource within an evolving program of Chinese brain–language datasets, situated between earlier constrained decoding studies and later open-vocabulary, brain-to-text alignment systems.