---
title: 'ChineseEEG-2: Multimodal EEG for Chinese Language'
url: https://www.emergentmind.com/topics/chineseeeg-2
type: topic
---

# ChineseEEG-2: Multimodal EEG for Chinese Language

Searching arXiv for ChineseEEG-2 and closely related papers to ground the article.
{"query":"ChineseEEG-2 EEG dataset multimodal semantic alignment neural decoding reading listening arXiv", "max_results": 10}
ChineseEEG-2 is a high-density EEG dataset for multimodal semantic alignment and neural decoding during Chinese language processing, designed to benchmark neural decoding models under real-world language tasks [2508.04240]. Building on the previous ChineseEEG dataset, which focused on silent reading, it adds two active modalities—Reading Aloud (RA) and Passive Listening (PL)—while retaining a shared Chinese corpus and explicit temporal alignment between brain activity, text, and speech [2508.04240]. The resource includes EEG signals, precise audio, aligned semantic embeddings from pre-trained language models, and task labels, and is organized in an EEG-BIDS-compliant format intended for cross-modal neural decoding, brain–LLM alignment, transfer learning, and benchmark development [2508.04240].

## 1. Origin, scope, and relation to earlier Chinese EEG language resources

ChineseEEG-2 was introduced as a dataset for reading and listening modalities in Chinese, with the explicit goal of supporting semantic alignment and neural decoding at benchmark scale [2508.04240]. In the dataset description, the Reading Aloud task comprises 4 participants (2 males, 2 females) and approximately 10.8 hours of EEG, while the Passive Listening task comprises 8 participants (4 males, 4 females) and approximately 21.6 hours, for a grand total of approximately 32.4 hours of continuous EEG [2508.04240]. The dataset is presented as the first large-scale, token-aligned Chinese EEG resource for reading \(\rightarrow\) speaking \(\rightarrow\) listening modalities, complete with triggers, high-density scalp coverage, synchronized audio, and precomputed embeddings [2508.04240].

Its immediate antecedent is the earlier ChineseEEG corpus focused on silent reading. An exploratory study using a subset of that corpus, “EEG2TEXT-CN,” described high-density 128-channel EEG sampled at 256 Hz from native Chinese readers silently viewing text one character at a time, and used six participants reading Chapters 1–18 of *Garnett Dream* for sentence-level EEG–text alignment and zero-shot sentence generation [2506.00854]. That work explicitly framed several design directions for a future ChineseEEG-2 resource, including multilingual expansion, richer annotations, larger subject pools, and additional evaluation metrics [2506.00854]. This suggests that ChineseEEG-2 can be understood not merely as a new recording campaign, but as a scaling and modality-expansion step from silent-reading alignment toward joint speaking–listening–reading studies.

## 2. Experimental paradigms and corpus design

ChineseEEG-2 contains two principal paradigms: Reading Aloud and Passive Listening [2508.04240]. In Reading Aloud, the stimuli are Chinese translations of *The Little Prince* and *Garnett Dream*. The former contributes 27 chapters and 21,829 characters, while the latter contributes 9 sections and 24,762 characters for one reader [2508.04240]. Text was presented as three scrolling lines on a \(1920 \times 1080\) display, with the middle line highlighted in red; participants read at a fixed pace of 0.25 s per character, pressed the spacebar after each line to proceed, paused 20 s between trials, and had a 1 h break between novels [2508.04240]. Each subject completed 4 runs split over two sessions.

In Passive Listening, the stimuli were noise-reduced recordings from two Reading Aloud subjects, one male and one female, selected by acoustic-quality metrics [2508.04240]. Participants listened binaurally via headphones while fixating on a central cross. They completed 4 runs per subject, except for one subject who completed 5 due to an interruption; the listened segments were matched to Reading Aloud chapters [2508.04240]. Attention was monitored through comprehension questionnaires after each segment, and post-session free recall was recorded and transcribed via Whisper-large-v3 [2508.04240].

The shared corpus across Reading Aloud and Passive Listening is central to the dataset’s alignment logic. Because the same linguistic material is used across modalities, EEG traces from production and perception can be related to the same textual and audio segments. A plausible implication is that ChineseEEG-2 enables controlled comparisons of neural dynamics across speech production and speech perception without changing the underlying semantic content.

## 3. Signal acquisition, synchronization, and data packaging

ChineseEEG-2 was acquired with an Electrical Geodesics, Inc. HydroCel Geodesic Sensor Net providing 128 channels and full-scalp coverage [2508.04240]. Electrode impedance was maintained below 50 k\(\Omega\) using saline-soaked sponges. The Reading Aloud recordings were sampled at 250 Hz, whereas Passive Listening recordings were sampled at 1000 Hz before later harmonization in preprocessing [2508.04240]. The initial reference followed EGI protocol, and average re-reference was applied after preprocessing.

Audio was recorded with a “high-fidelity” EGI-compatible microphone, although the exact model was not specified [2508.04240]. Synchronization relied on hardware trigger lines generated from PsychoPy and fed into the EEG amplifier. Marker types included BEGN/STOP for session start and stop, CHxx for chapter onset, ROWS/ROWE for text-line start and end, and PRES/PREE for preface start and end [2508.04240]. The dataset description states that character-rate timing at 0.25 s per character was used for millisecond-level “character-to-EEG” refinement.

The resource is distributed in BIDS form. Raw files are exported to `.mff` and converted to BrainVision (`.eeg/.vhdr/.vmrk`) under BIDS, while derivatives include filtered, ICA-cleaned, re-referenced data in `.fif` or `.set` formats [2508.04240]. Text and audio embeddings are stored under `/derivatives/embeddings/text/*.npy` and `audio/*.npy`, and event annotations are stored as `events.tsv` files. Access is provided through the DOI `https://doi.org/10.57760/sciencedb.CHNNeuro.00001` and the GitHub repository `https://github.com/ncclab-sustech/ChineseEEG-2` [2508.04240].

| Component | Specification |
|---|---|
| EEG system | EGI HydroCel Geodesic Sensor Net, 128 channels |
| Sampling | 250 Hz (RA), 1000 Hz (PL), then downsampled to 250 Hz in preprocessing |
| Formats | BrainVision raw under BIDS; derivatives in `.fif` or `.set` |
| Markers | BEGN/STOP, CHxx, ROWS/ROWE, PRES/PREE |
| Embeddings | BERT-base Chinese text embeddings; wav2vec2 audio embeddings |

## 4. Preprocessing and semantic alignment methodology

The preprocessing pipeline for ChineseEEG-2 is specified in MNE-Python v1.6.0 and MNE-BIDS v0.14 [2508.04240]. Non-task intervals such as practice trials and silent buffers are trimmed first. All EEG data are then downsampled to 250 Hz, including Passive Listening recordings originally acquired at 1000 Hz. A 50 Hz notch filter and a 1–40 Hz bandpass are applied, with the conventional band definitions \(\delta: 0.5\!-\!4\) Hz, \(\theta: 4\!-\!8\) Hz, \(\alpha: 8\!-\!12\) Hz, and \(\beta: 12\!-\!30\) Hz [2508.04240]. Bad channels are detected and interpolated using spherical-spline interpolation; ICA is run with FastICA and 40 components for semi-automated removal of ocular and cardiac artifacts; baseline correction and average re-reference follow; the outputs are saved in BIDS-Derivatives format [2508.04240].

Semantic alignment is based on paired embeddings and explicit temporal anchoring rather than a mandated end-to-end decoder [2508.04240]. Text embeddings are extracted from BERT-base Chinese using `transformers v4.36.2`, and audio embeddings are extracted from `ehcalabres/wav2vec2-lg-xlsr-en-speech-emotion-recognition` [2508.04240]. Text embeddings are generated at token level for each Chinese character or word span and saved as `.npy` arrays per segment; audio embeddings are generated at frame level and aligned to the same segments via trigger timestamps [2508.04240].

The alignment procedure has three stated stages. First, hardware triggers time-lock EEG to the onset and offset of each text line or audio file. Second, for reading, the fixed 0.25 s per character provides a per-character timestamp grid. Third, for listening, CHxx segment boundaries pin audio to EEG, and within-segment mapping is achieved by matching audio sample indices to the EEG clock [2508.04240]. The paper gives illustrative objectives often used in prior work, including a CCA objective,
\[
\max_{\mathbf{w}_e,\mathbf{w}_s}\;\mathrm{corr}\bigl(\mathbf{w}_e^T X_{EEG},\,\mathbf{w}_s^T X_{SEM}\bigr),
\]
and an alignment loss,
\[
\mathcal{L}_{align} \;=\;\bigl\|f_{EEG}(X_{EEG}) \;-\; f_{SEM}(X_{SEM})\bigr\|_2^2,
\]
but explicitly notes that ChineseEEG-2 supplies aligned data rather than prescribing a single alignment model [2508.04240]. This distinction is important: the dataset is a benchmark substrate, not a fixed algorithmic pipeline.

## 5. Validation, benchmark orientation, and intended use

The ChineseEEG-2 paper emphasizes dataset creation and validation rather than a single headline decoding score [2508.04240]. The reported validation metrics are Inter-Subject Correlation (ISC) per band and source-level cross-task correlation. Genuine ISC is reported to be much greater than a shuffled baseline, with significance assessed by one-sample \(t\)-test at \(p < 0.001\), and source-level cross-task correlation in language ROIs versus control regions is evaluated with Steiger’s \(Z\) test at \(p < 0.05\) [2508.04240]. These measures are used to establish that the recordings contain reliable task-related neural structure and cross-modal language-related effects.

The benchmark role of the dataset is defined broadly. The paper states that ChineseEEG-2 is designed to support future evaluations of linear models such as Ridge and Lasso, Canonical Correlation or PLS regression, deep architectures including LSTM and Transformer-based EEG encoders, and multimodal fusion and contrastive training strategies [2508.04240]. Intended use cases include cross-modal neural decoding, for example training on Reading Aloud and testing on Passive Listening or silent reading; brain–LLM alignment by mapping EEG signals into LLM embedding spaces for real-time semantic decoding; transfer learning by pre-training EEG encoders on multi-task data and fine-tuning for BCI text or command decoding; and benchmark development comparing linear, deep, and contrastive methods in a Chinese language setting [2508.04240].

A notable property of the resource is that it packages not only raw and cleaned EEG but also precomputed semantic representations. This lowers the barrier to comparative studies whose primary novelty lies in the neural encoder, cross-modal objective, or transfer protocol rather than in repeated feature extraction from text and audio. A plausible implication is that ChineseEEG-2 is particularly well suited for reproducible representation-alignment experiments.

## 6. Position within Chinese EEG language-decoding research

ChineseEEG-2 occupies a different methodological niche from earlier Chinese EEG language-decoding studies. In “Speech Recognition With No Speech Or With Noisy Speech Beyond English,” continuous noisy speech recognition was demonstrated using CTC models on a limited Chinese vocabulary and on a limited joint English and Chinese vocabulary, with 12 healthy adult subjects, a 32-channel EEG cap, handcrafted EEG feature sets and raw EEG, and Character Error Rate as the evaluation metric [1906.08045]. That work established feasibility for end-to-end EEG-based recognition under constrained vocabularies and showed that CER increased sharply as the number of unique characters grew [1906.08045]. By contrast, ChineseEEG-2 is a dataset resource rather than a single recognition model, and it is organized around multimodal semantic alignment across reading, speaking, and listening rather than limited-vocabulary CTC decoding.

A second relevant strand concerns language-specific neural structure. “Decoding Neural Correlation of Language-Specific Imagined Speech using EEG Signals” investigated English and Chinese imagined-speech tasks in 12 right-handed adults and reported that, in the Chinese task, theta-band relative power spectral density was significantly elevated over inferior parietal and occipital regions, including electrodes \(\{\mathrm{O1}, \mathrm{Oz}, \mathrm{O2}, \mathrm{P5}, \mathrm{PO7}\}\) [2204.07362]. That study did not report classifiers or decoding accuracies, but it identified spectral and spatial contrasts relevant to Chinese-language imagination [2204.07362]. Relative to such hypothesis-driven analyses, ChineseEEG-2 provides a much broader multimodal benchmark with synchronized language stimuli and embedding annotations.

The nearest methodological bridge to ChineseEEG-2 is the silent-reading alignment literature represented by EEG2TEXT-CN. That study used a subset of ChineseEEG consisting of six participants, approximately 9,000 sentence–EEG pairs for training and validation, and 300 held-out test sentences, with each sentence containing approximately 9–10 Chinese characters and each character highlighted for 350 ms [2506.00854]. Its NICE-EEG–MiniLM architecture aligned per-character EEG and text embeddings with masked pretraining and InfoNCE contrastive learning, and zero-shot sentence generation on a held-out subject achieved a best BLEU-1 score of 6.38% [2506.00854]. ChineseEEG-2 extends the ecosystem in a different direction: rather than only silent reading, it enables reading aloud and passive listening with shared corpora and token-level timing, thereby broadening the space of cross-modal alignment and transfer experiments.

## 7. Conceptual significance and open directions

ChineseEEG-2 is defined by the conjunction of three properties: high-density EEG, multimodal language tasks, and explicit semantic alignment scaffolding [2508.04240]. Because it pairs neural recordings with text and audio representations under matched linguistic content, it supports research questions that are difficult to address with single-modality corpora, including whether production- and perception-related EEG can be mapped into shared semantic spaces and whether encoders trained on one task transfer to another [2508.04240]. The dataset description explicitly identifies applications in neural semantic decoding and brain–LLM alignment.

Several future directions have already been articulated in adjacent work. EEG2TEXT-CN proposed multilingual expansion to English, Mandarin, and Cantonese; higher sampling rates or overlapping character windows; richer annotations such as eye-tracking, self-paced reading times, semantic role labeling, and subcharacter stroke order; a larger subject pool with cross-session recordings; and additional evaluation metrics such as BERTScore, cosine similarity in embedding space, and human fluency judgments [2506.00854]. These are not features of the released ChineseEEG-2 dataset, but they identify a clear trajectory for subsequent benchmark design. This suggests that ChineseEEG-2 should be seen as a foundational resource within an evolving program of Chinese brain–language datasets, situated between earlier constrained decoding studies and later open-vocabulary, brain-to-text alignment systems.

Source: https://www.emergentmind.com/topics/chineseeeg-2