MEEtBrain: Multimodal Brain-Mind Frameworks

• MEEtBrain is a family of research frameworks that leverage metastability to explain dynamic brain–mind coordination and temporally rich neural interactions.
• It operationalizes the mapping of M/EEG signals to pretrained multimodal spaces for applications like zero-shot visual decoding, speech reconstruction, and affective computing.
• The approach extends to portable, wearable systems and embodied control paradigms, enabling scalable speech decoding, cross-session neural ensemble modeling, and robust brain–robot interactions.

Searching arXiv for papers and usages of “MEEtBrain” to ground the article. MEEtBrain is a label used in several recent research contexts to denote frameworks that connect brain activity to metastable coordination, multimodal embedding spaces, portable affective sensing, or embodied control. In one formulation, it is explicitly the Metastable Brain–Mind Framework, derived from Coordination Dynamics and opposed to a narrow reading of the critical brain hypothesis (Kelso, 2023). In later work, the term is used directly for a portable EEG–fNIRS emotion-analysis system driven by AI-generated music (Zhao et al., 5 Aug 2025), and is also invoked more broadly for M/EEG-to-multimodal alignment, large-scale MEG speech decoding, edge-AI brain–robot interaction, and multi-view modeling of cross-regional neural dynamics (Xu et al., 18 Sep 2025, Özdogan et al., 2 Jun 2025, Ouyang et al., 2024, Mudrik et al., 2024). This suggests that MEEtBrain is best understood as a family of related research programs rather than a single canonical architecture.

1. Semantic range of the term

In the cited literature, “MEEtBrain” has both a narrow and a broad usage. Narrowly, it names a metastability-centered theory of brain~mind coordination. Broadly, it functions as a programmatic label for systems that map neural measurements to latent representations, semantic codes, affective states, or downstream actions.

Usage in the literature	Primary substrate	Core objective
Metastable Brain–Mind Framework	Coordination dynamics across behavior and neural recordings	Explain coexisting integration and segregation tendencies
UMind as MEEtBrain-style decoding	M/EEG aligned to CLIP image/text spaces	Zero-shot retrieval, classification, and reconstruction
LibriBrain for MEEtBrain	Single-subject MEG speech corpus	Scale speech, phoneme, and word decoding
Portable MEEtBrain	EEG–fNIRS with AI-generated music	Valence/arousal decoding in a wearable setup
MEEtBrain-aligned systems	Heterogeneous EEG, metasurfaces, multi-view recordings	Multi-robot control, EM communication, cross-session dynamics

A recurrent theme across these usages is the replacement of fixed-state or single-task interpretations with richer, time-varying, and multimodal formulations. In theoretical work, this appears as metastable coordination beyond critical tuning. In applied work, it appears as brain-to-embedding alignment, multimodal fusion, semantic communication, and cross-session sub-circuit discovery (Kelso, 2023, Xu et al., 18 Sep 2025, Ouyang et al., 2024).

2. Metastability and the original brain~mind formulation

In (Kelso, 2023), MEEtBrain is explicitly the Metastable Brain–Mind Framework. Its central claim is that the functioning brain has “a vast repertoire of coexisting tendencies/dispositions for regions of the brain to integrate and segregate at the same time,” and that brain~mind lives in a “sea of metastability” rather than at a single finely tuned critical point. The paper therefore contrasts metastability with the critical brain hypothesis, which emphasizes self-tuning near a critical point between order and disorder. By contrast, Coordination Dynamics emphasizes non-equilibrium phase transitions under control parameters such as neuromodulator concentration.

The formal backbone is the extended Haken–Kelso–Bunz phase equation, with relative phase $\phi$ as collective variable:

$$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$

Here $\Delta\omega$ denotes detuning, $a$ and $b$ are coupling parameters, and $\xi(t)$ is noise. When $\Delta\omega = 0$ and noise is neglected, the dynamics can be written in terms of the potential

$V(\phi) = -a\cos(\phi) - b\cos(2\phi).$

Canonical fixed points occur at $\phi = 0$ and $\phi = \pi$. In-phase is stable if $$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$0, and antiphase is stable if $$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$1, equivalently $$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$2. Loss of antiphase stability occurs when $$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$3. A sufficient condition for the disappearance of all fixed points is

$$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$4

in which case continuous phase drift replaces stationary coordination. The paper characterizes this regime as one in which only “ghost states” remain: no stable or unstable states survive, but there are still tendencies toward where such states once existed.

At scale, global coordination is summarized by the Kuramoto order parameter

$$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$5

Metastability is then indexed by fluctuations of $$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$6 over time, dwell-time distributions near $$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$7 or $$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$8, and time-varying phase-locking values. The same framework supports an informational reading. Whereas the critical brain hypothesis emphasizes “optimal information processing,” the metastable brain~mind is described as “creating information,” with transfer entropy, integrated information $$\frac{d\phi}{dt} = \Delta\omega - a\sin(\phi) - 2b\sin(2\phi) + \xi(t).$$9, and multivariate synergy presented as relevant descriptors of transient coalitions and relational codes (Kelso, 2023).

Empirically, the framework is connected to bimanual coordination, EEG and MEG phase coordination, fMRI dynamic functional connectivity, and meso/micro-scale neuronal assemblies. The paper also develops a translational vocabulary of dysmetastability: Parkinson’s disease is associated with rigidity or overly deep attractors, epilepsy with excessive synchronization and reduced variance in $\Delta\omega$0, schizophrenia with dysregulated integration–segregation balance, and disorders of consciousness with diminished metastability. In this sense, MEEtBrain is not simply a theory of fluctuation; it is a claim that adaptive brain function depends on the coexistence of coordination and independence across wide parameter ranges rather than on occupancy of a unique critical point (Kelso, 2023).

3. Multimodal decoding and generative alignment

A second major usage appears in work that treats MEEtBrain as an M/EEG-centered bridge to pretrained multimodal representation spaces. The clearest example is UMind, which explicitly states that it serves the aims behind a “MEEtBrain”-style approach by aligning time-resolved M/EEG to frozen CLIP image and text embeddings for zero-shot decoding and reconstruction (Xu et al., 18 Sep 2025).

UMind addresses three tasks jointly: visual stimulus retrieval, visual classification, and visual reconstruction. Zero-shot generalization is enforced by disjoint training and test categories, $\Delta\omega$1. Its architecture uses an M/EEG encoder $\Delta\omega$2, an image projector $\Delta\omega$3 producing neural-visual vectors $\Delta\omega$4, and a text projector $\Delta\omega$5 producing neural-semantic vectors $\Delta\omega$6. Coarse-grained text consists of category labels, while fine-grained text consists of captions generated by LLaVA-1.5. Retrieval and classification are performed by cosine matching in CLIP space, and reconstruction uses dual conditioning of SDXL-Turbo through an IP-Adapter path for visual guidance and a Q-Former path for semantic guidance. Pretraining combines CLIP-style InfoNCE alignment and MSE alignment:

$\Delta\omega$7

with $\Delta\omega$8, $\Delta\omega$9, and learnable temperature $a$0 initialized to $a$1 in the reported experiments.

On THINGS-EEG, UMind reports average retrieval of top-1 33.85% and top-5 66.70%, with average zero-shot classification of top-1 12.00% and top-5 34.40%. On THINGS-MEG, the corresponding averages are 19.38% and 44.75% for retrieval and 9.00% and 27.00% for classification. Reconstruction results on THINGS-EEG include PixCorr 0.156, SSIM 0.390, Inception 0.744, and CLIP 0.798; the paper emphasizes that the model trades some low-level metrics for stronger high-level visual and semantic fidelity. Ablations show that coarse and fine text are complementary: on THINGS-EEG, removing text entirely drops retrieval to 20.30% / 44.25% and classification to 0.55% / 2.40%, while using both coarse and fine text restores the full 33.85% / 66.70% retrieval and 12.00% / 34.40% classification performance (Xu et al., 18 Sep 2025).

The paper also reports spatiotemporal analyses. Performance stabilizes around $a$2 ms in expanding windows, above-chance retrieval is concentrated largely between 100–500 ms, and retrieval is supported primarily by occipital > parietal > temporal electrodes. Using only occipital electrodes yields an average top-1 retrieval of 19.7%, only 4.15% below using all channels. Within the broader MEEtBrain lexicon, this line of work operationalizes the idea that brain signals can be mapped directly into pretrained semantic and visual latent spaces, enabling zero-shot discrimination and multimodal generation without class-specific supervision on held-out categories (Xu et al., 18 Sep 2025).

4. Speech-decoding infrastructure and within-subject scaling

A third usage of MEEtBrain concerns large-scale speech decoding from MEG. LibriBrain is presented as “using LibriBrain to scale MEG speech decoding” in MEEtBrain, supplying infrastructure rather than a single end task (Özdogan et al., 2 Jun 2025). The dataset comprises 52.32 hours of MEG across 93 sessions in the current release, with two additional sessions held back for competitions. It contains 466,230 words and 1,511,732 phonemes overall, with the training split alone containing 459,227 words and 1,488,392 phonemes. The standard session split is 91 train / 1 val / 1 test, with validation on Sherlock book 1 chapter 11 and test on chapter 12, each recorded on a different day from training.

The rationale is explicitly within-subject depth. The paper argues that, when total hours are matched, deep within-subject training often outperforms broader-but-shallow training by reducing inter-subject variability and improving specialization. Recordings use a MEGIN Triux Neo system with 306 channels at 1,000 Hz, then undergo Maxwell filtering with spatial SSS, notch filtering at 50 Hz and 100 Hz, band-pass filtering at 0.1–125 Hz, and downsampling to 250 Hz. The release is sensor-space only, and the forward model is written as

$a$3

where $a$4 is the sensor measurement, $a$5 the leadfield, $a$6 the source distribution, and $a$7 measurement noise. The paper emphasizes that reproducing the baselines does not require source localization (Özdogan et al., 2 Jun 2025).

Three foundational tasks are supplied: speech detection, 39-way ARPAbet phoneme classification, and top-250 word classification. Baseline speech detection uses a lightweight CNN adapted from SEANet and reports F1 = 0.8989 \pm 0.0007, Balanced Accuracy = 0.7082 \pm 0.0031, AUROC = 0.8644 \pm 0.0017, Jaccard = 0.6077 \pm 0.0027, and cross-entropy = 0.3802 \pm 0.0044 on the test partition. Phoneme classification reports Micro F1 = 0.1168 \pm 0.0003, Balanced Accuracy = 0.0399 \pm 0.0003, Micro AUROC = 0.6360 \pm 0.0010, Macro AUROC = 0.6527 \pm 0.0013, and cross-entropy = 3.2332 \pm 0.0026. Averaging repeated phoneme tokens raises Balanced Accuracy above 60% with 100 repetitions. Word classification, implemented with a spatial-attention signal encoder and a 16-layer bidirectional Transformer aligned to T5 embeddings by a SigLIP-style contrastive loss, reaches 0.3621 \pm 0.0031 top-10 balanced accuracy, compared with 0.3261 \pm 0.0019 on the smaller Armeni et al. dataset and random chance of 0.04 (Özdogan et al., 2 Jun 2025).

The central systems point is scaling. Performance improves approximately logarithmically with training set size, and baseline runs are reproducible in <12 hours per run on NVIDIA H100 or V100 hardware with ≤64 GiB memory. In MEEtBrain terms, LibriBrain provides a deep within-subject substrate for speech-decoding pipelines, reproducible splits, and standardized loaders that make sensor-space MEG a practical target for large-scale machine learning (Özdogan et al., 2 Jun 2025).

5. Portable affective computing with AI-generated music

The most explicit applied use of the term appears in “Wearable Music2Emotion,” which proposes MEEtBrain as a portable and multimodal framework for emotion analysis using synchronized EEG–fNIRS acquisition and AI-generated music stimuli (Zhao et al., 5 Aug 2025). The paper identifies three limitations in prior work—stimulus constraints, modality specificity, and portability—and addresses them by combining large-scale MUSICGEN-generated clips with a wireless headband-style acquisition device.

Stimuli are generated from Russell’s Valence–Arousal model and targeted to the four quadrants HAHV, HALV, LAHV, and LALV. An initial 236 clips are produced, 157 remain after technical screening, and 101 survive final geometric selection in the VA plane, specifically 31 HAHV, 23 HALV, 21 LAHV, and 26 LALV. The acquisition hardware uses 2 EEG channels at Fp1 and Fp2 with reference A1, sampled at 250 Hz, together with 8 fNIRS optodes over bilateral prefrontal cortex at 735 nm and 850 nm, sampled at 25 Hz. EEG preprocessing applies artifact screening, 0.1–40 Hz bandpass filtering, epoch extraction over the 25–55 s interval of each trial, and baseline correction. fNIRS preprocessing converts intensity to optical density,

$a$8

uses the modified Beer–Lambert law to estimate HbO and HbR, defines HbT as HbO + HbR, applies a 0.5–4 Hz bandpass to derive PPG, and a 0.01–0.1 Hz bandpass to suppress cardiac, respiratory, and Mayer-wave components (Zhao et al., 5 Aug 2025).

The dataset initially covered 20 participants and later expanded to 44 participants, with ages 22–38 and mean age 25.75. The total synchronized recording duration is given as 1760 minutes, computed as 44 participants × 8 blocks × 5 trials × 60 s. Feature extraction uses EEG relative spectral powers in the canonical $a$9 bands over 3 s sub-epochs in the final 30 s of music, and fNIRS mean/variance features for HbO, HbR, and HbT across 8 channels, yielding a 48-dimensional hemodynamic feature vector. A deep neural network extending the Conformer architecture with an additional fNIRS branch performs learned multimodal fusion (Zhao et al., 5 Aug 2025).

Quantitatively, the best cross-subject performance on the first 20 participants is obtained with EEG+PPG+Hb fusion: for valence, ACC 0.663 ± 0.064 and MF1 0.636 ± 0.078; for arousal, ACC 0.719 ± 0.077 and MF1 0.657 ± 0.065. Intra-subject 10-fold cross-validation again favors EEG+PPG+Hb, reaching ACC 0.679 ± 0.066 and MF1 0.673 ± 0.069 for valence, and ACC 0.710 ± 0.065 and MF1 0.657 ± 0.076 for arousal. The paper also reports significant emotion-related structure in the signals: one-way ANOVA on music features finds that rhythmic articulation and pitch range strongly affect arousal with $b$0, mode and tempo differ with $b$1, and melodic direction shows $b$2; valence and liking correlate at $b$3, $b$4; arousal and valence correlate only weakly at $b$5, $b$6; and EEG $b$7-band relative power differs across emotion groups with $b$8, with LAHV differing significantly from the other three groups in Tukey’s HSD (Zhao et al., 5 Aug 2025).

This version of MEEtBrain is therefore a concrete wearable BCI framework rather than an abstract paradigm. Its novelty lies in combining AI-generated affective stimuli, synchronized EEG–fNIRS, and portable dry-electrode hardware to reduce stimulus-selection bias, exploit multimodal complementarity, and improve real-world deployability (Zhao et al., 5 Aug 2025).

6. Embodied communication and edge-AI control

Another strand treats MEEtBrain as an embodied control paradigm. BRIEDGE is described as “MEEtBrain-like” in its operational goal: multi-EEG acquisition at mobile or portable edge servers, robust intention decoding under heterogeneous devices, semantic encoding-decoding over physical channels, and concurrent multi-agent control (Ouyang et al., 2024). Its pipeline comprises EEG acquisition from devices such as Emotiv EPOC and Brainlink, dynamic path selection for heterogeneous signal dimensions, an EEG-adaptive neural network with intermittent mask augmentation and Informer-inspired ProbSparse attention, and a transmitter–receiver semantic communication stack. The communication model is

$b$9

with $\xi(t)$0. Training uses a composite objective

$\xi(t)$1

and the paper reports that the three-phase training improves accuracy by 24% relative to using only semantic loss.

On benchmark datasets, BRIEDGE reports classification accuracies of 0.9878 on Brainlink, 0.9251 on EyeState, 0.6602 on BCI-2000, and 0.8650 on the Hybrid dataset, outperforming listed baselines in each case. Model compression via pruning, weight sharing, and half-quantization reduces size from 3.309 MB to 1.458 MB with only 0.25% accuracy drop on the Hybrid dataset. The system also introduces a two-slot code map consisting of Agent ID and Action ID, enabling concurrent control of multiple devices while isolating command namespaces (Ouyang et al., 2024).

A more literal brain-to-physical-layer communication variant appears in the electromagnetic brain-computer-metasurface platform of (Ma et al., 2022). That paper does not explicitly use the term MEEtBrain, but the linkage has been proposed as a conceptual correspondence because the platform translates P300-based EEG into digital electromagnetic coding. EEG is acquired with a 30-channel cap at 250 Hz, band-pass filtered at 0.5–20 Hz, epoched over 0–600 ms post-flash with −200–0 ms baseline correction, downsampled by 6 to 25 time points per channel, and decoded by Bayesian linear regression after 30 calibration trials with 10 rounds per trial. Online transmission uses an adaptive stopping rule: once the gap between the highest and second-highest confidence scores exceeds 0.2, the selected character is serialized as ASCII with a frame header “11111111110000” and sent by toggling a 2-bit metasurface between high-amplitude and low-amplitude scattering patterns. Reported transmitted phrases include “HELLO WORLD”, “HI, SEU”, “HI, SCUT”, and “BCI METASURFACE”, with an average input speed of ~5 s per character, or about 12 characters/min. The same platform also demonstrates beam deflections of 15°, 30°, and 45°, OAM modes +1 and +2, and four RCS levels at −15 dB, −12 dB, −9 dB, and −6 dB (Ma et al., 2022).

Taken together, these systems extend MEEtBrain beyond representation learning into communication and actuation. The emphasis shifts from latent-state interpretation to low-latency intent extraction, semantic robustness under noisy channels, and the direct routing of decoded neural content into robots or electromagnetic hardware (Ouyang et al., 2024, Ma et al., 2022).

7. Multi-view ensemble interactions and cross-session dynamics

A final usage concerns systems neuroscience rather than downstream control. CREIMBO is presented as a computational substrate for MEEtBrain in the sense of Multi-view Ensemble interactions in the Brain (Mudrik et al., 2024). The problem setting is asynchronous multi-session neural recording in which sessions differ in neuron identities, recorded areas, and observation length. Each session $\xi(t)$2 provides a partial view $\xi(t)$3 of a common brain system.

CREIMBO assumes that session-specific observations are generated from area-structured neural ensembles together with a shared basis of global dynamical sub-circuits. The observation model is

$\xi(t)$4

where $\xi(t)$5 is block-diagonal across areas and $\xi(t)$6 is ensemble activity. Non-stationary dynamics are modeled through

$\xi(t)$7

with sparse, time-varying coefficients $\xi(t)$8 modulating shared sub-circuits $\xi(t)$9. Ensemble discovery uses graph-driven reweighted $\Delta\omega = 0$0 regularization over neuron-similarity graphs within each area. The authors describe this as graph-driven dictionary learning rather than a stationary factor model, and emphasize that multiple sub-circuits can be co-active, allowing overlapping temporal processes (Mudrik et al., 2024).

Synthetic experiments include settings with $\Delta\omega = 0$1 rotational sub-circuits and up to $\Delta\omega = 0$2 sessions, and the paper reports high correlations with ground truth for observations, ensembles, latent states, sub-circuits, and coefficients across repeated initializations. On human electrophysiology data, CREIMBO is demonstrated on single-unit spikes from 10 brain areas in medial temporal and medial frontal lobes across 21 subjects and 41 sessions, with spikes converted to firing rates by 30 ms Gaussian smoothing. The model uses up to $\Delta\omega = 0$3 sub-circuits and up to $\Delta\omega = 0$4 ensembles per region. Inferred ensemble matrices $\Delta\omega = 0$5 show one dense “mean-field” ensemble per area together with multiple sparse ensembles. Some global sub-circuits are predominantly inter-area, others include within-area motifs such as the reported $\Delta\omega = 0$6 and $\Delta\omega = 0$7, and still others are pruned to near-zero, including the reported $\Delta\omega = 0$8 and $\Delta\omega = 0$9. Within-session pairwise correlations of coefficient trajectories are typically low, < 0.1, supporting the claim that distinct latent processes are being separated (Mudrik et al., 2024).

Within the broader MEEtBrain landscape, CREIMBO provides the strongest formalism for session-invariant interaction primitives combined with session-specific ensemble projections and coefficient trajectories. It is a model of cross-regional organization rather than an interface, but it matches the same recurring themes: non-stationarity, sparse composition, multiscale coordination, and the rejection of a single fixed representational basis (Mudrik et al., 2024).

Across these formulations, MEEtBrain denotes a convergent research direction rather than a unique method. Its theoretical core is metastability and the coexistence of integration with segregation; its applied forms connect neural measurements to multimodal latent spaces, portable emotion decoding, speech-decoding infrastructure, robotic command channels, electromagnetic communication, and multi-view sub-circuit discovery. A plausible implication is that the term now marks a common methodological stance: brain signals are treated as dynamically structured, context-sensitive, and most informative when embedded in systems that preserve temporal variability, multimodal coupling, or cross-scale coordination (Kelso, 2023, Xu et al., 18 Sep 2025, Zhao et al., 5 Aug 2025, Ouyang et al., 2024, Mudrik et al., 2024).