---
title: 'Neuro-JEPA: Latent Predictive Learning for Brain Data'
url: https://www.emergentmind.com/topics/neuro-jepa
type: topic
---

# Neuro-JEPA: Latent Predictive Learning for Brain Data

Neuro-JEPA is an emerging family of Joint-Embedding Predictive Architectures adapted to neural recordings, neuroimaging, brain–behavior trajectories, and structured neurological data. Rather than reconstructing masked measurements directly, these systems generally predict latent representations of unobserved neural or patient-state information from partially observed context, often using a stop-gradient target encoder updated by exponential moving average (EMA). The term does not designate a single canonical architecture: published implementations include EEG-specific Signal-JEPA, fMRI-based Brain-JEPA, sparse multimodal MRI Neuro-JEPA, EEG-JEPA and STST-JEPA, graph-based clinical patient-state models, and action-conditioned or generative JEPA systems whose relevance is prospective rather than directly neuroscientific.

## 1. Conceptual foundations and terminology

JEPA-based learning separates an online or context branch from a target branch. The context branch receives visible portions of an input, while the target branch observes the complete input or target state. A predictor maps the context representation and positional information to the target representation. The training signal is therefore defined in latent space:

$$
\text{visible context}
\rightarrow
\text{context encoder}
\rightarrow
\text{predictor}
\rightarrow
\text{target representation}.
$$

The target is not normally a reconstruction of the masked waveform, voxel, image, or clinical record. This distinction is important for neural data because raw measurements contain low-signal, nuisance, and acquisition-dependent components. Masked waveform reconstruction can reward interpolation of predictable background activity, artifacts, subject identity, or device characteristics without necessarily producing representations that transfer across tasks. Latent prediction instead allows the target representation to determine which aspects of the signal are retained.

A common target-encoder update is:

$$
\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,
$$

where $\theta$ denotes online parameters, $\bar{\theta}$ target parameters, and $\mu$ the EMA coefficient. Target representations are typically stop-gradient quantities. This teacher-like mechanism supplies a slowly evolving prediction target and prevents direct co-adaptation of the context and target branches.

“Neuro-JEPA” is used here as an *Editor's term* for this broader design space rather than as the name of one universally defined model. Brain-JEPA applies the principle to resting-state fMRI [2409.19407], while Neuro-JEPA applies sparse latent prediction to multimodal structural MRI [2606.14957]. EEG implementations include Signal-JEPA [2403.11772], STST-JEPA [2607.06629], and EEG-JEPA [2608.00114]. Other studies extend JEPA toward clinical graphs [2608.22583], causal action-conditioned dynamics [2609.31161], policy learning [2609.09630], symbolic reasoning [2603.13265], and 3D medical-image generation [2608.28787]. These systems share latent predictive principles but differ substantially in target semantics, masking, modality handling, and downstream objectives.

The central conceptual issue is that predictive accuracy does not by itself establish causal-state recovery, anatomical validity, or biological plausibility. A representation may preserve information while mixing latent factors, exploit dataset-specific shortcuts, or encode acquisition conditions. Action-conditioned JEPA theory addresses this problem by combining transition likelihood with entropy maximization and requiring sufficient action-induced variation; under strong assumptions, the learned representation recovers latent causal components up to component-wise invertible transformations and permutation [2609.31161].

## 2. Architectural principles

### Latent target prediction

The defining operation is prediction of a target embedding rather than direct measurement reconstruction. Brain-JEPA represents an fMRI scan as an ROI-by-time matrix with $450$ ROIs and a default length of $160$ time points. Temporal patches of length $16$ produce ten patches per ROI. An observation encoder processes a sampled context block, an EMA target encoder processes target patches, and a narrower predictor estimates the target embeddings [2409.19407].

For EEG-JEPA, the target is a normalized contextual representation at multiple encoder depths:

$$
y_t^{(\ell)}
=
\operatorname{sg}
\left(
\operatorname{LN}
\left(
f_{\bar{\theta}}^{(\ell)}(x)_t
\right)
\right).
$$

The predictor receives visible context representations and a learned target-query token combined with a two-dimensional electrode–time positional embedding. This makes target location explicit without exposing the target signal content [2608.00114].

STST-JEPA adopts a different target semantics. Its EMA branch tracks only the shallow tokenizer rather than the deep context encoder. The model therefore predicts tokenizer-level representations with a deep context transformer and adds a weak waveform reconstruction term. This “shallow-target” arrangement is a deliberate deviation from canonical deep-target JEPA formulations [2607.06629].

### Structured positional information

Neural measurements are not arbitrary token sequences. Their spatial organization may be anatomical, functional, sensor-based, or graph-structured. Brain-JEPA introduces Brain Gradient Positioning, which derives a functional coordinate system from connectivity gradients rather than relying solely on atlas order or anatomical proximity. A functional-affinity matrix is converted into diffusion-map gradients, with thirty gradient components used in the main configuration. The resulting gradient coordinates are combined with temporal sinusoidal encodings [2409.19407].

Structural MRI implementations instead use world-space coordinates. COJEPA maps patch centers through the MRI affine into standardized physical coordinates and applies three-dimensional sinusoidal embeddings. Removing global positional encoding caused the largest retrieval degradation in its ablation, indicating that absolute anatomical location is central to subject-discriminative learning [2607.11962].

EEG systems use sensor-specific mechanisms. Signal-JEPA initializes trainable channel embeddings from three-dimensional electrode coordinates and uses spatial aggregation to learn sensor combinations [2403.11772]. STST-JEPA uses a common 128-channel budget, validity masks, coordinate embeddings, and Pooled Multihead Attention to pool channel representations independently at each temporal index [2607.06629]. EEG-JEPA uses a canonical 19-channel montage and defines topology-aware masking over electrode–time regions [2608.00114].

### Capacity and modality specialization

Neuro-JEPA uses a range of model classes. Brain-JEPA evaluates ViT-S, ViT-B, and ViT-L models, with the main configuration using ViT-B and $300$ pretraining epochs [2409.19407]. Sparse Neuro-JEPA is a 3D ViT-Base with twelve Transformer layers and Mixture-of-Experts feed-forward sublayers. It has $128$ million total parameters and approximately $86$ million activated parameters [2606.14957].

The sparse MRI model uses sixteen experts, two shared experts, six activated experts per token, and MoE layers in alternating Transformer layers. An auxiliary-loss-free bias-update procedure balances routing without adding a conventional load-balancing term to the JEPA objective. Routing differs by foreground/background status and anatomical region, while most experts are not strongly modality-exclusive [2606.14957].

COJEPA uses a hierarchical convolutional patch embedding before a ViT-Base encoder. Its contrastive objective operates on a global token, while the JEPA objective operates on spatial target tokens. This combines local anatomical predictivity with global subject discriminability [2607.11962].

## 3. Masking and target-support design

Masking determines the conditional inference problem presented during pretraining. In Neuro-JEPA systems, it is not merely a regularization device; it encodes assumptions about the generalization that the representation should support.

### Spatial and spatiotemporal EEG masking

Signal-JEPA introduces spatial block masking based on Euclidean distances between three-dimensional electrode coordinates. A sampled mask center defines a scalp neighborhood, and all temporal tokens belonging to masked channels are hidden. Mask diameters of approximately $40\%$, $60\%$, and $80\%$ of head size are evaluated [2403.11772].

The method is intended to force inference of missing scalp regions from remaining channels and temporal context. Its downstream experiments show no clear monotonic trend with mask diameter. Spatial filtering, rather than mask size alone, is the more consistent determinant of performance: the strongest downstream arrangement applies learned spatial aggregation before local temporal feature extraction and discards the contextual Transformer.

STST-JEPA samples four rectangular blocks on the channel–time grid. Channel fractions are drawn from $U[0.2,0.5]$ and temporal fractions from $U[0.1,0.3]$, producing an empirical average masked fraction of approximately $24\%$. The blocks may represent local sensor dropout, channel groups, temporal intervals, or multiple spatiotemporal regions [2607.06629].

EEG-JEPA’s Neurotopology-Aware Multi-scale Electrode-Temporal Masking, or N-MET, samples six masking primitives:

| Primitive | Mixture mass | Intended structure |
|---|---:|---|
| Temporal stripe | 35.0% | Missing intervals and temporal continuity |
| Channel stripe | 17.5% | Regional sensor loss |
| Local block | 17.5% | Focal electrode–time activity |
| Topo-pair stripe | 10.0% | Bilateral correspondence |
| Full channel | 15.0% | Failed or absent electrode |
| Full topo-pair | 5.0% | Bilateral dropout |

The selected targets cover approximately $40\%$–$55\%$ of valid tokens. The supplied description does not specify explicit distance formulas or adjacency matrices for the topology sampler; exact reproduction therefore requires the implementation or supplementary material [2608.00114].

### Functional and volumetric masking

Brain-JEPA partitions the fMRI patch grid into an observation region and three target regions: cross-ROI, cross-time, and double-cross. The double-cross region requires simultaneous generalization across unseen ROIs and unseen time positions. The procedure uses overlapped sampling and one target from each region [2409.19407].

For structural MRI, Neuro-JEPA uses multiscale volumetric block masking at a fixed ratio of $75\%$. Spatial scale is sampled from $[0.0,0.2]$, $[0.2,0.5]$, and $[0.5,0.7]$, while depth scale is sampled from $[0.0,1.0]$. Boundary erosion and dilation adjust masks to the desired token count. This avoids both overly local interpolation and excessive depth masking that could remove fine anatomical details or small lesions [2606.14957].

COJEPA uses one context block and three target blocks. The context scale is $[0.80,1.00]$, target scale is $[0.20,0.25]$, and target aspect ratio is $[0.75,1.25]$. Target blocks are foreground-aware: background tokens are excluded from the JEPA loss and target-block centers must be foreground patches [2607.11962].

### Graph and generative masks

Clinical Graph-JEPA masks connected graph patches obtained through balanced multi-source breadth-first search. The target-node mask ratio is $0.40$. The target encoder receives the full graph while the online encoder receives visible context patches and learned mask content. The model predicts hidden node or patch representations rather than missing edges directly [2608.22583].

Med-D-JEPA masks continuous latent tokens produced by a 3D KL-regularized adversarial VAE. The context encoder receives visible latent tokens, while the target encoder receives the complete latent sequence. Unlike a discriminative Neuro-JEPA, the target embeddings additionally condition per-token diffusion, enabling iterative generation of missing latent tokens [2608.28787].

## 4. Training objectives and representation geometry

Neuro-JEPA objectives differ in their distance functions, target depths, and auxiliary constraints.

### Latent prediction

Signal-JEPA compares predicted and target representations with an $L_1$ loss over masked positions. Brain-JEPA uses a squared Euclidean loss over three target regions. EEG-JEPA uses Smooth L1 prediction across four supervised depths, together with variance–covariance regularization. Neuro-JEPA for structural MRI uses a foreground-weighted $L_1$ loss. COJEPA uses mean squared error for JEPA target prediction and a symmetric InfoNCE loss for global contrastive learning [2403.11772; 2409.19407; 2608.00114; 2606.14957; 2607.11962].

EEG-JEPA’s hierarchical objective supervises layers 3, 6, 9, and 12. Each depth-specific state is fused before prediction, encouraging representations at multiple levels of abstraction. In its ablation, multi-level prediction produced the largest single Stage-1 improvement, increasing macro balanced accuracy from $46.47\%$ to $50.13\%$ [2608.00114].

### Contrastive and diversity constraints

COJEPA combines local prediction and global discriminability:

$$
\mathcal{L}_{\mathrm{total}}
=
1.0\,\mathcal{L}_{\mathrm{pred}}
+
0.1\,\mathcal{L}_{\mathrm{con}}.
$$

The JEPA term promotes local anatomical structure, while the contrastive term uses two augmented crops from the same subject as a positive pair and other subjects in the minibatch as negatives. COJEPA achieves the strongest monozygotic twin retrieval at rank one, with recall $0.84$, while pure JEPA reaches $0.26$ [2607.11962].

EEG-JEPA adds variance–covariance regularization to reduce feature collapse. In contrast, the action-conditioned A-JEPA study argues that entropy maximization alone does not resolve component mixing. Its information-theoretic objective combines conditional likelihood maximization with entropy maximization. Under strong assumptions concerning invertibility, transition factorization, dimensionality, and action-induced variation, the representation recovers causal components up to permutation and component-wise invertible transformations [2609.31161].

### Value and action-conditioned objectives

Value-guided JEPA world models shape latent geometry so that distance approximates negative goal-conditioned value:

$$
\|\mathcal{E}_\theta(s)-\mathcal{E}_\theta(g)\|_2
\approx
-V^\star(s,g).
$$

The experiments show that a learned quasi-distance outperforms symmetric Euclidean distance on wall and maze tasks. This work is not a neuroscience model, but it suggests that predictive representations may require explicit behavioral or goal-related geometry to support intelligent action [2601.00844].

JEPA Policy adds future-representation prediction to action-chunk prediction. Action and future tokens interact within a shared Transformer, and the future loss shapes the same action-generating representation. Across nine simulated tasks, the method reaches a mean success of $83.0\%$, compared with $77.4\%$ for MIP and $75.1\%$ for the evaluated Diffusion Policy [2609.09630]. Its relevance to Neuro-JEPA is prospective: sensorimotor neural representations could be trained to predict action-conditioned future neural, sensory, or proprioceptive states.

### Logical and graph constraints

Rule-informed JEPA, or RiJEPA, interprets prediction error as a logical energy. Valid antecedent–consequent pairs are assigned low energy, while corrupted pairs are required to exceed a margin. This produces low-energy basins for logically valid relationships and high-energy regions for invalid configurations [2603.13265].

Clinical Graph-JEPA instead predicts latent graph states using cosine prediction error and evaluates the frozen encoder through type-matched edge recovery. In a MIMIC-IV study, entity-grounded discharge-note injection increases overall leave-one-out MRR from $0.364$ to $0.477$, a relative improvement of approximately $31\%$. This evaluates agreement with held-out graph relations under the construction pipeline, not verified clinical truth or causal validity [2608.22583].

## 5. Empirical evidence across neural and clinical domains

### EEG

Signal-JEPA uses a dataset of 54 subjects spanning motor imagery, ERP, and SSVEP paradigms. The best reported pipelines reach $97\%$ ERP AUC, $94\%$ SSVEP accuracy, and $65\%$ motor-imagery accuracy. The strongest downstream architecture is pre-local: learned spatial aggregation precedes local temporal extraction and the contextual encoder is discarded. The authors interpret this as evidence that early spatial filtering is important, but the study primarily tests held-out-subject transfer within one dataset rather than strict cross-dataset transfer [2403.11772].

STST-JEPA is pretrained on 47,703 sessions from brain.space and HBN, spanning ages 5–81. A frozen attentive probe reaches a best validation MAE of $3.06$ years with Pearson correlation $r=0.924$ on 3,367 sessions. On NeuralBench × brain.space, partial fine-tuning reaches balanced accuracy $0.911$ for sex classification, age correlation $0.749$, and psychopathology correlation $0.215$ [2607.06629].

EEG-JEPA improves the 14-task frozen macro balanced accuracy from $40.49\%$ for a CBraMod-style masked-waveform objective to $50.42\%$ under matched backbone, corpus, and duration. Multi-source continuation increases the result to $52.94\%$. Under protocol-matched full fine-tuning, the nine-task average increases from $68.98\%$ to $70.65\%$ [2608.00114].

### fMRI and structural MRI

Brain-JEPA is pretrained on 40,162 UK Biobank participants and evaluated on demographic, cognitive, disease, and cross-ethnic tasks. On HCP-Aging, it reports age correlation $0.844$, sex accuracy $81.52\%$, Neuroticism correlation $0.307$, and Flanker correlation $0.406$. On ADNI, amyloid-positive versus amyloid-negative classification reaches $71.00\%$ accuracy. On the predominantly Asian MACC cohort, NC/MCI classification reaches $65.98\%$ accuracy despite pretraining on the predominantly Caucasian UK Biobank cohort [2409.19407].

Sparse Neuro-JEPA is pretrained on $1,551,862$ scans from $428,647$ studies and evaluated across 25 tasks from three health systems and 22 tasks from 12 public datasets. Across 41 public dataset–task–modality combinations, it reports mean AUROC $0.785$ and AUPRC $0.649$, exceeding the evaluated CNN and other neuroimaging foundation models under the stated protocols. Its multimodal evaluation reaches mean AUROC $0.805$ and AUPRC $0.637$ on twelve public multimodal combinations [2606.14957].

COJEPA combines JEPA and contrastive learning on T1-weighted MRI from 2,286 participants. It reaches monozygotic twin retrieval recall at rank one of $0.84$, fine-tuned OpenBHB age MAE of $2.55$ years, and whole-tumor BraTS Dice comparable to contrastive learning. Removing world-space positional encoding sharply degrades retrieval, while removing the convolutional stem produces a moderate decline [2607.11962].

### Clinical graphs and neuro-symbolic extensions

Clinical Graph-JEPA demonstrates that a graph can be treated as an uncertain, evidence-scored patient-state representation rather than a fixed extraction artifact. It learns to recover held-out relations from visible graph context. A neurological extension could represent symptoms, lesions, EEG events, imaging findings, medications, cognitive scores, and temporal events as typed nodes and relations. Such a system would need explicit event-time representations because admission-level graphs do not establish exact order or causal direction [2608.22583].

RiJEPA demonstrates a complementary route in which symbolic rules reshape latent geometry. In a synthetic experiment, classic JEPA yields mean valid and OOD-invalid energies of $0.003$ and $0.150$, whereas RiJEPA yields $0.006$ and $12.713$. In the reported clinical use case, RiJEPA achieves $100\%$ zero-shot logical accuracy under the stated setup. These results support rule-constrained representation learning in controlled settings but do not establish general causal reasoning or broad clinical robustness [2603.13265].

## 6. Limitations, controversies, and research directions

Neuro-JEPA remains methodologically heterogeneous and empirically uneven. Several recurrent limitations constrain interpretation.

**Cross-domain transfer is often narrower than the terminology suggests.** Signal-JEPA is motivated by cross-dataset EEG transfer but evaluates primarily held-out subjects within the Lee2019 dataset. Brain-JEPA uses external cohorts, but its functional gradients are derived from UK Biobank and may encode population-specific organization. Sparse Neuro-JEPA evaluates multiple institutions and public datasets, yet its pretraining corpus originates from one health system. STST-JEPA uses a shared 128-channel budget but does not fully harmonize heterogeneous montages.

**Predictive success does not establish causal or biological validity.** A-JEPA shows theoretically that accurate prediction can coexist with component mixing. High nonlinear decoding of state information can therefore coexist with low component-wise alignment. Attention concentration, expert routing, and latent clustering are useful analyses but do not prove causal neural importance. Clinical Graph-JEPA evaluates recovery of relations in a constructed graph, not independent clinical truth.

**Target semantics remain unresolved.** Brain-JEPA uses deep ViT target representations, STST-JEPA uses a shallow EMA tokenizer, EEG-JEPA uses multi-depth contextual targets, and Med-D-JEPA uses target embeddings to condition generation. There is no established basis for deciding whether the optimal target should encode local physiology, deep context, anatomical structure, cross-subject invariance, future state, or a hierarchy of these quantities.

**Masking can encode unwanted shortcuts.** Spatial or temporal redundancy may make target prediction possible through local interpolation. Conversely, excessive masking may remove fine anatomical or physiological details. N-MET, multiscale volumetric masking, Brain-JEPA’s cross-ROI/cross-time partition, and graph patch masking are domain-specific hypotheses rather than universally optimal procedures.

**Clinical and demographic confounding remains substantial.** Performance can reflect scanner, site, field strength, demographic composition, documentation patterns, or disease prevalence. Brain-age residuals may contain task-state information, and strong demographic prediction does not establish disease-mechanistic validity. Clinical Graph-JEPA’s use of discharge notes may introduce information that is temporally downstream of a relation intended for prospective decision support.

**Biological plausibility is limited.** Existing systems rely on backpropagation, EMA target networks, global attention, offline batches, explicit masking, and task-specific optimization. Their relation to predictive coding, cortical dynamics, hippocampal representations, basal-ganglia value computation, or local synaptic learning is conceptual rather than demonstrated.

Future Neuro-JEPA research directions include multimodal neural recordings, source-space and anatomy-informed coordinates, subject- and session-invariant objectives, hierarchical temporal prediction, cross-modal target design, graph and geodesic masking, missing-sensor robustness, longitudinal modeling, and explicit intervention-based evaluation. Action-conditioned models suggest using motor commands, sensory perturbations, stimulation, task context, or pharmacological conditions as variables that modulate latent transitions [2609.31161]. Value-guided models suggest shaping latent geometry around reachability or behavioral utility [2601.00844]. Policy models suggest coupling future-state prediction directly to action representations rather than treating it as a detached auxiliary loss [2609.09630].

Generative extensions such as Med-D-JEPA add per-token diffusion to JEPA latent prediction and use iterative next-set-of-token sampling for 3D brain-MRI synthesis. They report improvements in generation metrics and downstream classification or segmentation after synthetic pretraining, but also introduce risks of anatomical hallucination, pathology artifacts, pseudo-label contamination, and privacy leakage [2608.28787]. Rule-informed and graph-based extensions further suggest that Neuro-JEPA could incorporate logical constraints, provenance, uncertainty, and structured patient-state relations [2603.13265; 2608.22583].

The most defensible characterization of Neuro-JEPA is therefore not a single foundation model but a research program organized around three questions: what latent neural content should be predicted, where should prediction occur in neural space and time, and at what representational depth should supervision be applied? Its strongest demonstrated principles are latent rather than raw-signal prediction, EMA-stabilized targets, domain-specific masking, meaningful spatial or functional coordinates, and evaluation across frozen transfer and task adaptation. Its central unresolved problem is whether these predictive representations encode mechanisms that remain valid under changes in subjects, tasks, modalities, interventions, anatomy, and acquisition systems.

Source: https://www.emergentmind.com/topics/neuro-jepa