Neuro-JEPA: Latent Predictive Learning for Brain Data

• Neuro-JEPA is an emerging family of JEPA models that predict latent representations of hidden neural or patient states from partial EEG, fMRI, MRI, or graph data rather than reconstructing raw measurements.
• These systems combine EMA-updated target encoders with domain-specific masking and spatial coordinates, enabling applications such as cross-subject EEG classification, brain-age prediction, disease screening, anatomical retrieval, and clinical relation recovery.
• Reported results are promising but do not prove causal or biological validity, because representations may retain scanner, demographic, subject, or documentation shortcuts and require stronger cross-site, intervention-based, and longitudinal evaluation.

Neuro-JEPA is an emerging family of Joint-Embedding Predictive Architectures adapted to neural recordings, neuroimaging, brain–behavior trajectories, and structured neurological data. Rather than reconstructing masked measurements directly, these systems generally predict latent representations of unobserved neural or patient-state information from partially observed context, often using a stop-gradient target encoder updated by exponential moving average (EMA). The term does not designate a single canonical architecture: published implementations include EEG-specific Signal-JEPA, fMRI-based Brain-JEPA, sparse multimodal MRI Neuro-JEPA, EEG-JEPA and STST-JEPA, graph-based clinical patient-state models, and action-conditioned or generative JEPA systems whose relevance is prospective rather than directly neuroscientific.

1. Conceptual foundations and terminology

JEPA-based learning separates an online or context branch from a target branch. The context branch receives visible portions of an input, while the target branch observes the complete input or target state. A predictor maps the context representation and positional information to the target representation. The training signal is therefore defined in latent space:

$$\text{visible context} \rightarrow \text{context encoder} \rightarrow \text{predictor} \rightarrow \text{target representation}.$$

The target is not normally a reconstruction of the masked waveform, voxel, image, or clinical record. This distinction is important for neural data because raw measurements contain low-signal, nuisance, and acquisition-dependent components. Masked waveform reconstruction can reward interpolation of predictable background activity, artifacts, subject identity, or device characteristics without necessarily producing representations that transfer across tasks. Latent prediction instead allows the target representation to determine which aspects of the signal are retained.

A common target-encoder update is:

$$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$

where $\theta$ denotes online parameters, $\bar{\theta}$ target parameters, and $\mu$ the EMA coefficient. Target representations are typically stop-gradient quantities. This teacher-like mechanism supplies a slowly evolving prediction target and prevents direct co-adaptation of the context and target branches.

“Neuro-JEPA” is used here as an Editor's term for this broader design space rather than as the name of one universally defined model. Brain-JEPA applies the principle to resting-state fMRI (Dong et al., 2024), while Neuro-JEPA applies sparse latent prediction to multimodal structural MRI (Huang et al., 12 Jun 2026). EEG implementations include Signal-JEPA (Guetschel et al., 2024), STST-JEPA (Segal et al., 7 Jul 2026), and EEG-JEPA (Li et al., 31 Jul 2026). Other studies extend JEPA toward clinical graphs (Yadav et al., 23 Aug 2026), causal action-conditioned dynamics (Liu et al., 25 Sep 2026), policy learning (Xu et al., 9 Sep 2026), symbolic reasoning (Huang et al., 25 Feb 2026), and 3D medical-image generation (Zhou et al., 28 Aug 2026). These systems share latent predictive principles but differ substantially in target semantics, masking, modality handling, and downstream objectives.

The central conceptual issue is that predictive accuracy does not by itself establish causal-state recovery, anatomical validity, or biological plausibility. A representation may preserve information while mixing latent factors, exploit dataset-specific shortcuts, or encode acquisition conditions. Action-conditioned JEPA theory addresses this problem by combining transition likelihood with entropy maximization and requiring sufficient action-induced variation; under strong assumptions, the learned representation recovers latent causal components up to component-wise invertible transformations and permutation (Liu et al., 25 Sep 2026).

2. Architectural principles

Latent target prediction

The defining operation is prediction of a target embedding rather than direct measurement reconstruction. Brain-JEPA represents an fMRI scan as an ROI-by-time matrix with $450$ ROIs and a default length of $160$ time points. Temporal patches of length $16$ produce ten patches per ROI. An observation encoder processes a sampled context block, an EMA target encoder processes target patches, and a narrower predictor estimates the target embeddings (Dong et al., 2024).

For EEG-JEPA, the target is a normalized contextual representation at multiple encoder depths:

$$y_t^{(\ell)} = \operatorname{sg} \left( \operatorname{LN} \left( f_{\bar{\theta}}^{(\ell)}(x)_t \right) \right).$$

The predictor receives visible context representations and a learned target-query token combined with a two-dimensional electrode–time positional embedding. This makes target location explicit without exposing the target signal content (Li et al., 31 Jul 2026).

STST-JEPA adopts a different target semantics. Its EMA branch tracks only the shallow tokenizer rather than the deep context encoder. The model therefore predicts tokenizer-level representations with a deep context transformer and adds a weak waveform reconstruction term. This “shallow-target” arrangement is a deliberate deviation from canonical deep-target JEPA formulations (Segal et al., 7 Jul 2026).

Structured positional information

Neural measurements are not arbitrary token sequences. Their spatial organization may be anatomical, functional, sensor-based, or graph-structured. Brain-JEPA introduces Brain Gradient Positioning, which derives a functional coordinate system from connectivity gradients rather than relying solely on atlas order or anatomical proximity. A functional-affinity matrix is converted into diffusion-map gradients, with thirty gradient components used in the main configuration. The resulting gradient coordinates are combined with temporal sinusoidal encodings (Dong et al., 2024).

Structural MRI implementations instead use world-space coordinates. COJEPA maps patch centers through the MRI affine into standardized physical coordinates and applies three-dimensional sinusoidal embeddings. Removing global positional encoding caused the largest retrieval degradation in its ablation, indicating that absolute anatomical location is central to subject-discriminative learning (Mager et al., 12 Jul 2026).

EEG systems use sensor-specific mechanisms. Signal-JEPA initializes trainable channel embeddings from three-dimensional electrode coordinates and uses spatial aggregation to learn sensor combinations (Guetschel et al., 2024). STST-JEPA uses a common 128-channel budget, validity masks, coordinate embeddings, and Pooled Multihead Attention to pool channel representations independently at each temporal index (Segal et al., 7 Jul 2026). EEG-JEPA uses a canonical 19-channel montage and defines topology-aware masking over electrode–time regions (Li et al., 31 Jul 2026).

Capacity and modality specialization

Neuro-JEPA uses a range of model classes. Brain-JEPA evaluates ViT-S, ViT-B, and ViT-L models, with the main configuration using ViT-B and $300$ pretraining epochs (Dong et al., 2024). Sparse Neuro-JEPA is a 3D ViT-Base with twelve Transformer layers and Mixture-of-Experts feed-forward sublayers. It has $$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$0 million total parameters and approximately $$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$1 million activated parameters (Huang et al., 12 Jun 2026).

The sparse MRI model uses sixteen experts, two shared experts, six activated experts per token, and MoE layers in alternating Transformer layers. An auxiliary-loss-free bias-update procedure balances routing without adding a conventional load-balancing term to the JEPA objective. Routing differs by foreground/background status and anatomical region, while most experts are not strongly modality-exclusive (Huang et al., 12 Jun 2026).

COJEPA uses a hierarchical convolutional patch embedding before a ViT-Base encoder. Its contrastive objective operates on a global token, while the JEPA objective operates on spatial target tokens. This combines local anatomical predictivity with global subject discriminability (Mager et al., 12 Jul 2026).

3. Masking and target-support design

Masking determines the conditional inference problem presented during pretraining. In Neuro-JEPA systems, it is not merely a regularization device; it encodes assumptions about the generalization that the representation should support.

Spatial and spatiotemporal EEG masking

Signal-JEPA introduces spatial block masking based on Euclidean distances between three-dimensional electrode coordinates. A sampled mask center defines a scalp neighborhood, and all temporal tokens belonging to masked channels are hidden. Mask diameters of approximately $$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$2, $$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$3, and $$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$4 of head size are evaluated (Guetschel et al., 2024).

The method is intended to force inference of missing scalp regions from remaining channels and temporal context. Its downstream experiments show no clear monotonic trend with mask diameter. Spatial filtering, rather than mask size alone, is the more consistent determinant of performance: the strongest downstream arrangement applies learned spatial aggregation before local temporal feature extraction and discards the contextual Transformer.

STST-JEPA samples four rectangular blocks on the channel–time grid. Channel fractions are drawn from $$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$5 and temporal fractions from $$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$6, producing an empirical average masked fraction of approximately $$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$7. The blocks may represent local sensor dropout, channel groups, temporal intervals, or multiple spatiotemporal regions (Segal et al., 7 Jul 2026).

EEG-JEPA’s Neurotopology-Aware Multi-scale Electrode-Temporal Masking, or N-MET, samples six masking primitives:

Primitive	Mixture mass	Intended structure
Temporal stripe	35.0%	Missing intervals and temporal continuity
Channel stripe	17.5%	Regional sensor loss
Local block	17.5%	Focal electrode–time activity
Topo-pair stripe	10.0%	Bilateral correspondence
Full channel	15.0%	Failed or absent electrode
Full topo-pair	5.0%	Bilateral dropout

The selected targets cover approximately $$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$8–$$\bar{\theta}\leftarrow \mu\bar{\theta}+(1-\mu)\theta,$$9 of valid tokens. The supplied description does not specify explicit distance formulas or adjacency matrices for the topology sampler; exact reproduction therefore requires the implementation or supplementary material (Li et al., 31 Jul 2026).

Functional and volumetric masking

Brain-JEPA partitions the fMRI patch grid into an observation region and three target regions: cross-ROI, cross-time, and double-cross. The double-cross region requires simultaneous generalization across unseen ROIs and unseen time positions. The procedure uses overlapped sampling and one target from each region (Dong et al., 2024).

For structural MRI, Neuro-JEPA uses multiscale volumetric block masking at a fixed ratio of $\theta$0. Spatial scale is sampled from $\theta$1, $\theta$2, and $\theta$3, while depth scale is sampled from $\theta$4. Boundary erosion and dilation adjust masks to the desired token count. This avoids both overly local interpolation and excessive depth masking that could remove fine anatomical details or small lesions (Huang et al., 12 Jun 2026).

COJEPA uses one context block and three target blocks. The context scale is $\theta$5, target scale is $\theta$6, and target aspect ratio is $\theta$7. Target blocks are foreground-aware: background tokens are excluded from the JEPA loss and target-block centers must be foreground patches (Mager et al., 12 Jul 2026).

Graph and generative masks

Clinical Graph-JEPA masks connected graph patches obtained through balanced multi-source breadth-first search. The target-node mask ratio is $\theta$8. The target encoder receives the full graph while the online encoder receives visible context patches and learned mask content. The model predicts hidden node or patch representations rather than missing edges directly (Yadav et al., 23 Aug 2026).

Med-D-JEPA masks continuous latent tokens produced by a 3D KL-regularized adversarial VAE. The context encoder receives visible latent tokens, while the target encoder receives the complete latent sequence. Unlike a discriminative Neuro-JEPA, the target embeddings additionally condition per-token diffusion, enabling iterative generation of missing latent tokens (Zhou et al., 28 Aug 2026).

4. Training objectives and representation geometry

Neuro-JEPA objectives differ in their distance functions, target depths, and auxiliary constraints.

Latent prediction

Signal-JEPA compares predicted and target representations with an $\theta$9 loss over masked positions. Brain-JEPA uses a squared Euclidean loss over three target regions. EEG-JEPA uses Smooth L1 prediction across four supervised depths, together with variance–covariance regularization. Neuro-JEPA for structural MRI uses a foreground-weighted $\bar{\theta}$0 loss. COJEPA uses mean squared error for JEPA target prediction and a symmetric InfoNCE loss for global contrastive learning (Guetschel et al., 2024, Dong et al., 2024, Li et al., 31 Jul 2026, Huang et al., 12 Jun 2026, Mager et al., 12 Jul 2026).

EEG-JEPA’s hierarchical objective supervises layers 3, 6, 9, and 12. Each depth-specific state is fused before prediction, encouraging representations at multiple levels of abstraction. In its ablation, multi-level prediction produced the largest single Stage-1 improvement, increasing macro balanced accuracy from $\bar{\theta}$1 to $\bar{\theta}$2 (Li et al., 31 Jul 2026).

Contrastive and diversity constraints

COJEPA combines local prediction and global discriminability:

$\bar{\theta}$3

The JEPA term promotes local anatomical structure, while the contrastive term uses two augmented crops from the same subject as a positive pair and other subjects in the minibatch as negatives. COJEPA achieves the strongest monozygotic twin retrieval at rank one, with recall $\bar{\theta}$4, while pure JEPA reaches $\bar{\theta}$5 (Mager et al., 12 Jul 2026).

EEG-JEPA adds variance–covariance regularization to reduce feature collapse. In contrast, the action-conditioned A-JEPA study argues that entropy maximization alone does not resolve component mixing. Its information-theoretic objective combines conditional likelihood maximization with entropy maximization. Under strong assumptions concerning invertibility, transition factorization, dimensionality, and action-induced variation, the representation recovers causal components up to permutation and component-wise invertible transformations (Liu et al., 25 Sep 2026).

Value and action-conditioned objectives

Value-guided JEPA world models shape latent geometry so that distance approximates negative goal-conditioned value:

$\bar{\theta}$6

The experiments show that a learned quasi-distance outperforms symmetric Euclidean distance on wall and maze tasks. This work is not a neuroscience model, but it suggests that predictive representations may require explicit behavioral or goal-related geometry to support intelligent action (Destrade et al., 28 Dec 2025).

JEPA Policy adds future-representation prediction to action-chunk prediction. Action and future tokens interact within a shared Transformer, and the future loss shapes the same action-generating representation. Across nine simulated tasks, the method reaches a mean success of $\bar{\theta}$7, compared with $\bar{\theta}$8 for MIP and $\bar{\theta}$9 for the evaluated Diffusion Policy (Xu et al., 9 Sep 2026). Its relevance to Neuro-JEPA is prospective: sensorimotor neural representations could be trained to predict action-conditioned future neural, sensory, or proprioceptive states.

Logical and graph constraints

Rule-informed JEPA, or RiJEPA, interprets prediction error as a logical energy. Valid antecedent–consequent pairs are assigned low energy, while corrupted pairs are required to exceed a margin. This produces low-energy basins for logically valid relationships and high-energy regions for invalid configurations (Huang et al., 25 Feb 2026).

Clinical Graph-JEPA instead predicts latent graph states using cosine prediction error and evaluates the frozen encoder through type-matched edge recovery. In a MIMIC-IV study, entity-grounded discharge-note injection increases overall leave-one-out MRR from $\mu$0 to $\mu$1, a relative improvement of approximately $\mu$2. This evaluates agreement with held-out graph relations under the construction pipeline, not verified clinical truth or causal validity (Yadav et al., 23 Aug 2026).

5. Empirical evidence across neural and clinical domains

EEG

Signal-JEPA uses a dataset of 54 subjects spanning motor imagery, ERP, and SSVEP paradigms. The best reported pipelines reach $\mu$3 ERP AUC, $\mu$4 SSVEP accuracy, and $\mu$5 motor-imagery accuracy. The strongest downstream architecture is pre-local: learned spatial aggregation precedes local temporal extraction and the contextual encoder is discarded. The authors interpret this as evidence that early spatial filtering is important, but the study primarily tests held-out-subject transfer within one dataset rather than strict cross-dataset transfer (Guetschel et al., 2024).

STST-JEPA is pretrained on 47,703 sessions from brain.space and HBN, spanning ages 5–81. A frozen attentive probe reaches a best validation MAE of $\mu$6 years with Pearson correlation $\mu$7 on 3,367 sessions. On NeuralBench × brain.space, partial fine-tuning reaches balanced accuracy $\mu$8 for sex classification, age correlation $\mu$9, and psychopathology correlation $450$0 (Segal et al., 7 Jul 2026).

EEG-JEPA improves the 14-task frozen macro balanced accuracy from $450$1 for a CBraMod-style masked-waveform objective to $450$2 under matched backbone, corpus, and duration. Multi-source continuation increases the result to $450$3. Under protocol-matched full fine-tuning, the nine-task average increases from $450$4 to $450$5 (Li et al., 31 Jul 2026).

fMRI and structural MRI

Brain-JEPA is pretrained on 40,162 UK Biobank participants and evaluated on demographic, cognitive, disease, and cross-ethnic tasks. On HCP-Aging, it reports age correlation $450$6, sex accuracy $450$7, Neuroticism correlation $450$8, and Flanker correlation $450$9. On ADNI, amyloid-positive versus amyloid-negative classification reaches $160$0 accuracy. On the predominantly Asian MACC cohort, NC/MCI classification reaches $160$1 accuracy despite pretraining on the predominantly Caucasian UK Biobank cohort (Dong et al., 2024).

Sparse Neuro-JEPA is pretrained on $160$2 scans from $160$3 studies and evaluated across 25 tasks from three health systems and 22 tasks from 12 public datasets. Across 41 public dataset–task–modality combinations, it reports mean AUROC $160$4 and AUPRC $160$5, exceeding the evaluated CNN and other neuroimaging foundation models under the stated protocols. Its multimodal evaluation reaches mean AUROC $160$6 and AUPRC $160$7 on twelve public multimodal combinations (Huang et al., 12 Jun 2026).

COJEPA combines JEPA and contrastive learning on T1-weighted MRI from 2,286 participants. It reaches monozygotic twin retrieval recall at rank one of $160$8, fine-tuned OpenBHB age MAE of $160$9 years, and whole-tumor BraTS Dice comparable to contrastive learning. Removing world-space positional encoding sharply degrades retrieval, while removing the convolutional stem produces a moderate decline (Mager et al., 12 Jul 2026).

Clinical graphs and neuro-symbolic extensions

Clinical Graph-JEPA demonstrates that a graph can be treated as an uncertain, evidence-scored patient-state representation rather than a fixed extraction artifact. It learns to recover held-out relations from visible graph context. A neurological extension could represent symptoms, lesions, EEG events, imaging findings, medications, cognitive scores, and temporal events as typed nodes and relations. Such a system would need explicit event-time representations because admission-level graphs do not establish exact order or causal direction (Yadav et al., 23 Aug 2026).

RiJEPA demonstrates a complementary route in which symbolic rules reshape latent geometry. In a synthetic experiment, classic JEPA yields mean valid and OOD-invalid energies of $16$0 and $16$1, whereas RiJEPA yields $16$2 and $16$3. In the reported clinical use case, RiJEPA achieves $16$4 zero-shot logical accuracy under the stated setup. These results support rule-constrained representation learning in controlled settings but do not establish general causal reasoning or broad clinical robustness (Huang et al., 25 Feb 2026).

6. Limitations, controversies, and research directions

Neuro-JEPA remains methodologically heterogeneous and empirically uneven. Several recurrent limitations constrain interpretation.

Cross-domain transfer is often narrower than the terminology suggests. Signal-JEPA is motivated by cross-dataset EEG transfer but evaluates primarily held-out subjects within the Lee2019 dataset. Brain-JEPA uses external cohorts, but its functional gradients are derived from UK Biobank and may encode population-specific organization. Sparse Neuro-JEPA evaluates multiple institutions and public datasets, yet its pretraining corpus originates from one health system. STST-JEPA uses a shared 128-channel budget but does not fully harmonize heterogeneous montages.

Predictive success does not establish causal or biological validity. A-JEPA shows theoretically that accurate prediction can coexist with component mixing. High nonlinear decoding of state information can therefore coexist with low component-wise alignment. Attention concentration, expert routing, and latent clustering are useful analyses but do not prove causal neural importance. Clinical Graph-JEPA evaluates recovery of relations in a constructed graph, not independent clinical truth.

Target semantics remain unresolved. Brain-JEPA uses deep ViT target representations, STST-JEPA uses a shallow EMA tokenizer, EEG-JEPA uses multi-depth contextual targets, and Med-D-JEPA uses target embeddings to condition generation. There is no established basis for deciding whether the optimal target should encode local physiology, deep context, anatomical structure, cross-subject invariance, future state, or a hierarchy of these quantities.

Masking can encode unwanted shortcuts. Spatial or temporal redundancy may make target prediction possible through local interpolation. Conversely, excessive masking may remove fine anatomical or physiological details. N-MET, multiscale volumetric masking, Brain-JEPA’s cross-ROI/cross-time partition, and graph patch masking are domain-specific hypotheses rather than universally optimal procedures.

Clinical and demographic confounding remains substantial. Performance can reflect scanner, site, field strength, demographic composition, documentation patterns, or disease prevalence. Brain-age residuals may contain task-state information, and strong demographic prediction does not establish disease-mechanistic validity. Clinical Graph-JEPA’s use of discharge notes may introduce information that is temporally downstream of a relation intended for prospective decision support.

Biological plausibility is limited. Existing systems rely on backpropagation, EMA target networks, global attention, offline batches, explicit masking, and task-specific optimization. Their relation to predictive coding, cortical dynamics, hippocampal representations, basal-ganglia value computation, or local synaptic learning is conceptual rather than demonstrated.

Future Neuro-JEPA research directions include multimodal neural recordings, source-space and anatomy-informed coordinates, subject- and session-invariant objectives, hierarchical temporal prediction, cross-modal target design, graph and geodesic masking, missing-sensor robustness, longitudinal modeling, and explicit intervention-based evaluation. Action-conditioned models suggest using motor commands, sensory perturbations, stimulation, task context, or pharmacological conditions as variables that modulate latent transitions (Liu et al., 25 Sep 2026). Value-guided models suggest shaping latent geometry around reachability or behavioral utility (Destrade et al., 28 Dec 2025). Policy models suggest coupling future-state prediction directly to action representations rather than treating it as a detached auxiliary loss (Xu et al., 9 Sep 2026).

Generative extensions such as Med-D-JEPA add per-token diffusion to JEPA latent prediction and use iterative next-set-of-token sampling for 3D brain-MRI synthesis. They report improvements in generation metrics and downstream classification or segmentation after synthetic pretraining, but also introduce risks of anatomical hallucination, pathology artifacts, pseudo-label contamination, and privacy leakage (Zhou et al., 28 Aug 2026). Rule-informed and graph-based extensions further suggest that Neuro-JEPA could incorporate logical constraints, provenance, uncertainty, and structured patient-state relations (Huang et al., 25 Feb 2026, Yadav et al., 23 Aug 2026).

The most defensible characterization of Neuro-JEPA is therefore not a single foundation model but a research program organized around three questions: what latent neural content should be predicted, where should prediction occur in neural space and time, and at what representational depth should supervision be applied? Its strongest demonstrated principles are latent rather than raw-signal prediction, EMA-stabilized targets, domain-specific masking, meaningful spatial or functional coordinates, and evaluation across frozen transfer and task adaptation. Its central unresolved problem is whether these predictive representations encode mechanisms that remain valid under changes in subjects, tasks, modalities, interventions, anatomy, and acquisition systems.