JEPA: Joint-Embedded Predictive Architectures

• Joint-Embedded Predictive Architectures (JEPA) are a family of representation-learning models that project context and target views into a shared latent space, eliminating the need for pixel-level reconstruction.
• JEPA models employ a versatile architecture with context and target encoders, a predictor, and a latent comparator, making them applicable across images, video, audio, time series, and more.
• Advanced regularization strategies and energy-based or probabilistic formulations in JEPA prevent collapse and improve planning performance in control and multimodal tasks.

Joint-Embedding Predictive Architectures (JEPAs) are a family of representation-learning models in which a context view and a target view are mapped into a common latent space, a predictor estimates the target embedding from the context embedding, and training minimizes discrepancy in that latent space rather than reconstructing raw pixels, waveforms, coordinates, or tokens. Across recent work, the same template has been interpreted as a self-supervised predictor, an energy-based model, a latent world model, a quasimetric value function, and, in probabilistic extensions, a predictive information state for control; it has also been instantiated in images, video, time series, audio, trajectories, molecular graphs, ultrasound, text-to-image generation, vision-language modeling, and masked diffusion LLMs (Kobanda et al., 12 Feb 2026, Terver et al., 3 Feb 2026, Ennadir et al., 29 Sep 2025, Tuncay et al., 25 Jun 2025, Li et al., 2024, Chen et al., 11 Dec 2025, Nam, 24 May 2026).

1. Formal structure and architectural template

In the standard formulation, a JEPA contains a context encoder, a target encoder, a predictor, and a comparator in latent space. A common notation is

$$z_x = f_o(x), \qquad z_y = f_a(y), \qquad \hat z_y = p_\epsilon(z_x; c),$$

where $x$ is a context input, $y$ is a target input, $c$ is optional conditioning, and $D(\cdot,\cdot)$ measures discrepancy between the predicted and target embeddings. This induces a scalar compatibility or energy

$E(x,y) := D\big(p_\epsilon(f_o(x)); f_a(y)\big),$

so that observed context–target pairs are assigned low energy and incompatible pairs higher energy (Kobanda et al., 12 Feb 2026).

This template is deliberately broader than masked-image prediction. In image JEPA variants, the context and target are different spatial regions or masked blocks. In video and time-indexed settings, the context is a past or partially observed segment and the target is a future or masked segment. In planning-oriented variants, the same idea becomes latent dynamics: $$z_t = \mathcal{E}_\theta(s_t), \qquad \hat z_{t+1} = f_\psi(z_t, a_t),$$ or, with an explicit action encoder, $u = q_\omega(a)$ and a predictor $g_\phi(z,u)$. In all cases, prediction is performed in representation space rather than observation space (Destrade et al., 28 Dec 2025, Terver et al., 3 Feb 2026).

A central distinction from pixel-level or autoregressive prediction is that JEPA does not require the model to regenerate the full raw output. The target of learning is a latent representation that can discard nuisance factors and preserve predictive or task-relevant structure. This is the recurring rationale across image, video, time-series, audio, and world-model instantiations (Ennadir et al., 29 Sep 2025, Tuncay et al., 25 Jun 2025).

2. Objectives, regularization, and collapse avoidance

The simplest JEPA loss is a latent regression objective,

$$\mathcal{L}_{\text{pred}}=\mathbb{E}\big[\ell(g_\phi(f_\theta(\text{context})), f_{\bar\theta}(\text{target}))\big],$$

often implemented with an EMA target encoder, stop-gradient on the target branch, and either squared error, $x$0, cosine, or Smooth $x$1. However, predictive consistency alone admits trivial collapse. In action-conditioned settings, for example, a constant encoder and constant latent transition can minimize the latent dynamics loss exactly, which makes collapse an explicit design problem rather than a pathology confined to a few implementations (Yu et al., 12 Sep 2025).

Several stabilization strategies have therefore appeared. One line uses variance–covariance regularization, as in VCReg or VICReg, either directly on embeddings or on a projector space. Another uses SIGReg, which regularizes projected embeddings toward an isotropic Gaussian. EB-JEPA packages these mechanisms in a modular energy-based formulation and reports that ablations on Two Rooms planning drop from $x$2 success for the full model to $x$3 without the variance term, $x$4 without the covariance term, $x$5 without temporal similarity, and $x$6 without the inverse-dynamics loss, illustrating that non-collapse and control utility depend strongly on the regularizer class (Terver et al., 3 Feb 2026).

A second line critiques the sufficiency of EMA asymmetry by itself. C-JEPA argues that EMA in I-JEPA is ineffective in preventing entire collapse and that I-JEPA prediction is inadequate for learning the mean of patch representations; it therefore augments JEPA with VICReg-style invariance, variance, and covariance terms (Mo et al., 2024). A third line replaces the online teacher entirely: US-JEPA adopts Static-teacher Asymmetric Latent Training, using a frozen domain-specific ultrasound encoder to provide stable latent targets, thereby decoupling student–teacher optimization and avoiding the hyperparameter-brittle EMA loop in noisy medical imaging (Radhachandran et al., 22 Feb 2026).

Recent theory makes the role of auxiliary tasks explicit. For a deterministic MDP, a JEPA encoder $x$7, latent transition $x$8, and auxiliary head $x$9 trained jointly on

$y$0

satisfy a No Unhealthy Representation Collapse theorem: if both latent-transition consistency and auxiliary regression are driven to zero, then any pair of non-equivalent observations—those that do not share the same transition dynamics or auxiliary label—must map to distinct latent states (Yu et al., 12 Sep 2025). This establishes that auxiliary tasks do not merely improve optimization; they define which distinctions the representation is forced to preserve.

Rectified LpJEPA generalizes Gaussian regularization by replacing dense isotropic targets with a sparse, non-negative Rectified Generalized Gaussian prior and matching projected feature distributions to that prior via Rectified Distribution Matching Regularization. In that formulation, the expected $y$1 norm is analytically controllable, and sparsity becomes an explicit property of the target latent law rather than an indirect consequence of post hoc thresholding (Kuang et al., 1 Feb 2026).

3. Energy, geometry, and probabilistic reformulations

One influential reinterpretation treats JEPA as an energy-based model over state pairs. In that view, the JEPA-induced discrepancy is not merely a training loss but a scalar compatibility energy $y$2. A particularly structured class is the intrinsic or least-action energy

$y$3

where $y$4 is a set of admissible trajectories and $y$5 is a non-negative local effort density. Under closure, additivity, and mild regularity assumptions, this intrinsic energy is a quasimetric: it satisfies reflexivity, non-negativity, identity of indiscernibles, and the triangle inequality, but not necessarily symmetry (Kobanda et al., 12 Feb 2026).

The asymmetry is not incidental. If admissibility is directed or local effort is anisotropic, then generally $y$6. The same paper proves that a symmetric finite energy cannot encode genuinely one-way reachability: if finite energy is equivalent to reachability and the energy is symmetric, then reachability itself must be symmetric. This makes symmetric latent distances structurally mismatched to irreversible dynamics, directional graphs, or goal-reaching control under asymmetric constraints (Kobanda et al., 12 Feb 2026).

A parallel reformulation makes JEPA explicitly probabilistic. VJEPA replaces deterministic latent regression with a predictive distribution $y$7 and an inference model $y$8, trained with a variational latent objective,

$y$9

In this formulation, deterministic JEPA appears as the fixed-variance Gaussian special case of latent maximum likelihood. The paper further connects VJEPA to predictive state representations and Bayesian filtering, argues that sequential modeling does not require autoregressive observation likelihoods, and proves that under stated realizability conditions the learned latent can act as a sufficient information state for optimal control without pixel reconstruction (Huang, 20 Jan 2026).

These geometric and probabilistic views are complementary rather than exclusive. The energy-based perspective emphasizes directed compatibility and compositionality; the variational perspective emphasizes uncertainty, belief propagation, and latent predictive sufficiency. Together they broaden JEPA from a training recipe into a representation-theoretic framework.

4. JEPA as a world model and planning substrate

In model-based control, JEPA is typically instantiated as an action-conditioned latent dynamics model. The canonical setup uses a state encoder $c$0, an action encoder or direct action concatenation, and a predictor rolled out recursively over a planning horizon. A standard planning cost in earlier JEPA world models is the latent distance between predicted future states and a goal embedding,

$c$1

but purely predictive JEPA representations are often not aligned with control difficulty or reachability (Destrade et al., 28 Dec 2025).

Value-guided JEPA addresses this by shaping the latent geometry so that a distance or quasi-distance between embeddings approximates the negative goal-conditioned value for a reaching cost: $c$2 The encoder is trained with an IQL-style Bellman consistency loss, and planning then combines the learned JEPA dynamics with MPC and MPPI. On Wall and Maze benchmarks, the best-performing variant is the separate-training quasidistance model VF_quasi, with success rates $c$3 on WS, $c$4 on WB, and $c$5 on Maze; joint training with prediction or VCReg generally degrades planning performance relative to this value-shaped geometry (Destrade et al., 28 Dec 2025).

The control-theoretic connection becomes sharper under the intrinsic-energy interpretation. Optimal goal-reaching cost-to-go functions have exactly the least-action form above, so any JEPA trained to approximate such intrinsic energies lies in the same quasimetric value-function class targeted by quasimetric reinforcement learning. This is the basis of the IE-JEPA notion: a JEPA whose induced energy is intrinsic is, by construction, a quasimetric cost-to-go representation (Kobanda et al., 12 Feb 2026).

Practical world-model implementations follow the same pattern. EB-JEPA includes an action-conditioned Video-JEPA with multistep rollout losses, temporal similarity, and inverse-dynamics regularization, and reports a $c$6 planning success rate on the Two Rooms navigation task. VJEPA and BJEPA go further by factorizing predictive belief into a learned dynamics expert and a modular prior expert combined as a Product of Experts, thereby enabling zero-shot insertion of goals or constraints into latent planning (Terver et al., 3 Feb 2026, Huang, 20 Jan 2026).

The practical implication is that JEPA becomes most effective for control when its latent geometry is not merely predictive but also value-aware, asymmetric where necessary, and explicitly regularized against collapse.

5. Modality-specific instantiations

Recent work has turned JEPA from a vision-centric template into a broadly reusable design pattern.

Domain	Representative variant	Distinctive JEPA mechanism
Time series	TS-JEPA (Ennadir et al., 29 Sep 2025)	Temporal patch masking with Transformer encoder, predictor, and EMA target encoder
Motion and content	MC-JEPA (Bardes et al., 2023)	Shared encoder for optical flow prediction and VICReg-style content learning
Trajectories	T-JEPA (Li et al., 2024)	Representation-space target resampling from GPS or check-in trajectories
Audio	Audio-JEPA (Tuncay et al., 25 Jun 2025)	Masked spectrogram-patch prediction with ViT context and EMA target encoders
Polymer graphs	Graph JEPA (Piccoli et al., 22 Jun 2025)	Context and target subgraph embeddings predicted with positional conditioning
Ultrasound	US-JEPA (Radhachandran et al., 22 Feb 2026)	SALT objective with frozen ultrasound teacher and region-conditioned masking

TS-JEPA adapts JEPA to univariate time series by dividing sequences into non-overlapping temporal patches, masking $c$7 of them during pretraining, encoding visible patches with a Transformer, predicting masked latent codes, and using an EMA target encoder with $c$8. It is reported to match or surpass current state-of-the-art baselines on standard datasets while maintaining a strong balance across classification and forecasting, with better long-term rollout behavior than autoregressive baselines on two of three forecasting datasets (Ennadir et al., 29 Sep 2025).

MC-JEPA unifies motion and content learning inside a shared ConvNeXt-T encoder by combining a motion-prediction branch for optical flow with a VICReg content branch. The model achieves performance on-par with existing unsupervised optical flow benchmarks and with common self-supervised approaches on downstream semantic segmentation, while showing that motion and content objectives can improve one another within a single JEPA-style encoder (Bardes et al., 2023).

T-JEPA applies the paradigm to GPS and check-in trajectories by predicting latent subsets of a trajectory from partially observed context trajectories, using AdjFuse for local spatial aggregation and Transformer encoders on both target and context branches. The method is reported to outperform or match strong contrastive baselines on three urban trajectory datasets and two Foursquare datasets, particularly under distortion and irregular sampling (Li et al., 2024).

Audio-JEPA is a direct translation of I-JEPA to mel-spectrograms. It uses a ViT-Base backbone, random masking of $c$9 to $D(\cdot,\cdot)$0 of patches, an EMA target encoder, and an $D(\cdot,\cdot)$1 latent prediction objective on masked spectrogram regions. On X-ARES, it shows comparable performance to wav2vec 2.0 and data2vec while using less than one-fifth of their training data, with especially strong kNN results on ESC-50, FMA small, and GTZAN (Tuncay et al., 25 Jun 2025).

Graph JEPA for polymers defines context and target subgraphs, encodes them with weighted D-MPNN backbones, and predicts target subgraph embeddings from a context embedding plus random-walk structural encodings. The paper reports that JEPA-based self-supervised pretraining improves downstream performance across all tested polymer datasets when labels are scarce, with particularly strong gains in the smallest labeled-data regimes (Piccoli et al., 22 Jun 2025).

US-JEPA shows how JEPA can be specialized for a harsh imaging modality. By pairing a randomly initialized ViT-B/16 student with a frozen URFM teacher under SALT and restricting masks to valid anatomical regions via Ultrasound Region Conditioning, it achieves performance competitive with or superior to domain-specific and universal vision foundation model baselines on UltraBench, while exhibiting stronger robustness to blur and speckle than reconstruction-style ultrasound models (Radhachandran et al., 22 Feb 2026).

6. Multimodal, generative, and language-model extensions

JEPA has recently moved beyond pure representation learning into multimodal prediction and conditional generation. JEPA-T is a text-to-image generator in which images are tokenized by a VAE, text is encoded by a frozen CLIP ViT-B/16 text encoder, and a joint-embedding predictive Transformer predicts masked or noisy visual token embeddings. The model combines a JEPA masked-prediction loss with conditional flow matching,

$D(\cdot,\cdot)$2

and uses late cross-attention after the predictor plus objective-level text injection. On ImageNet-1K at $D(\cdot,\cdot)$3, the reported full model reaches FID $D(\cdot,\cdot)$4, IS $D(\cdot,\cdot)$5, Precision $D(\cdot,\cdot)$6, and Recall $D(\cdot,\cdot)$7, outperforming non-fusion and late-fusion ablations (Wan et al., 1 Oct 2025).

VL-JEPA replaces token autoregression with prediction of continuous target-text embeddings. With a V-JEPA 2 vision encoder, a predictor initialized from the last 8 layers of Llama-3.2-1B, a Y-Encoder initialized from EmbeddingGemma-300M, and an optional lightweight text decoder used only when needed, the model trains in embedding space via InfoNCE rather than token cross-entropy. In a controlled comparison against standard token-space VLM training with the same vision encoder and training data, VL-JEPA achieves stronger performance while having $D(\cdot,\cdot)$8 fewer trainable parameters. It also supports selective decoding, reducing decoding operations by $D(\cdot,\cdot)$9 at similar performance, and its embedding space directly supports open-vocabulary classification, text-to-video retrieval, and discriminative VQA without architecture modification (Chen et al., 11 Dec 2025).

DLLM-JEPA adapts the idea to masked diffusion LLMs. Two semantically distinct views of the same sequence are produced by different masking rates $E(x,y) := D\big(p_\epsilon(f_o(x)); f_a(y)\big),$0, the online backbone processes the lower-mask view, an EMA target encoder processes the higher-mask view under no-grad, and a predictor aligns the pooled embeddings with a cosine JEPA loss added to the diffusion denoising objective. Because bidirectional attention supplies both views within the same substrate, DLLM-JEPA requires only one gradient-carrying forward pass and cuts training FLOPs by $E(x,y) := D\big(p_\epsilon(f_o(x)); f_a(y)\big),$1 relative to LLM-JEPA. It improves over diffusion-only fine-tuning in every evaluated task–architecture combination, including gains of up to $E(x,y) := D\big(p_\epsilon(f_o(x)); f_a(y)\big),$2 percentage points on LLaDA-8B GSM8K and $E(x,y) := D\big(p_\epsilon(f_o(x)); f_a(y)\big),$3 on Dream-7B GSM8K (Nam, 24 May 2026).

These extensions have sharpened several ongoing debates. One concerns collapse prevention: C-JEPA argues that EMA alone is not a principled safeguard and adds variance–invariance–covariance regularization; US-JEPA avoids the issue by using a frozen teacher; auxiliary-task theory instead treats the preserved distinctions as those induced by dynamics and labels (Mo et al., 2024). A second concerns the geometry of the latent space: intrinsic-energy work argues that symmetric energies are structurally mismatched to one-way reachability, while value-guided planning reports gains from quasi-distances over Euclidean distances (Kobanda et al., 12 Feb 2026). A third concerns the latent prior itself: VJEPA replaces deterministic point prediction with predictive latent distributions, whereas Rectified LpJEPA replaces isotropic Gaussian targets with sparse maximum-entropy Rectified Generalized Gaussians (Huang, 20 Jan 2026, Kuang et al., 1 Feb 2026).

Taken together, the recent literature suggests that JEPA is no longer a single image-pretraining recipe. It is a general latent predictive paradigm whose specific behavior depends on four coupled choices: how views are constructed, how target embeddings are supplied, what geometric or probabilistic structure is imposed on the latent space, and which invariances or control-relevant distinctions are allowed to survive.