I-JEPA: Image-based JEPA Architecture

• I-JEPA is an image-based joint-embedding predictive architecture that learns by predicting latent representations of masked regions to capture semantic content.
• It employs a context encoder, an EMA-updated target encoder, and a bottlenecked predictor using Vision Transformers for efficient self-supervised learning.
• Its strategic multi-block masking and latent target prediction improve performance on tasks such as object counting and depth prediction.

Searching arXiv for the cited I-JEPA papers and closely related variants. Image-JEPA, or I-JEPA (Image-based Joint-Embedding Predictive Architecture), is a non-generative self-supervised learning method for images that learns by predicting the representations of masked image regions from visible context, rather than reconstructing pixels. In the original formulation, a single context block is used to predict several target blocks from the same image, so the training signal is applied in embedding space and is intended to bias the model toward semantic structure rather than low-level appearance. Within the broader JEPA family, I-JEPA has become a reference formulation for masked latent prediction with Vision Transformers, and later work has extended it through spatial conditioning, CNN backbones, discriminative sequential prediction, text conditioning, and object-centric alignment (Assran et al., 2023, Littwin et al., 2024, Kalapos et al., 2024, He et al., 21 Nov 2025, Huang et al., 5 May 2026, Geusen et al., 2 Jul 2026).

1. Conceptual position within self-supervised learning

I-JEPA is the image instantiation of a Joint-Embedding Predictive Architecture. The original JEPA distinction is three-way. A Joint-Embedding Architecture encodes two views and makes their embeddings similar. A Generative Architecture predicts the raw pixels or tokens of missing content. A JEPA predicts the embeddings of missing content, not the content itself. In that sense, I-JEPA is like masked image modeling, but the prediction target is latent rather than pixel-level (Assran et al., 2023).

This positioning is central to how I-JEPA is interpreted. The method is neither purely contrastive nor purely generative; it is a predictive embedding method. Unlike contrastive and invariance-based methods such as SimCLR, MoCo, DINO, and iBOT, it does not rely on multiple augmented global views, explicit negative pairs, or hand-crafted view augmentations during pretraining. Unlike MAE-, BEiT-, SimMIM-, or SparK-style reconstruction methods, it does not ask the model to reproduce raw image content. The intended consequence is that prediction targets become more abstract and compact, which the original work argues is better aligned with semantic representation learning (Assran et al., 2023, Kalapos et al., 2024).

A common simplification is to treat I-JEPA as merely “MAE in latent space.” That is incomplete. The original formulation makes the masking strategy a core design choice, with large semantic targets and a sufficiently informative context block required to steer the system toward semantic rather than local texture prediction. This suggests that I-JEPA is defined not only by latent regression, but by a specific coupling of latent prediction and structured masking (Assran et al., 2023).

2. Core architecture and predictive objective

The baseline architecture has three components: a context encoder $f_\theta$, a target encoder $f_{\bar{\theta}}$, and a predictor $g_\phi$. All three are instantiated with Vision Transformers in the original image formulation, with the target encoder updated by exponential moving average (EMA) of the context encoder rather than direct gradient optimization (Assran et al., 2023).

Given target blocks $B_i$, the learning objective is the average squared $L_2$ distance between predicted and target patch embeddings: $$\frac{1}{M} \sum_{i=1}^{M} D\left(\hat{y}(i), y(i)\right) = \frac{1}{M} \sum_{i=1}^{M} \sum_{j \in B_i} \left\lVert \hat{y}_j - y_j \right\rVert_2^2 .$$ Here, $y(i)=\{y_j\}_{j\in B_i}$ are target-encoder representations, and $\hat{y}(i)=\{\hat{y}_j\}_{j\in B_i}$ are predictor outputs. The predictor receives the context-encoder output together with positional mask tokens indicating where the missing target block is located. The target encoder provides the latent targets, and its EMA update stabilizes those targets during training (Assran et al., 2023).

The original paper describes the roles of the components in patch-level terms. The context encoder processes only the visible context block and produces patch-level context representations. The target encoder produces patch-level representations for the image. The predictor, implemented as a narrower ViT, uses context features plus positional mask tokens to predict the target block representation at the masked location. In the default image setup, I-JEPA is trained without a $[CLS]$ token, and evaluation is performed by average pooling over patch representations (Assran et al., 2023).

The predictor architecture is intentionally bottlenecked. Its embedding dimension is fixed at 384, it uses the same number of attention heads as the backbone, and its depth depends on model size: 6 for ViT-B/16, 12 for ViT-L/16, ViT-H/16, and ViT-H/14, and 16 for ViT-G/16. The original ablations report that deeper predictors improve performance and that a bottleneck predictor width helps, indicating that the predictor is not a neutral accessory but part of the representational bias of the method (Assran et al., 2023).

3. Masking geometry and the semantics of prediction

The masking policy is one of the defining technical features of I-JEPA. By default, the method samples $M=4$ target blocks per image. Each target block is a random rectangle with scale in $f_{\bar{\theta}}$0 and aspect ratio in $f_{\bar{\theta}}$1. One large context block is sampled with scale in $f_{\bar{\theta}}$2 and unit aspect ratio, and any overlap between context and targets is removed (Assran et al., 2023).

The original paper emphasizes that the targets are obtained by masking the output of the target encoder, not the raw input image. This is a decisive difference from pixel-reconstruction schemes. If the loss is applied in pixel space rather than representation space, downstream performance drops substantially. Likewise, the paper reports that multi-block masking outperforms rasterized, block, and random masking; predicting larger target blocks helps; using a larger context block helps; and predicting more target blocks helps (Assran et al., 2023).

The stated rationale is that semantic learning depends on forcing the model to infer object parts, layout, and scene content from informative but incomplete context. If targets are too small or too random, the task becomes too local and pixel-like. If the context is too small, the model lacks the information needed to infer higher-level structure. In that sense, the masking scheme defines the prediction problem, not merely its computational footprint (Assran et al., 2023).

Later work makes this dependency more explicit. EC-IJEPA argues that predictability is strongly spatially biased in natural images and that baseline I-JEPA encoders are not given sufficient geometric conditioning about the context-target relationship. In standard I-JEPA, the predictor gets target positions, but the encoders themselves do not. EC-IJEPA therefore conditions the context encoder on target positions and the target encoder on context positions: $f_{\bar{\theta}}$3

$f_{\bar{\theta}}$4

This modification is motivated by the claim that poor context-target geometries can lead to brittle features or even representational collapse, while overly easy geometries may produce shallow representations. The same work reports that baseline I-JEPA is sensitive to the size and separation of context and target windows, whereas spatial conditioning reduces that sensitivity (Littwin et al., 2024).

4. Training regime, scaling behavior, and original empirical profile

The original I-JEPA training setup uses AdamW, batch size 2048, a learning rate warmed from $f_{\bar{\theta}}$5 to $f_{\bar{\theta}}$6 over the first 15 epochs and then cosine-decayed to $f_{\bar{\theta}}$7, weight decay increased linearly from 0.04 to 0.4, and target-encoder EMA momentum increased from 0.996 to 1.0. The mask sampler returns mask indices per image from the data loader. These implementation details matter because the method’s stability is tied to the EMA teacher and the masking geometry (Assran et al., 2023).

A major claim of the original paper is scalability. It reports that a ViT-H/14 pretrained on ImageNet can be trained in under 72 hours on 16 A100 GPUs, and more generally that pretraining a ViT-H/14 requires less than 1200 GPU hours. This is described as over 2.5× faster than a ViT-S/16 pretrained with iBOT and over 10× more efficient than a ViT-H/14 pretrained with MAE. Although I-JEPA incurs a small overhead from computing representation targets, it is reported to converge in about 5× fewer iterations than MAE (Assran et al., 2023).

On ImageNet linear evaluation, the reported accuracies are 72.9 for ViT-B/16, 77.5 for ViT-L/16, 79.3 for ViT-H/14, and 81.1 for ViT-H/16$f_{\bar{\theta}}$8. On ImageNet 1% semi-supervised evaluation, the reported accuracies are 69.4 for ViT-L/16, 73.3 for ViT-H/14, and 77.3 for ViT-H/16$f_{\bar{\theta}}$9. For full ImageNet fine-tuning, ViT-H/16$g_\phi$0 reaches 87.1 top-1 accuracy, which the paper states is within less than 1% of the best MAE result despite using about 5.3× fewer epochs (Assran et al., 2023).

The downstream profile is not restricted to classification. The original paper emphasizes stronger performance on object counting and much better performance on depth prediction than view-invariance methods such as DINO and iBOT. It also reports improved transfer on downstream classification tasks such as CIFAR100, Places205, and iNaturalist18, stating that I-JEPA closes the gap to view-invariance methods and surpasses DINO on some tasks like CIFAR100 and Places205 under linear probing (Assran et al., 2023).

These results are often interpreted as evidence that I-JEPA trades off low-level fidelity for semantics. The reported gains on counting and depth prediction complicate that interpretation. A plausible implication is that latent prediction, when coupled to the original masking policy, can preserve spatial structure while still suppressing unnecessary pixel detail (Assran et al., 2023).

5. Architectural variants and extensions

Several later works retain the I-JEPA principle of latent target prediction while modifying either the conditioning signal, the backbone, or the region-selection procedure.

EC-IJEPA (“Encoder Conditioned I-JEPA”) modifies baseline I-JEPA by giving the context encoder information about where the target lies and the target encoder information about where the context lies. Conditioning is implemented by appending positions as additional tokens to the transformer input sequence. To keep the overhead small, training uses 1D average pooling over conditioning positions with kernel and stride $g_\phi$1, adding about 3% FLOPs, while inference uses 2D average pooling over all positions with kernel and stride $g_\phi$2, adding about $g_\phi$3 extra tokens. On ImageNet-1k linear probing, reported gains are 74.8 $g_\phi$4 76.7 for ViT-L/16 and 77.4 $g_\phi$5 78.1 for ViT-H/14. RankMe improves from 488.6 $g_\phi$6 533.0 and 540.8 $g_\phi$7 567.3, and LiDAR improves from 385.2 $g_\phi$8 486.5 and 437.2 $g_\phi$9 547.0, for ViT-L/16 and ViT-H/14 respectively. The paper also reports greater robustness to context-window size and higher sample-efficiency throughout pretraining (Littwin et al., 2024).

CNN-JEPA adapts the I-JEPA training logic to convolutional backbones. Its key changes are a sparse CNN encoder for masked inputs, a fully convolutional predictor built from depthwise separable convolutions, and a modified masking strategy in which the union of all masked blocks is treated as one target region. The predictor uses repeated blocks of

$B_i$0

The masking policy is mixed: with probability 0.75, multi-block masking is used; otherwise, random masking is used. On ImageNet-100, CNN-JEPA with ResNet-50 reports 73.26% linear top-1 and 59.22% k-NN top-1, compared with 42.30% and 34.84% for I-JEPA with ViT-Small and 46.36% and 31.36% for I-JEPA with ViT-Base. The paper also states that CNN-JEPA requires 17–35% less pretraining time than SimCLR and BYOL respectively for the same number of epochs, with a concrete runtime of 13 hours over 200 epochs on 4 A100 GPUs (Kalapos et al., 2024).

DSeq-JEPA retains the JEPA objective but changes which regions are predicted and in what order. It derives a saliency map from the transformer, extracts discriminative regions, and predicts them sequentially from primary to secondary cues, which the paper describes as bridging JEPA-style latent prediction with GPT-style sequential reasoning. The work characterizes vanilla I-JEPA as “flat” over regions: targets are sampled uniformly and predicted independently. On ImageNet linear probing, DSeq-JEPA improves 72.4 $B_i$1 73.5 for ViT-B/16 and 77.1 $B_i$2 77.9 for ViT-L/16. The ablations are especially explicit: uniform + sequential slightly hurts, selective + flat underperforms, selective + random sequential hurts more, and selective + sequential is best, which suggests that selection and ordering are jointly important rather than independently sufficient (He et al., 21 Nov 2025).

TC-JEPA adds text conditioning to reduce uncertainty at masked positions. It keeps the image encoder, EMA target encoder, and narrow ViT predictor, but injects caption information into the predictor through lightweight sparse cross-attention over T5 token embeddings at multiple layers. The full objective is

$B_i$3

with reported defaults $B_i$4 and $B_i$5. For multi-caption conditioning, captions are handled separately and the conditioned patch features are max-pooled across captions; the paper reports that MaxPool performs best overall. On IN-1k linear probing, the reported gains are 72.9 $B_i$6 75.8 for ViT-B/16, 77.5 $B_i$7 79.6 for ViT-L/16, and 79.3 $B_i$8 80.4 for ViT-H/14. On dense tasks, the reported ViT-H/14 gains include COCO AP$B_i$9 53.7 $L_2$0 55.2, ADE20k linear mIoU 36.9 $L_2$1 39.5, ADE20k finetune mIoU 51.2 $L_2$2 55.7, VOC linear mIoU 64.6 $L_2$3 70.4, and VOC finetune mIoU 81.4 $L_2$4 83.8. The paper further states that inference cost is unchanged because text is used only during pretraining (Huang et al., 5 May 2026).

Within the wider JEPA lineage, Object-centric LeJEPA addresses a limitation attributed to image-level self-supervision: whole-image alignment can be too coarse because two random crops of one image may contain different objects. It therefore aligns object-centric representations using SAM 2 proposals and adds an instance-separating loss. The paper states that with only 10% of COCO, Object LeJEPA (ViT-B) matches or exceeds image-level LeJEPA trained on 100% of COCO on DAVIS tracking (0.648 vs 0.613), ImageNet-1k (0.472 vs 0.463), ADE20k (0.380 vs 0.339), and NAVI 1-shot (0.343 vs 0.337). This does not modify I-JEPA directly, but it marks a significant conceptual shift away from scene-level alignment toward object-level training units (Geusen et al., 2 Jul 2026).

6. Limitations, sensitivities, and recurrent misconceptions

One recurrent misconception is that I-JEPA is simply a reconstruction system with a different loss. The original formulation explicitly rejects pixel reconstruction as the primary target, and later extensions preserve that commitment. The target is a learned representation, not the raw content of the masked region. This distinction is not cosmetic: the original ablations report that predicting in representation space is better than predicting in pixel space (Assran et al., 2023).

A second misconception is that I-JEPA is agnostic to masking details. The published evidence points in the opposite direction. The original method requires large semantic targets, an informative context block, and multi-block masking; EC-IJEPA reports that baseline I-JEPA is sensitive to context-window size and context-target separation; and TC-JEPA argues that low mutual information between context and target can make feature prediction unstable or lead to poor semantics. This suggests that the success of I-JEPA depends on the geometry and information content of the masked-prediction task, not only on the latent-regression objective (Littwin et al., 2024, Huang et al., 5 May 2026).

A third misconception is that latent prediction necessarily discards local structure in favor of global semantics. The original paper reports stronger object counting and much better depth prediction than view-invariance methods, while later variants such as TC-JEPA improve substantially on dense prediction and DSeq-JEPA improves on structured reasoning tasks such as Clevr/Count and Clevr/Dist. These findings indicate that JEPA-style latent prediction can retain local and spatial information when the prediction target and region selection are appropriately designed (Assran et al., 2023, He et al., 21 Nov 2025, Huang et al., 5 May 2026).

At the same time, the extension literature also documents limits. EC-IJEPA improves transfer on most reported benchmarks, but it is worse on EuroSAT for ViT-L/16 (96.7 $L_2$5 95.7) and markedly worse on CLEVR/Dist for ViT-L/16 (65.3 $L_2$6 60.0), with near-parity on the ViT-H/14 CLEVR tasks. Object-centric LeJEPA, although more data efficient than image-level LeJEPA on several tasks, still relies on external SAM 2 masks and remains far below DINOv3 on the most semantic tasks. These results indicate that the JEPA design space is productive but not uniformly solved, and that improvements in one regime do not imply universal gains across all tasks and inductive biases (Littwin et al., 2024, Geusen et al., 2 Jul 2026).