DéjàView: Looping Transformers & Beyond

• DéjàView is a multi-domain concept encompassing looping transformer architectures for 3D reconstruction, unintended memorization in SSL/VLMs, video query acceleration, and view-driven deduplication.
• In multi-view 3D reconstruction, the looping transformer iteratively refines depth maps, ray projections, and camera parameters, achieving competitive benchmarks with fewer parameters and computational efficiency.
• Other applications leverage DéjàView for analyzing memorization effects, scalable place recognition through repetitive landmark frequencies, and screen-guided deduplication in visual analytics.

“DéjàView” is not a single standardized research term. In arXiv usage, it most explicitly names a looping transformer architecture for multi-view 3D reconstruction, while closely related “Déjà Vu” formulations designate unintended memorization in self-supervised and vision-language representations, a VideoLM query engine based on inter-frame computation reuse, a grid-cell-inspired place-recognition method, and a view-driven deduplication paradigm in visual analytics (Burzio et al., 28 May 2026, Meehan et al., 2023, Kokhlikyan et al., 8 Apr 2025, Hwang et al., 17 Jun 2025, Jacobson et al., 2017, Morton et al., 2016). Across these lines, the recurrent theme is the exploitation of structure already present in a “view,” whether that view is a set of camera images, a background crop, a video frame sequence, a repetitive landmark pattern, or a relational visualization.

1. Research usages of the term

The term is used across several technically distinct literatures.

Usage	Domain	Core object
“Déjà View”	Multi-view 3D reconstruction	Looped transformer block with explicit iteration
“Déjà vu memorization”	SSL and VLM auditing	Foreground recovery from background/context beyond correlation
“Déjà Vu” query engine	Video analytics	ReuseViT with GPU compaction for VideoLMs
“Deja vu” place recognition	Robotics	Mutually supportive feature frequencies
“DéjàView idea”	Visual analytics	View-driven deduplication under labeling budget

The most direct nominal match is “Déjà View: Looping Transformers for Multi-View 3D Reconstruction,” which treats model depth as partially buying iteration and replaces a deep decoder stack with a single recurrently applied transformer block (Burzio et al., 28 May 2026). The memorization line, by contrast, uses “déjà vu” to denote the recovery of training-image-specific foreground information from background-only or context-only inputs, first for SSL encoders and then for pre-trained open-source image and vision-LLMs (Meehan et al., 2023, Kokhlikyan et al., 8 Apr 2025). The video query-engine work uses the same name for a systems pipeline that accelerates ViT-based VideoLMs through learned inter-frame reuse and memory-compute joint compaction (Hwang et al., 17 Jun 2025). Earlier work in robotics used “Deja vu” for a place-recognition algorithm based on repetitive, mutually complementary landmark frequencies rather than unique landmarks (Jacobson et al., 2017). A further related usage appears in view-driven deduplication, where the system prioritizes record cleaning according to impact on the current visualization rather than on the whole dataset (Morton et al., 2016).

2. DéjàView as a looping transformer for multi-view 3D reconstruction

In its most explicit contemporary usage, DéjàView is a multi-view 3D reconstruction model that takes a set of RGB images $\{I_i\}_{i=1}^V$ and predicts per-view depth maps $D_i \in \mathbb{R}^{H \times W}$, per-view ray maps $R_i \in \mathbb{R}^{H \times W \times 6}$, dense 3D pointmaps, and camera parameters in a common 3D frame (Burzio et al., 28 May 2026). The per-pixel 3D point is represented as

$\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$

and the camera head predicts

$$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$

This formulation targets metric geometry directly rather than implicit fields or Gaussian splats.

The architectural claim is that contiguous transformer layers often behave like repeated applications of similar operations, and that multi-view reconstruction transformers refine their predictions progressively across decoder depth. DéjàView therefore applies a single looped transformer block recurrently for $K$ refinement steps:

$z_{k+1} = f_\theta(z_k, t_k, t_{k+1}),$

with the same block reused at every step and conditioned on a continuous time interval. During training, $K \sim \text{Beta}(2,1)$ scaled to $[K_{\min}, K_{\max}] = [8, 16]$, so the trained checkpoint exposes $K$ as an inference-time compute knob (Burzio et al., 28 May 2026).

The significance of the shared-block design is not merely parameter efficiency. An ablation comparing a decoupled 16-block variant against a shared recurrent block reports that the decoupled model reaches Rel L2 $D_i \in \mathbb{R}^{H \times W}$0, IR $D_i \in \mathbb{R}^{H \times W}$1, and AUC@30 $D_i \in \mathbb{R}^{H \times W}$2, whereas the full shared model with residual and state gating reaches Rel L2 $D_i \in \mathbb{R}^{H \times W}$3, IR $D_i \in \mathbb{R}^{H \times W}$4, and AUC@30 $D_i \in \mathbb{R}^{H \times W}$5 (Burzio et al., 28 May 2026). The paper explicitly interprets this as evidence that explicit iteration is a stronger inductive bias for multi-view 3D reconstruction, not only a compute-efficient substitute for depth.

On efficiency and benchmarks, DéjàView is reported at 117M parameters, 75.9 TFLOPs total, 3.2 TFLOPs/image for 24 views, 4.9 GiB peak GPU memory, average IR $D_i \in \mathbb{R}^{H \times W}$6, and average AUC@30° $D_i \in \mathbb{R}^{H \times W}$7 (Burzio et al., 28 May 2026). It is evaluated on DTU, ETH3D, 7-Scenes, ScanNet++, and nuScenes, and is described as matching or outperforming substantially larger feed-forward baselines across these five benchmarks while using a fraction of their parameters and comparable or lower compute (Burzio et al., 28 May 2026). A common misconception is that weight tying is only compression; the shared-vs-decoupled ablation directly argues against that interpretation.

3. Déjà vu memorization in SSL and VLMs

A second major usage of the term concerns unintended memorization in representation learning. The original SSL work defines “déjà vu memorization” as the retention of image-specific information such that, given a trained model and a crop of a training image containing only the background, it is possible to infer the foreground object with high accuracy or even visually reconstruct it (Meehan et al., 2023). The formal setup writes a labeled image as

$D_i \in \mathbb{R}^{H \times W}$8

and defines a point as memorized if there exists a predictor $D_i \in \mathbb{R}^{H \times W}$9 such that

$R_i \in \mathbb{R}^{H \times W \times 6}$0

The distinction is therefore not mere background-label correlation, but performance beyond what can be explained by dataset-level correlations (Kokhlikyan et al., 8 Apr 2025).

The original SSL study reports that, for VICReg with ResNet-101 and 300k training images, label inference from background-only crops reaches almost $R_i \in \mathbb{R}^{H \times W \times 6}$1 accuracy on the top $R_i \in \mathbb{R}^{H \times W \times 6}$2 most confident crops, and that for VICReg at $R_i \in \mathbb{R}^{H \times W \times 6}$3 confidence the déjà vu score grows from less than $R_i \in \mathbb{R}^{H \times W \times 6}$4 at 250 epochs to more than $R_i \in \mathbb{R}^{H \times W \times 6}$5 at 1000 epochs (Meehan et al., 2023). It further reports that the memorized set exceeds $R_i \in \mathbb{R}^{H \times W \times 6}$6 of training samples for VICReg trained 1000 epochs on 300k images, whereas a supervised model trained on the same data exhibits a much smaller déjà vu score of about $R_i \in \mathbb{R}^{H \times W \times 6}$7 (Meehan et al., 2023). The paper also states that this behavior cannot be detected by conventional techniques for evaluating representation quality, because linear-probe validation can improve while déjà vu memorization worsens (Meehan et al., 2023).

The efficient-measurement extension removes the need to train two large shadow models. For image SSL, it proposes simple correlation estimators, including a ResNet-50 background classifier and a Naive Bayes classifier over detected objects, and defines

$R_i \in \mathbb{R}^{H \times W \times 6}$8

For individual samples it defines a memorization confidence as

$R_i \in \mathbb{R}^{H \times W \times 6}$9

so high-confidence SSL prediction paired with high-entropy correlation prediction indicates likely memorization (Kokhlikyan et al., 8 Apr 2025). Aggregate results are reported as very stable across measurement methods: for VICReg and Barlow Twins trained on 300k images, the two-model KNN setup gives DV $\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$0–$\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$1 and DV@20% $\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$2–$\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$3, while DINO is much lower at approximately $\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$4–$\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$5 (Kokhlikyan et al., 8 Apr 2025). For pre-trained open-source SSL models, the same paper reports lower aggregate memorization than similar architectures trained on a subset of the data; for example, VICReg OSS has DV $\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$6 versus $\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$7 on the subset, and DV@20% $\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$8 versus $\mathbf{X}(u,v) = R^o(u,v) + D(u,v)\, R^d(u,v),$9 (Kokhlikyan et al., 8 Apr 2025).

The same efficient framework extends “DéjàView-style” measurement to vision-LLMs by treating the caption as context $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$0 and the detected objects in the paired image as attributes $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$1 (Kokhlikyan et al., 8 Apr 2025). It uses population precision gap and population recall gap, and reports that a one-model test with a pre-trained text embedding model slightly overestimates memorization but is very close at population level: on Shutterstock models, the two-model test gives PPG $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$2 and PRG $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$3, while one-model tests give PPG $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$4 and PRG $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$5, or PPG $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$6 and PRG $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$7 depending on retrieval mode (Kokhlikyan et al., 8 Apr 2025). This directly addresses the misconception that “foreground from background” or “objects from caption” is automatically correlation; both papers insist on separating correlation from memorization.

4. Déjà Vu as a VideoLM query engine

In video analytics, “Déjà Vu” denotes a video-language query engine that accelerates ViT-based VideoLMs by reusing computations across consecutive frames (Hwang et al., 17 Jun 2025). The paper’s systems target is large-scale video query processing, where VideoLMs sample video at 2 FPS and visual embedding extraction dominates end-to-end FLOPs. Its core model, ReuseViT, is a modified ViT that learns token-level inter-frame reuse opportunities rather than relying on fixed hand-designed policies (Hwang et al., 17 Jun 2025).

ReuseViT focuses on reusing QKV projections and FFNs rather than self-attention. The reuse decision for token $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$8 is computed from a concatenation of four cues: similarity $$\mathbf{c}_\theta = (\mathbf{t}_\theta,\, \mathbf{q}_\theta,\, \mathbf{f}_\theta) \in \mathbb{R}^3 \times \mathbb{S}^3 \times \mathbb{R}^2.$$9, token importance $K$0, reference type $K$1, and codec metadata $K$2,

$K$3

with a hard gate

$K$4

A Restoration Layer then calibrates reused outputs via

$K$5

and training uses a Gumbel-Softmax relaxation plus a similarity-and-reuse objective (Hwang et al., 17 Jun 2025).

A central systems result is that FLOP reduction does not directly translate into GPU speedup. Déjà Vu therefore introduces memory-compute joint compaction: layer-wise scheduling, cached memory compaction, and sparse computation compaction implemented as GPU gather-compute-scatter over active tokens (Hwang et al., 17 Jun 2025). In an ablation on video QA at 61% reuse, hard gating alone yields 1.25× speedup, adding sparse computation compaction yields 1.45×, and adding cached memory compaction yields 1.62× (Hwang et al., 17 Jun 2025). This is the paper’s explicit answer to the misconception that algorithmic sparsity automatically becomes wall-clock acceleration.

The evaluation covers three VideoLM tasks: video retrieval with CLIP4Clip on MSR-VTT, video QA with FrozenBiLM on How2QA, and video question grounding with TempCLIP on NExT-GQA (Hwang et al., 17 Jun 2025). Reported embedding-generation speedups reach up to 1.81× for retrieval, 2.64× for video QA, and 2.54× for grounding, all within roughly 2% accuracy degradation; the abstract summarizes this as acceleration by up to 2.64× within a 2% error bound (Hwang et al., 17 Jun 2025). The same work notes that embedding storage is small relative to video size and that caching and reuse make VideoLMs more practical for large-scale video analytics (Hwang et al., 17 Jun 2025).

5. Deja vu as scalable place recognition

An earlier robotics usage appears in “Deja vu: Scalable Place Recognition Using Mutually Supportive Feature Frequencies,” which proposes a place-recognition algorithm inspired by mammalian place, head-direction, and grid cells (Jacobson et al., 2017). The motivating observation is that most place-recognition systems attempt to extract salient features, landmarks, or signatures, with storage and computation growing as the environment increases, whereas biological spatial encoding uses a fixed-size neural encoding framework and, paradoxically, grid cells employ a highly aliased repetitive encoding in which one neuron represents an unbounded number of places in the world (Jacobson et al., 2017).

The algorithmic inversion is to seek out and leverage repetitive, mutually complementary landmark frequencies rather than suppress repetition (Jacobson et al., 2017). The paper states that the combinatorial encoding capacity of multiple different frequencies enables efficient data storage and the potential for sub-linear storage growth in a learning and recall system. It reports experiments on both ground-based and aerial camera datasets, and discusses how the method might scale to arbitrarily large global datasets and dimensions (Jacobson et al., 2017).

This work is important partly because it directly reverses a standard assumption in place recognition. In conventional formulations, aliasing is a failure mode; here repetitive structure becomes the representational substrate. A plausible implication is that the “déjà vu” label is used deliberately to denote repeated structure that is informative rather than confounding.

6. View-driven deduplication and broader conceptual links

A related, though not identically named, “DéjàView idea” appears in view-driven deduplication for visual analytics (Morton et al., 2016). The problem is to produce the cleanest view possible with a limited budget for data labeling, rather than to clean an entire dataset. The formal objective is to minimize

$K$6

subject to a small labeling budget, where $K$7 is the current relational view, $K$8 is the ideal deduplicated view, and the view impact score of a tuple is

$K$9

The paper uses Earth Mover’s Distance as the view distance and combines view impact with classifier uncertainty through

$z_{k+1} = f_\theta(z_k, t_k, t_{k+1}),$0

Candidate pairs are restricted via view provenance and feature-based blocking, and cleaning stops when consecutive view changes stabilize (Morton et al., 2016).

Empirically, experiments on nine different visualizations for two real-world datasets show that the approach produces significantly cleaner views for small labeling budgets than state-of-the-art alternatives and also stops the cleaning process after requesting fewer labels (Morton et al., 2016). On the harder products dataset, all product views are cleaned within fewer than 18 batches, whereas uncertainty-based baselines fail to completely clean any product view within 40 or more batches; on the restaurants dataset, all views are fully cleaned in three batches (Morton et al., 2016).

This usage is not a nominal “DéjàView” system title in the same way as the 2026 reconstruction model, but it is presented as “exactly the ‘DéjàView’ idea” in the sense that cleaning is guided by what is on the screen rather than by global data-quality objectives (Morton et al., 2016). This suggests a broader semantic pattern across the literature: “DéjàView” and “Déjà Vu” repeatedly name methods that exploit partial, repeated, or context-only observations to recover structure that would otherwise require full recomputation, full visibility, or full-data processing.