EA-Swin: An Embedding-Agnostic Swin Transformer for AI-Generated Video Detection

Abstract: Recent advances in foundation video generators such as Sora2, Veo3, and other commercial systems have produced highly realistic synthetic videos, exposing the limitations of existing detection methods that rely on shallow embedding trajectories, image-based adaptation, or computationally heavy MLLMs. We propose EA-Swin, an Embedding-Agnostic Swin Transformer that models spatiotemporal dependencies directly on pretrained video embeddings via a factorized windowed attention design, making it compatible with generic ViT-style patch-based encoders. Alongside the model, we construct the EA-Video dataset, a benchmark dataset comprising 130K videos that integrates newly collected samples with curated existing datasets, covering diverse commercial and open-source generators and including unseen-generator splits for rigorous cross-distribution evaluation. Extensive experiments show that EA-Swin achieves 0.97-0.99 accuracy across major generators, outperforming prior SoTA methods (typically 0.8-0.9) by a margin of 5-20%, while maintaining strong generalization to unseen distributions, establishing a scalable and robust solution for modern AI-generated video detection.

• The paper presents EA-Swin, an embedding-agnostic spatiotemporal transformer that distinguishes synthetic from real videos using patch token representations.
• It employs alternating temporal and spatial attention blocks with sliding window shifts, achieving 0.97–0.99 accuracy and 0.999 AUC on challenging benchmarks.
• Extensive evaluations on the balanced EA-Video benchmark show robust generalization across unseen generators, validating its practical forensic and moderation applications.

Embedding-Agnostic Spatiotemporal Modeling for AI-Generated Video Detection

Motivation and Problem Setting

The proliferation of high-fidelity AI-generated video content from foundation models such as Sora-2 and Veo-3 necessitates forensic solutions capable of distinguishing synthetic from real videos with high reliability, especially as recent generators produce outputs nearly indistinguishable from real-world footage (Figure 1).

Figure 1: Synthetic videos from Sora2 and Veo3 exhibit photorealistic qualities and realistic motion, surpassing earlier models with evident spatial/temporal artifacts.

Prior detection paradigms—such as frame-level image-based adaptation, shallow embedding trajectory analysis, or multimodal LLMs—are insufficient for modern AI-generated video due to a lack of temporal structure exploitation, limited generalization, or prohibitive inference cost. Existing benchmarks are rapidly rendered obsolete by advances in synthesis, with commercial models often underrepresented.

EA-Swin Architecture

Design Principles

EA-Swin introduces a unified spatiotemporal transformer architecture, embedding-agnostic by operating directly on patch token representations extracted from frozen ViT-style encoders. Architectural efficiency is achieved via factorized windowed attention, alternating temporal and spatial blocks rather than quadratic joint attention. This enables seamless compatibility with vision encoder outputs and reduces computational overhead.

Spatiotemporal Windowed Attention

The input video is partitioned along spatial and temporal axes, forming local windows for efficient self-attention. Subsequent layers shift these windows, facilitating cross-window interaction for both spatial adjacency and temporal continuity, as shown in Figure 2.

Figure 2: Spatiotemporal window shifting enables boundary token interaction across local regions and frames, providing enhanced global context modeling.

Temporal attention is applied independently for each spatial token trajectory with learnable relative positional bias and cyclic window shifts, maintaining linear complexity in sequence length. Spatial attention then aggregates features across the patch grid per frame, again alternating shift/no-shift to bridge window boundaries. Non-geometric layouts revert to 1D attention, ensuring robustness.

Detection head composition integrates $D_t$ temporal blocks and $D_s$ spatial blocks, each comprising windowed multi-head self-attention, MLP, and layer normalization, followed by video-level aggregation and a lightweight classifier.

EA-Video Dataset

Dataset Composition and Diversity

The EA-Video benchmark comprises 130K videos, balanced between real and AI-generated samples. It pools videos from recent and legacy generators—including Veo3, Sora2, Hunyuan, CogVideoX, EasyAnimate, and numerous commercial/open platforms—and real video sources such as PEVideo, DVSC, and synthetic game artifacts (Figure 3).

Figure 3: Distribution of real and AI video content across generator platforms and dataset sources.

Generator splits ensure rigorous cross-distribution evaluation, with unseen-generator test sets formed from generators not represented during training, specifically addressing generalization requirements as synthesis technology evolves.

Experimental Results

Benchmark Evaluation

EA-Swin achieves 0.97–0.99 accuracy and 0.999 AUC across major generators in both seen and unseen splits, quantitatively outperforming prior models (e.g., DeMamba, NPR, WaveRep, ResTraV) by 5–20% margin in accuracy and recall on challenging benchmarks.

Previously dominant methods collapse near-random on modern content, especially embedding-trajectory (D3, ResTraV), whereas structured spatiotemporal modeling in DeMamba remains competitive but has higher complexity. EA-Swin maintains high precision and recall across both seen and unseen generators, demonstrating robust generalization.

Ablation Studies

Incremental ablations reveal that removing window shifting, factorized attention, attention pooling, or swapping for MLP baselines each results in measurable performance degradation, confirming the impact of model design choices (Figure 4).

Figure 4: Architecture ablation illustrates the necessity of shifted windows, factorized attention, and adaptive pooling for optimal recall and discrimination.

Encoder Analysis and Temporal Robustness

Evaluation using different vision backbones (V-JEPA2, CLIP, DINOv3, DINOv2) substantiates V-JEPA2 as the most discriminative and generalizable, attributable to strong self-supervised spatiotemporal feature learning. Reduction in sampled frames marginally decreases recall and F1, but overall robustness remains high, indicating temporal aggregation stability.

Figure 5: Video frame samples underscore the qualitative diversity in EA-Video benchmark, spanning both real and synthetic distributions.

Implications and Outlook

EA-Swin sets a practical standard for scalable and rigorous AI-generated video detection, leveraging embedding-agnostic hierarchical spatiotemporal modeling to generalize across generator distributions and rapid synthesis advancements. The approach’s compatibility with patch-based encoders and its computational efficiency make it suitable for real-world deployment in both forensic and moderation settings.

Theoretical implications include evidence that explicit spatiotemporal modeling—using factorized windowed attention and boundary-shifting mechanisms—remains essential for discriminating generative artifacts even as synthetic content converges towards real distributions. Practically, the architecture’s robustness across unseen generators suggests its utility for situations where detector retraining is cost-prohibitive.

Future directions include lightweight variant exploration, dynamic attention allocation, broader application to dense tasks and cross-modal domains, and further leveraging self-supervised/foundation-model pretraining for improved scalability and fine-grained video understanding.

Conclusion

EA-Swin advances AI-generated video detection by integrating factorized, embedding-agnostic Swin Transformer heads atop generic vision encoders, attaining superior accuracy, recall, and generalization across diverse generator distributions. Dataset construction and extensive evaluation validate its effectiveness and practical viability. Architectural refinements, as demonstrated in ablation, are critical for robust discrimination; the framework provides a scalable reference as video synthesis fidelity continues to accelerate (2602.17260).