ReConFuse: Reconstruction-Error Guided Semantic Fusion for AI-Generated Video Detection

Abstract: AI-generated videos are becoming increasingly realistic, raising serious concerns about misinformation, content authenticity, and media trust. Reliable AI-generated video detection is therefore essential for multimedia forensics, yet remains challenging due to the need to capture spatial artifacts, temporal dynamics, and generalize to evolving generative models. In this paper, we explore reconstruction error as a discriminative forensic cue for AI-generated video detection. By reconstructing input videos with a pretrained WF-VAE, we observe that real and generated videos exhibit distinguishable frame-wise reconstruction error patterns, suggesting that reconstruction errors can reveal their distributional discrepancies. However, extending reconstruction-based image detection to videos is non-trivial, since video reconstruction errors are temporally organized across frames and require semantic context for effective interpretation. To address these challenges, we propose ReConFuse, a reconstruction-guided semantic fusion framework for video-level AI-generated video detection. ReConFuse extracts reconstruction error cues from WF-VAE reconstructed videos, aligns them with multi-frame semantic features, and uses a Mamba-based module to model temporal evolution for video-level classification. Experiments across multiple generators and evaluation settings demonstrate the effectiveness and strong generalization ability of ReConFuse.

• The paper introduces ReConFuse, a video detector that combines signed WF-VAE reconstruction errors, XCLIP semantic features, gated patch-level fusion, and Mamba temporal modeling to identify generative artifacts.
• The method achieves 91.49% F1 and 95.82% AUROC on GenVideo many-to-many testing, 96.66% accuracy and 99.37% AUROC on GenBuster, and more than 13 F1 points over DeMamba in GenVideo one-to-many evaluation.
• Ablations show that semantic conditioning and temporal modeling are essential—error-only detection reaches just 43.65% accuracy, while signed errors and gated fusion substantially outperform magnitude-only errors and XCLIP alone, despite weak performance on Sora content.

Motivation and problem setting

ReConFuse addresses AI-generated video detection by introducing reconstruction error as a forensic cue at the video level. The authors observe that when input videos are passed through the encoder–decoder reconstruction path of a pretrained WF-VAE, real videos and videos produced by generative models (e.g., Zhipu Qingyan) exhibit distinguishable frame-wise reconstruction-error patterns. This observation extends a line of work established for image-level detection—DIRE, AEROBLADE, LaRE², and FIRE—to video, where such cues had remained underexplored.

The extension is non-trivial for two reasons stated explicitly in the paper. First, video reconstruction errors are temporally organized across frames, so independent frame-level modeling or naive frame aggregation is insufficient. Second, reconstruction errors are ambiguous without semantic context: large errors can arise from motion blur, compression artifacts, illumination changes, or complex textures rather than generative artifacts. ReConFuse is designed around both challenges.

Method

The framework has four stages:

1. Reconstruction error extraction: An input video $V=\{x_t\}_{t=1}^T$ is encoded and decoded through pretrained WF-VAE to obtain $\hat{V}$, and signed per-frame errors $e_t = x_t - \hat{x}_t$ are computed. Signed errors are used deliberately; ablations show they substantially outperform magnitude-only formulations.
2. Patch-level tokenization: Each error map is resized to the XCLIP visual encoder's input resolution and embedded via a lightweight patch embedding layer, producing dense reconstruction-error tokens that preserve localized anomalies (texture distortion, boundary inconsistency).
3. Gated semantic-error fusion: XCLIP patch-level semantic tokens are projected into a shared dimension with the error tokens, and a sigmoid gate $$a_t^{(i)} = \sigma(W_a[\tilde{s}_t^{(i)};\tilde{p}_t^{(i)}])$$ controls how much error evidence is injected: $$f_t^{(i)} = \tilde{s}_t^{(i)} + a_t^{(i)} \odot \tilde{p}_t^{(i)}$$. This lets the model suppress content-dependent reconstruction difficulty while emphasizing generation-related discrepancies on the same patch grid.
4. Temporal modeling and prediction: Fused tokens pass through a Mamba-based sequence module producing a video-level representation, which is concatenated with global semantic context and classified with binary cross-entropy loss.

Notably, no diffusion sampling or inversion is performed—the WF-VAE acts purely as an external reconstruction prior, keeping inference cheap relative to full diffusion-based methods like DIRE.

Experimental results

Evaluation covers GenVideo (one-to-many and many-to-many protocols) and GenBuster (eight commercial generators), against DeMamba, DIRE, AEROBLADE, and D3.

GenVideo one-to-many (trained on Kinetics-400 reals + Pika fakes): ReConFuse achieves 85.56% average Accuracy and 85.03% average F1, versus 79.56% / 71.75% for DeMamba—a gain of over 13 points in F1. D3 attains higher Recall (90.30%) but much lower Accuracy (62.03%), reflecting its recall-oriented bias; DIRE retains the best AUROC (93.02% vs. 92.72%).

GenVideo many-to-many: ReConFuse leads on all four averaged metrics—93.44% Recall, 91.93% Accuracy, 91.49% F1, 95.82% AUROC—with perfect recall on several generators (Moonway, Craft). The Sora subset remains difficult for all trained detectors (55–56% recall), which the paper does not analyze in depth.

GenBuster: ReConFuse achieves the best averages (96.66% Accuracy, 91.23% F1, 99.37% AUROC), though margins over DeMamba are modest (+0.83 F1, +0.12 AUROC), indicating that on this benchmark most of the discriminative power already resides in the DeMamba-style baseline.

Ablation findings

The component ablation yields two strong claims worth highlighting:

• Error-only detection fails: using reconstruction errors without semantics yields 43.65% Accuracy—below chance—confirming that raw reconstruction discrepancy is not directly discriminative and requires semantic interpretation. This is the paper's clearest evidence for the fusion design.
• Each component matters: removing gated fusion drops Accuracy from 88.33% to 60.11%; replacing Mamba with mean pooling drops it to 68.55%. The full model reaches 88.33% Accuracy / 88.18% F1 / 95.15% AUROC versus 79.56% / 77.14% / 82.78% for XCLIP alone.

On error formulation, signed error (88.33% Acc) decisively outperforms absolute error (62.33%), squared error (81.11%), and the XCLIP-only baseline, supporting the claim that the direction of reconstruction deviation carries forensic information discarded by magnitude-only measures. On temporal modeling, Mamba achieves the best Accuracy/F1 among tested modules (mean pooling, Transformer, ResNet50, LSTM, 1D CNN) at moderate parameter count (129M), though mean pooling attains higher AUROC (96.17%). A prior comparison across Wan 2.2 VAE, CogVideoX VAE, and SD1.5 VAE motivates WF-VAE: video VAEs produce temporally coherent error responses, whereas the image-level SD1.5 VAE yields texture-dominated frame-wise residuals.

Limitations and open questions

Several caveats bear directly on the reported results. First, the method requires benchmark-specific training (like DeMamba and DIRE), so its generalization claims rest on cross-generator evaluation rather than training-free applicability; the comparison with training-free D3 and AEROBLADE is therefore asymmetric. Second, performance on Sora-generated content degrades sharply under both GenVideo protocols (55.36% recall in many-to-many), leaving open whether reconstruction cues from WF-VAE transfer to generators whose latent spaces differ substantially from the reconstruction prior. Third, the choice of WF-VAE as the reconstruction prior is justified empirically by qualitative error-map comparisons rather than a systematic quantitative study across priors. Fourth, AUROC results are mixed—DIRE and mean pooling variants exceed ReConFuse on some settings—so the advantage is specific to thresholded metrics rather than universal ranking quality. Finally, robustness to perturbations common in deployment (compression, resizing, adversarial attack) is not evaluated.

Conclusion

ReConFuse demonstrates that reconstruction error from a pretrained video VAE, when densely tokenized, semantically gated against XCLIP features, and temporally aggregated with Mamba, constitutes an effective forensic signal for AI-generated video detection. The central empirical finding—that signed, semantically contextualized reconstruction errors improve F1 by over 13 points over a strong baseline on GenVideo one-to-many—supports reconstruction-guided fusion as a viable direction, while the Sora subset weakness and reliance on supervised training delineate the boundaries of the current result.