Learning Expressive And Generalizable Motion Features For Face Forgery Detection (2403.05172v1)
Abstract: Previous face forgery detection methods mainly focus on appearance features, which may be easily attacked by sophisticated manipulation. Considering the majority of current face manipulation methods generate fake faces based on a single frame, which do not take frame consistency and coordination into consideration, artifacts on frame sequences are more effective for face forgery detection. However, current sequence-based face forgery detection methods use general video classification networks directly, which discard the special and discriminative motion information for face manipulation detection. To this end, we propose an effective sequence-based forgery detection framework based on an existing video classification method. To make the motion features more expressive for manipulation detection, we propose an alternative motion consistency block instead of the original motion features module. To make the learned features more generalizable, we propose an auxiliary anomaly detection block. With these two specially designed improvements, we make a general video classification network achieve promising results on three popular face forgery datasets.
- “Faceforensics++: Learning to detect manipulated facial images,” in ICCV, 2019, pp. 1–11.
- “Deeperforensics-1.0: A large-scale dataset for real-world face forgery detection,” in CVPR, 2020, pp. 2889–2898.
- “Celeb-df: A large-scale challenging dataset for deepfake forensics,” in CVPR, 2020, pp. 3207–3216.
- “Domain general face forgery detection by learning to weight,” in AAAI, 2021, vol. 35, pp. 2638–2646.
- François Chollet, “Xception: Deep learning with depthwise separable convolutions,” in CVPR, 2017, pp. 1251–1258.
- “Multi-attentional deepfake detection,” in CVPR, June 2021, pp. 2185–2194.
- “Thinking in frequency: Face forgery detection by mining frequency-aware clues,” in ECCV. Springer, 2020, pp. 86–103.
- “Local relation learning for face forgery detection,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2021, vol. 35, pp. 1081–1088.
- “Detecting and simulating artifacts in gan fake images,” in 2019 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, 2019, pp. 1–6.
- “Deepfake video detection using recurrent neural networks,” in proceedings of the IEEE international conference on advanced video and signal based surveillance. IEEE, 2018, pp. 1–6.
- “Deepfake detection using spatiotemporal convolutional networks,” CoRR, vol. abs/2006.14749, 2020.
- “Spatio-temporal features for generalized detection of deepfake videos,” CoRR, vol. abs/2010.11844, 2020.
- “Wilddeepfake: A challenging real-world dataset for deepfake detection,” in MM ’20: The 28th ACM International Conference on Multimedia, 2020, 2020, pp. 2382–2390.
- “Stm: Spatiotemporal and motion encoding for action recognition,” in ICCV, 2019, pp. 2000–2009.
- “Face x-ray for more general face forgery detection,” in CVPR, 2020, pp. 5001–5010.
- “Quo vadis, action recognition? a new model and the kinetics dataset,” in CVPR, 2017, pp. 6299–6308.
- “Learning spatiotemporal features with 3d convolutional networks,” in ICCV, 2015, pp. 4489–4497.
- “A closer look at spatiotemporal convolutions for action recognition,” in CVPR, 2018, pp. 6450–6459.
- “Temporal segment networks: Towards good practices for deep action recognition,” in ECCV. Springer, 2016, pp. 20–36.
- Davis E King, “Dlib-ml: A machine learning toolkit,” The Journal of Machine Learning Research, vol. 10, pp. 1755–1758, 2009.
- “Efficientnet: Rethinking model scaling for convolutional neural networks,” in International Conference on Machine Learning. PMLR, 2019, pp. 6105–6114.
- “End-to-end reconstruction-classification learning for face forgery detection,” in CVPR, 2022, pp. 4113–4122.
- “Two-branch recurrent network for isolating deepfakes in videos,” in ECCV. Springer, 2020, pp. 667–684.