WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection (2101.01456v2)

Abstract: In recent years, the abuse of a face swap technique called deepfake has raised enormous public concerns. So far, a large number of deepfake videos (known as "deepfakes") have been crafted and uploaded to the internet, calling for effective countermeasures. One promising countermeasure against deepfakes is deepfake detection. Several deepfake datasets have been released to support the training and testing of deepfake detectors, such as DeepfakeDetection and FaceForensics++. While this has greatly advanced deepfake detection, most of the real videos in these datasets are filmed with a few volunteer actors in limited scenes, and the fake videos are crafted by researchers using a few popular deepfake softwares. Detectors developed on these datasets may become less effective against real-world deepfakes on the internet. To better support detection against real-world deepfakes, in this paper, we introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. WildDeepfake is a small dataset that can be used, in addition to existing datasets, to develop and test the effectiveness of deepfake detectors against real-world deepfakes. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically. We also propose two (eg. 2D and 3D) Attention-based Deepfake Detection Networks (ADDNets) to leverage the attention masks on real/fake faces for improved detection. We empirically verify the effectiveness of ADDNets on both existing datasets and WildDeepfake. The dataset is available at: https://github.com/OpenTAI/wild-deepfake.

An Evaluation of "WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection"

The paper "WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection" presents a focused investigation into the development and effectiveness of deepfake detection systems within the context of real-world applications. This work introduces the WildDeepfake dataset, thereby addressing the limitations of existing datasets that predominantly consist of artificially constructed deepfakes, which may not fully encapsulate the complexity encountered in genuine internet-sourced deepfakes.

Dataset Characteristics

The WildDeepfake dataset is a noteworthy contribution in the field of deepfake detection research. It comprises 7,314 sequences extracted from 707 videos, meticulously gathered from online platforms. This collection process emphasizes the diversity and authenticity of video content, capturing variations in scenes, faces, activities, and other contextual factors that typify actual deepfakes found online. The dataset's composition is distinctly different from prior datasets where deepfakes were mostly generated using limited techniques and crafted in controlled environments, lacking the intra-class variance evident in wild deepfakes.

Critique of Existing Datasets

The authors provide a comprehensive critique of existing datasets, such as DeepfakeDetection and FaceForensics++, arguing that these datasets have significant constraints due to their reliance on synthetic deepfakes crafted in limited settings. They argue that this approach fails to mirror the vast diversity of deepfakes proliferating online, potentially hindering the generalizability of models trained on such datasets.

Attention-Based Detection Networks

To address the challenges of detecting increasingly sophisticated wild deepfakes, the authors propose two novel detection networks: the 2D and 3D Attention-based Deepfake Detection Networks (ADDNets). These models utilize attention masks derived from facial landmarks to enhance feature extraction during the detection process. This technique focuses on critical facial regions such as eyes, nose, and mouth to improve the network's sensitivity to subtle manipulations characteristic of deepfake videos.

Experimental Evaluation

The paper conducts an extensive experimental evaluation comparing ADDNets against several established baseline models, including XceptionNet and various CNN and 3D CNN architectures. Results indicate that while ADDNet-2D performs robustly across all datasets, including the challenging WildDeepfake, sequence-level detection with ADDNet-3D showed potential but requires further refinement to handle temporal inconsistencies in wild deepfake sequences. Despite these promising findings, the nuanced difficulties presented by real-world deepfakes remain apparent, as detection accuracies drop significantly when applied to the WildDeepfake dataset.

Significance and Implications

The introduction of the WildDeepfake dataset manifests an important step in the ongoing effort to enhance the adaptability and resilience of deepfake detection models. The application of real-world data pushes researchers to develop more sophisticated algorithms that can effectively navigate the complex and diverse landscape of deepfakes encountered on the internet. By bridging the gap between controlled synthetic datasets and real-world applications, this research paves the way for designing detection systems capable of maintaining high efficacy in operational environments.

Future Directions

Future research could extend the work by exploring novel architectures that further integrate temporal dynamics and contextual cues, improving the robustness of sequence-level detection networks against wild deepfakes. Additionally, expanding the WildDeepfake dataset to include even broader scenarios would enrich training data diversity, potentially leading to more generalized detection solutions.

In summary, "WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection" makes significant strides towards achieving effective real-world deepfake detection, underscoring the necessity of data-driven approaches tailored to the internet's vast and varied multimedia landscape.