LENS-DF: Audio DeepFake & Localization Recipe

• LENS-DF is a comprehensive framework that simulates realistic deepfake scenarios by integrating long-form audio, controlled noise, and multi-speaker segments.
• It employs a four-stage pipeline—audio standardization, noise augmentation, long-form construction, and re-segmentation—to generate training data with segment-level labels.
• Empirical results indicate that models trained on LENS-DF achieve robust performance in realistic conditions, overcoming the limitations of short, clean, single-speaker paradigms.

Searching arXiv for the exact topic and closely related papers to ground the article. arXiv search query: LENS-DF ([2507.16220](/papers/2507.16220)) LENS-DF, expanded as Longer duration, Enhanced multi-speaker, Noisy Speech for audio DeepFake detection, is a training and evaluation recipe for audio deepfake detection and temporal localization under realistic acoustic conditions (Liu et al., 22 Jul 2025). It is designed to address three factors that are underrepresented in conventional recipes and datasets: long-form audio, controlled noisy conditions, and multiple speakers within a single sample. In contrast to conventional pipelines centered on short, clean, single-speaker utterances, LENS-DF generates longer clips via concatenation, injects background noise with controllable SNRs, and supports segment-level labels for temporal localization in addition to utterance-level detection (Liu et al., 22 Jul 2025).

1. Definition and design objectives

LENS-DF is formulated as a recipe rather than a single detector architecture. Its purpose is to make training and evaluation conditions better match realistic deployment scenarios in which spoofed content may appear intermittently inside long recordings, under background noise, and across changing speaker identities (Liu et al., 22 Jul 2025). The source corpus is ASVspoof 2019 LA, whose original clips are clean, single-speaker, 2–10 s long, sampled at 16 kHz mono, and whose spoofed utterances are generated using 17 different TTS/VC algorithms (or hybrids) across 107 speakers (Liu et al., 22 Jul 2025).

The central premise is that models trained only on short, clean speech degrade substantially when evaluated on long-form, noisy, multi-speaker conditions (Liu et al., 22 Jul 2025). LENS-DF therefore operationalizes a controlled shift from conventional laboratory conditions to more realistic ones by modifying duration, acoustic corruption, and speaker composition while preserving the original train/dev/eval partitioning of ASVspoof 2019 LA (Liu et al., 22 Jul 2025).

A key feature of the framework is that it treats temporal localization as segment-level classification. Each fixed-length segment is assigned a bonafide or spoof label, enabling localization of spoofed regions without introducing boundary regression, confidence maps, or post-hoc temporal fusion (Liu et al., 22 Jul 2025). This places LENS-DF in continuity with audio-specific localization protocols such as PartialSpoof and ADD, while maintaining a deliberately simple decision structure (Liu et al., 22 Jul 2025).

2. Data generation and long-form construction

The LENS-DF generation pipeline has four explicit stages: audio standardization, noise augmentation, long-form concatenation, and re-segmentation for training efficiency and localization (Liu et al., 22 Jul 2025). Standardization removes leading and trailing silence using LibROSA and applies randomized loudness normalization based on ITU P.56; bonafide/spoof labels remain unchanged (Liu et al., 22 Jul 2025).

Noise augmentation uses MUSAN noise sources drawn from babble, music, and environmental noise (Liu et al., 22 Jul 2025). Each audio segment is assigned either no noise or exactly one noise type, and the noise intensity is uniformly randomized between 0 and 10 dB SNR (Liu et al., 22 Jul 2025). The recipe intentionally excludes room impulse responses and media codecs in the main pipeline so that the induced degradation remains attributable to the MUSAN categories (Liu et al., 22 Jul 2025). The SNR used for analysis is

$$\mathrm{SNR}_{\mathrm{dB}} = 10 \log_{10}\!\left(\frac{\sum_t s(t)^2}{\sum_t n(t)^2}\right).$$

Long-form construction is based on exactly 10 segments per clip (Liu et al., 22 Jul 2025). Fully bonafide samples contain 10 bonafide segments, whereas partially spoofed samples contain 3 bonafide + 7 spoofed segments in randomized order (Liu et al., 22 Jul 2025). Multi-speaker presence arises naturally from concatenating segments from different speakers; there is no speaker overlap or simultaneous speech, only speaker changes across segments (Liu et al., 22 Jul 2025). The resulting long-form clips are typically 15–50 seconds long (Liu et al., 22 Jul 2025).

For localization and training efficiency, each long-form recording is then uniformly cut into fixed-length segments of $N$ seconds, with default $N=4$, and any residual tail audio is discarded (Liu et al., 22 Jul 2025). A segment is labeled spoof if it contains spoofed content; otherwise it is labeled bonafide (Liu et al., 22 Jul 2025). The long-form set is referred to as long, and the re-segmented set as SEG-$N$, with SEG-4 the default (Liu et al., 22 Jul 2025).

Split	long	SEG-4
Train	2,580 bonafide, 22,800 spoofed	17,857 bonafide, 129,805 spoofed
Dev	1,000 bonafide, 1,000 spoofed	5,132 bonafide, 5,640 spoofed
Eval	1,000 bonafide, 1,000 spoofed	4,984 bonafide, 5,663 spoofed

These counts preserve the asymmetry of the underlying spoof-rich training regime while increasing sample count substantially after re-segmentation (Liu et al., 22 Jul 2025).

3. Detection and localization protocol

The reported experiments use a self-supervised learning front-end with a simple back-end (Liu et al., 22 Jul 2025). The front-end is based on Wav2Vec 2.0 encoders from Meta’s Massive Multilingual Speech (MMS) framework, specifically MMS-1B (~965M parameters) and MMS-300M (~317M), pre-trained on ~55k hours covering 1,000+ languages (Liu et al., 22 Jul 2025). The back-end consists of Global Average Pooling (GAP) over frame-level SSL embeddings followed by a single fully connected layer that outputs a scalar logit per segment or utterance (Liu et al., 22 Jul 2025). The SSL front-end and the back-end are fine-tuned jointly (Liu et al., 22 Jul 2025).

If the frame-level SSL features are $h_t \in \mathbb{R}^D$ over $T$ frames, then GAP produces

$\bar{h} = \frac{1}{T}\sum_{t=1}^{T} h_t,$

followed by

$l = w^\top \bar{h} + b,\qquad p=\sigma(l).$

The training objective is binary cross-entropy:

$$L_{\mathrm{CE}} = -\frac{1}{B}\sum_{i=1}^{B}\big[y_i\log p_i + (1-y_i)\log(1-p_i)\big].$$

No additional temporal regularization, smoothing, sequence models, or boundary-regression heads are used (Liu et al., 22 Jul 2025). That architectural minimalism is intentional: LENS-DF is meant to test how far realism in data generation and evaluation protocol can improve performance even with a simple back-end (Liu et al., 22 Jul 2025).

Training uses Adam, dynamic batching with total audio duration per mini-batch capped at 100 seconds, and 16 kHz mono audio throughout (Liu et al., 22 Jul 2025). The learning rate is linearly increased to $1\times10^{-7}$ over the first 80k steps, then linearly decayed to zero by 800k steps, and training stops at 100 epochs or 800k steps, whichever comes first (Liu et al., 22 Jul 2025). Each model is trained on a single NVIDIA TESLA A100 GPU (Liu et al., 22 Jul 2025).

Decision thresholding is based on the development-set EER threshold (Liu et al., 22 Jul 2025). The two principal metrics are Equal Error Rate (EER) and Half Total Error Rate (HTER). The threshold $N$0 satisfies

$N$1

and

$N$2

with $N$3 fixed from the development set (Liu et al., 22 Jul 2025).

4. Empirical performance and ablation findings

The strongest conventional-condition detection result is obtained on 19LA_eval, where MMS-300M trained on 19LA_train achieves 0.15% EER and 0.52% HTER (Liu et al., 22 Jul 2025). Under realistic long-form conditions, the best reported long_eval EER is 0.60%, achieved by MMS-300M trained on SEG-4_train; the best reported long_eval HTER is 1.40%, achieved by MMS-300M trained on long_train (Liu et al., 22 Jul 2025). For localization on SEG-4_eval, the best result is 14.62% EER and 14.08% HTER, again with MMS-300M trained on SEG-4_train (Liu et al., 22 Jul 2025).

The reported cross-condition degradation is one of the paper’s main findings. A model trained on 19LA_train can perform extremely well on 19LA_eval, yet deteriorate markedly on long_eval: for MMS-300M, EER rises from 0.15% to 2.90%, and HTER from 0.52% to 4.05% (Liu et al., 22 Jul 2025). This supports the argument that laboratory-style short, clean, single-speaker training is insufficient for realistic long-form detection (Liu et al., 22 Jul 2025).

The ablation studies isolate the contribution of the realism factors. Controlled MUSAN noise inside LENS-DF is more effective for complex-condition robustness than removing noise or relying only on RawBoost (Liu et al., 22 Jul 2025). With MMS-300M trained on SEG-4_train with RawBoost, evaluation under increasing noise causes both detection and localization to degrade; at noise-free evaluation, detection is reported as ≈0% EER and localization as ≈5% EER (Liu et al., 22 Jul 2025). This suggests that noise robustness remains a dominant unresolved variable.

Speaker composition is especially consequential. In the single-speaker vs multi-speaker study with MMS-300M, long-form detection yields 7.90% EER for single/single and 11.10% EER for multi/single, whereas multi-speaker training/evaluation yields 0.60–1.30% EER (Liu et al., 22 Jul 2025). The authors interpret this as evidence that single-speaker training fosters shortcut learning based on speaker identity (Liu et al., 22 Jul 2025). Localization remains difficult in all speaker configurations, with the best result 13.68% EER under multi/multi (Liu et al., 22 Jul 2025).

Segment duration also matters. Varying evaluation segment length from 0.01 s to 4 s shows that extremely short segments yield poor localization, while increasing segment length consistently improves localization (Liu et al., 22 Jul 2025). This implies that the current GAP-plus-FC architecture benefits from longer temporal context for segment-level decisions, even though the framework itself makes no use of sequence modeling (Liu et al., 22 Jul 2025).

For the adapted LAV-DF comparison, LENS-DF-trained models are evaluated at 0.04 s resolution after binarization using IoU-based AP and AR (Liu et al., 22 Jul 2025). Reported values are [email protected]/0.5 = 100.00 / 33.50 for LENS-DF versus 99.32 / 29.28 for LAV-DF, and AR@100/50/20/10 = 38.56 / 37.52 / 36.61 / 35.70 for LENS-DF versus 27.77 / 26.27 / 24.63 / 23.73 for LAV-DF (Liu et al., 22 Jul 2025). The paper treats this as evidence that the LENS-DF recipe is useful even when compared against an audio-visual localization benchmark (Liu et al., 22 Jul 2025).

5. Interpretation, misconceptions, and limitations

Several points are easy to misconstrue. First, LENS-DF is not a specialized temporal localization network. Localization is defined purely as fixed-window classification, and the method explicitly uses no boundary regression, no confidence map, and no post-hoc fusion across segments (Liu et al., 22 Jul 2025). Second, “multi-speaker” does not mean overlapping speech or speaker diarization complexity; speakers change across concatenated segments, not simultaneously within the same segment (Liu et al., 22 Jul 2025).

A third common misconception is that robustness is obtained mainly from a stronger back-end. The reported experiments instead emphasize that a simple SSL front-end + GAP + FC pipeline can benefit substantially when trained on LENS-DF variants, indicating that the realism of the generation protocol is itself a primary variable (Liu et al., 22 Jul 2025). This suggests that benchmark construction and evaluation design are at least as consequential as classifier complexity for long-form audio deepfake detection.

The limitations are explicit. Localization performance remains weak relative to utterance-level detection, with best SEG-4_eval performance around 14–15% EER (Liu et al., 22 Jul 2025). Noise causes sharp degradation as SNR falls, and the paper identifies noise as a dominant challenge (Liu et al., 22 Jul 2025). Single-speaker training encourages shortcut reliance on speaker identity, while ultra-short segments perform poorly (Liu et al., 22 Jul 2025). The future directions proposed in the paper include boundary-aware models or sequence labeling, methods adapted to multi-speaker, multi-language, and noisy scenarios, and possibly the incorporation of additional modalities or richer supervision (Liu et al., 22 Jul 2025).

6. Reproducibility and terminological context

LENS-DF is presented as a reproducible framework. The data-generation code and sample data are available at https://github.com/nii-yamagishilab/LENS-DF-DataGen, the SSL checkpoints are provided for MMS-1B and MMS-300M, and the RawBoost implementation used in ablation is also public (Liu et al., 22 Jul 2025). The operational pipeline is straightforward: generate long-form noisy multi-speaker audio, optionally re-segment it into fixed windows, train the SSL-plus-linear detector, then evaluate utterance-level detection or segment-level localization using the development-set EER threshold (Liu et al., 22 Jul 2025).

Outside this audio-deepfake context, the string “LENS-DF” is not used consistently across arXiv. In other papers represented here, it is interpreted as a lensing deflection field in Hubble Frontier Fields analysis (Diego, 2014), a deep learning-based lens detection framework for strong gravitational lens finding (Lanusse et al., 2017), a lens distortion field in differentiable camera modeling (Xian et al., 2023), a lens blur field understood as a practical instantiation of a lens defocus/depth field (Lin et al., 2023), a differentiable lens design framework for object detection (Côté et al., 2022), and an end-to-end DESI Strong Lens Foundry workflow for discovery, confirmation, and modeling of strong gravitational lenses (Huang et al., 5 Feb 2025). This suggests that the acronym is context-dependent across the literature, whereas “LENS-DF” in the strict sense of a named method refers to the 2025 audio recipe for long-form, noisy, multi-speaker deepfake detection and temporal localization (Liu et al., 22 Jul 2025).