TDAM-AvgPool: Temporal Diff in Deepfake Speech

• The paper introduces a novel framework that detects partial deepfake speech by modeling unnatural temporal changes using a temporal difference attention module.
• It combines a dual-level TDAM with adaptive average pooling to convert variable-length speech embeddings into fixed-length representations while preserving essential temporal patterns.
• Empirical results demonstrate state-of-the-art performance with EERs as low as 0.59% on PartialSpoof and 0.03% on HAD, highlighting its efficacy.

Searching arXiv for the cited TDAM-AvgPool paper and closely related usages of “TDAM” to support the article. TDAM-AvgPool is a frame-level temporal difference learning framework for partial deepfake speech detection that treats synthetic speech as exhibiting abnormal temporal dynamics rather than merely anomalous content. It is designed for the setting in which only some segments of an utterance are manipulated, and it avoids frame-level supervision by learning from utterance-level labels only. The method combines a Temporal Difference Attention Module (TDAM), which emphasizes suspicious temporal variation, with adaptive average pooling, which converts variable-length self-supervised speech embeddings into a fixed-length representation while preserving essential temporal patterns. In reported experiments, the model achieves state-of-the-art performance on PartialSpoof and HAD, including an EER of 0.59% on the PartialSpoof evaluation set and 0.03% on HAD (Li et al., 20 Jul 2025).

1. Problem formulation and temporal-difference premise

TDAM-AvgPool is introduced for partial deepfake speech detection, a regime in which manipulated and genuine segments coexist within the same utterance. This setting is more difficult than fully synthetic speech detection because the detector must remain sensitive to short fake spans without being dominated by long bonafide regions. A central motivation of the method is that earlier approaches often rely on frame-level or segment-level annotations during training, and these are described as costly and time-consuming to obtain at scale, hard to define precisely when fake segments are short or boundaries are ambiguous, and brittle when deepfake generation methods smooth out the transitions between bonafide and fake regions (Li et al., 20 Jul 2025).

The method therefore reframes detection: instead of locating explicit fake boundaries, it models deepfake speech as containing erratic directional changes and unnatural local transitions in feature space. Given frame-wise embeddings

$\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_T,$

the normalized direction vectors between consecutive frames are defined as

$$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$

Local directional consistency is then measured through cosine similarity between adjacent directional vectors:

$$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$

The reported empirical pattern is that bonafide speech has higher utterance-level mean $\mu$ and lower standard deviation $\sigma$, whereas deepfake speech has lower $\mu$ and higher $\sigma$. This observation motivates using directional irregularity as the core detection cue (Li et al., 20 Jul 2025).

2. Architectural composition of TDAM-AvgPool

The TDAM-AvgPool pipeline begins with wav2vec2-XLSR as the self-supervised feature extractor. The resulting variable-length embeddings are regularized and length-normalized by adaptive average pooling, then refined by fully connected layers and a 1D ResNet block. A temporal difference map is constructed from the refined sequence, and TDAM converts that map into a frame-sensitive attention signal that reweights the utterance representation before classification (Li et al., 20 Jul 2025).

Let the refined embedding be

$\mathbf{E} \in \mathbb{R}^{T' \times D},$

where $T'$ is the pooled temporal length and $D$ is the embedding dimension. Before differences are computed, the model applies a 1D convolution across time to obtain a contextualized embedding $$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$0. The directional difference map is then defined by

$$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$1

A zero vector is appended at the end so that the shape remains consistent. This construction explicitly encodes frame-to-frame temporal change rather than raw content (Li et al., 20 Jul 2025).

TDAM itself uses a dual-level hierarchical difference representation. At the first level, $$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$2 is reshaped into a time-spatial map and processed by a 2D convolution that expands the channel dimension from 1 to 32, producing

$$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$3

This representation captures fine local artifacts such as micro-prosodic inconsistencies and abrupt local changes. At the second level, $$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$4 is downsampled to 4 channels, passed through another convolution with a larger receptive field, and upsampled back to 32 channels to yield

$$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$5

The two scales are fused into confidence weights through

$$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$6

where $$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$7 is a $$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$8 2D convolution, $$\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.$$9 is the sigmoid function, and $$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$0 denotes element-wise multiplication. A fully connected layer with softmax is then applied to $$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$1, and frame predictions are averaged across time to obtain the utterance-level decision (Li et al., 20 Jul 2025).

3. Adaptive average pooling and the “AvgPool” designation

The “AvgPool” component refers to adaptive average pooling, which is used to handle variable-length utterances without discarding potentially informative regions. If the self-supervised extractor outputs

$$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$2

the model partitions the $$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$3 frames into $$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$4 segments and computes

$$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$5

where $$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$6 is the set of frame indices in segment $$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$7, $$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$8 is the number of frames in that segment, and $$\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.$$9 is the fixed-length pooled sequence. If $\mu$0, zero-padding is used (Li et al., 20 Jul 2025).

The rationale given for average pooling is specific. It converts variable-length utterances into a fixed-length representation, preserves global temporal structure better than truncation, avoids discarding potentially fake regions, is less biased toward extreme peaks than max pooling, and retains both strong and weak temporal cues. The reported ablation results support this choice directly: TDAM-AvgPool outperforms both TDAM-Trim-Pad and TDAM-MaxPool on the PartialSpoof evaluation set (Li et al., 20 Jul 2025).

This design is consequential because the method is not based on isolated transient spikes alone. Its detection signal depends on how directional changes evolve across the utterance. Average pooling therefore acts less as a generic compression operator than as a mechanism for preserving temporal variation patterns in a standardized sequence length. A plausible implication is that the pooling stage is structurally aligned with the method’s emphasis on distributed temporal irregularity rather than only boundary-local artifacts.

4. Supervision regime, optimization, and decision rule

TDAM-AvgPool is trained using utterance-level labels only. This is central to the method’s practical positioning: it does not require explicit supervision for where fake speech begins or ends, yet it still produces frame-sensitive attention through the temporal-difference pathway (Li et al., 20 Jul 2025).

The training objective is cross-entropy loss with class weighting to address imbalance. The reported class weights are 9 for bonafide and 1 for partial deepfakes. Optimization uses Adam with learning rate $\mu$1, weight decay $\mu$2, batch size 2, and 10 epochs (Li et al., 20 Jul 2025).

At inference time, the model produces frame-level predictions and then averages them over time for the final utterance-level score. This architecture distinguishes between two roles: TDAM generates confidence-weighted frame emphasis, while the final averaging step converts that frame-sensitive representation into a global decision. The method therefore remains weakly supervised at the label level while still being temporally localized in its internal representation (Li et al., 20 Jul 2025).

5. Empirical performance and ablation evidence

The reported evaluation uses PartialSpoof (PS) and HAD (Half-Truth), with additional cross-dataset comparison involving ASVspoof 2019 LA. The performance metrics are EER on PartialSpoof and EER and AUC on HAD. On PartialSpoof, TDAM-AvgPool achieves 0.19% EER on the development set and 0.59% EER on the evaluation set. On HAD, it achieves 0.03% EER and 99.99% AUC. The comparison tables place these results ahead of the listed prior systems, including 0.64% EER by Zhang et al. on PartialSpoof evaluation and 0.06% EER by Cai et al. on HAD (Li et al., 20 Jul 2025).

The ablation results are especially informative because they isolate the contributions of temporal differences, hierarchical modeling, directional information, and pooling choice. On the PartialSpoof evaluation set, the following results are reported:

Variant	PS evaluation EER
w/o TDAM	0.87%
w/o $\mu$3 and $\mu$4	0.74%
w/o $\mu$5	0.80%
w/o directional information	0.76%
TDAM-Trim-Pad	1.03%
TDAM-MaxPool	1.00%
TDAM-AvgPool	0.59%

These numbers support several conclusions stated in the paper. The full TDAM is essential; both fine- and coarse-scale difference representations contribute; directional information is not redundant; and adaptive average pooling is more effective than trimming-padding or max pooling for this task (Li et al., 20 Jul 2025).

The accompanying visual analysis is consistent with these results: the model highlights not only transition boundaries but also internal fake regions, despite being trained only with utterance-level labels. This indicates that the learned representation is sensitive to broader temporal inconsistency rather than narrowly to boundary artifacts alone (Li et al., 20 Jul 2025).

6. Scope, related terminology, and common confusions

Within the cited arXiv literature, the explicit use of TDAM-AvgPool as a model name corresponds to the partial deepfake speech detector described in “Frame-level Temporal Difference Learning for Partial Deepfake Speech Detection” (Li et al., 20 Jul 2025). This specificity matters because the acronym TDAM is used in unrelated ways in other domains.

In “DeU-Net: Deformable U-Net for 3D Cardiac MRI Video Segmentation,” TDAM denotes a Temporal Deformable Aggregation Module for deformable temporal fusion in cardiac MRI video segmentation, and the paper explicitly states that it does not describe a TDAM-AvgPool variant or use average pooling as TDAM’s temporal aggregation mechanism (Dong et al., 2020). In “Temporal Pooling Strategies for Training-Free Anomalous Sound Detection with Self-Supervised Audio Embeddings,” the paper explicitly states that TDAM-AvgPool is not mentioned and that the closest correspondence is simply temporal mean pooling rather than a separate TDAM-AvgPool method (Wilkinghoff et al., 4 Mar 2026).

A common misconception is therefore to treat TDAM-AvgPool as a generic name for any temporal average pooling module. The cited literature does not support that reading. In the deepfake-speech context, the term refers to a particular combination of temporal-difference modeling, hierarchical attention, and adaptive average pooling (Li et al., 20 Jul 2025). This suggests that TDAM-AvgPool is best understood as an application-specific architecture rather than a general-purpose pooling family.

7. Methodological significance

TDAM-AvgPool differs from prior partial-deepfake approaches in three stated respects. First, it does not depend on transition artifacts remaining visibly detectable; instead, it targets temporal inconsistency throughout the utterance, including within fake regions. Second, it removes the need for dense frame-level annotations, using utterance-level supervision while still learning frame-sensitive weighting. Third, it preserves directional change rather than relying solely on magnitude-based differences, and the ablation study indicates that removing directional information degrades performance (Li et al., 20 Jul 2025).

The method’s significance lies in the way these components reinforce one another. The temporal-difference formulation supplies a representation of unnatural evolution in feature space; the dual-level TDAM captures that irregularity at fine and coarse scales; and adaptive average pooling preserves the relevant temporal structure when variable-length utterances are normalized. The resulting system is therefore not merely an average-pooled classifier, but a temporally structured weakly supervised detector whose decision rule is organized around the hypothesis that partial deepfakes are marked by unstable directional trajectories in acoustic embedding space (Li et al., 20 Jul 2025).