---
title: 'TDAM-AvgPool: Temporal Diff in Deepfake Speech'
url: https://www.emergentmind.com/topics/tdam-avgpool
type: topic
---

# TDAM-AvgPool: Temporal Diff in Deepfake Speech

Searching arXiv for the cited TDAM-AvgPool paper and closely related usages of “TDAM” to support the article.
TDAM-AvgPool is a frame-level temporal difference learning framework for partial deepfake speech detection that treats synthetic speech as exhibiting abnormal temporal dynamics rather than merely anomalous content. It is designed for the setting in which only some segments of an utterance are manipulated, and it avoids frame-level supervision by learning from utterance-level labels only. The method combines a Temporal Difference Attention Module (TDAM), which emphasizes suspicious temporal variation, with adaptive average pooling, which converts variable-length self-supervised speech embeddings into a fixed-length representation while preserving essential temporal patterns. In reported experiments, the model achieves state-of-the-art performance on PartialSpoof and HAD, including an EER of 0.59% on the PartialSpoof evaluation set and 0.03% on HAD [2507.15101].

## 1. Problem formulation and temporal-difference premise

TDAM-AvgPool is introduced for **partial deepfake speech detection**, a regime in which manipulated and genuine segments coexist within the same utterance. This setting is more difficult than fully synthetic speech detection because the detector must remain sensitive to short fake spans without being dominated by long bonafide regions. A central motivation of the method is that earlier approaches often rely on **frame-level or segment-level annotations** during training, and these are described as costly and time-consuming to obtain at scale, hard to define precisely when fake segments are short or boundaries are ambiguous, and brittle when deepfake generation methods smooth out the transitions between bonafide and fake regions [2507.15101].

The method therefore reframes detection: instead of locating explicit fake boundaries, it models deepfake speech as containing **erratic directional changes** and **unnatural local transitions** in feature space. Given frame-wise embeddings
$$
\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_T,
$$
the normalized direction vectors between consecutive frames are defined as
$$
\Delta \mathbf{x}_t = \frac{\mathbf{x}_{t+1} - \mathbf{x}_t}{\|\mathbf{x}_{t+1} - \mathbf{x}_t\|}, \quad t = 1, \ldots, T-1.
$$
Local directional consistency is then measured through cosine similarity between adjacent directional vectors:
$$
\cos(\theta_t) = \frac{\Delta \mathbf{x}_t \cdot \Delta \mathbf{x}_{t+1}}{\|\Delta \mathbf{x}_t\| \, \|\Delta \mathbf{x}_{t+1}\|}, \quad t = 1, \ldots, T-2.
$$
The reported empirical pattern is that bonafide speech has higher utterance-level mean $\mu$ and lower standard deviation $\sigma$, whereas deepfake speech has lower $\mu$ and higher $\sigma$. This observation motivates using directional irregularity as the core detection cue [2507.15101].

## 2. Architectural composition of TDAM-AvgPool

The TDAM-AvgPool pipeline begins with **wav2vec2-XLSR** as the self-supervised feature extractor. The resulting variable-length embeddings are regularized and length-normalized by adaptive average pooling, then refined by fully connected layers and a **1D ResNet block**. A temporal difference map is constructed from the refined sequence, and TDAM converts that map into a frame-sensitive attention signal that reweights the utterance representation before classification [2507.15101].

Let the refined embedding be
$$
\mathbf{E} \in \mathbb{R}^{T' \times D},
$$
where $T'$ is the pooled temporal length and $D$ is the embedding dimension. Before differences are computed, the model applies a 1D convolution across time to obtain a contextualized embedding $\mathbf{E}^{\text{conv}}$. The directional difference map is then defined by
$$
\mathbf{M}[t] = \mathbf{E}^{\text{conv}}[t+1] - \mathbf{E}[t], \quad \text{for } t = 1, \dots, T'-1.
$$
A zero vector is appended at the end so that the shape remains consistent. This construction explicitly encodes frame-to-frame temporal change rather than raw content [2507.15101].

TDAM itself uses a **dual-level hierarchical difference representation**. At the first level, $\mathbf{M}$ is reshaped into a time-spatial map and processed by a 2D convolution that expands the channel dimension from 1 to 32, producing
$$
\mathbf{X}_1 \in \mathbb{R}^{C \times T' \times D}.
$$
This representation captures fine local artifacts such as micro-prosodic inconsistencies and abrupt local changes. At the second level, $\mathbf{X}_1$ is downsampled to 4 channels, passed through another convolution with a larger receptive field, and upsampled back to 32 channels to yield
$$
\mathbf{X}_2 \in \mathbb{R}^{C \times T' \times D}.
$$
The two scales are fused into confidence weights through
$$
\mathbf{Y} = \sigma\!\left[l * (\mathbf{X}_1 + \mathbf{X}_2)\right] \odot \mathbf{E},
$$
where $l$ is a $1 \times 1$ 2D convolution, $\sigma(\cdot)$ is the sigmoid function, and $\odot$ denotes element-wise multiplication. A fully connected layer with softmax is then applied to $\mathbf{Y}$, and frame predictions are averaged across time to obtain the utterance-level decision [2507.15101].

## 3. Adaptive average pooling and the “AvgPool” designation

The “AvgPool” component refers to **adaptive average pooling**, which is used to handle variable-length utterances without discarding potentially informative regions. If the self-supervised extractor outputs
$$
\mathbf{F} \in \mathbb{R}^{T \times D},
$$
the model partitions the $T$ frames into $T'$ segments and computes
$$
\mathbf{F}'[t'] = \frac{1}{|S_{t'}|} \sum_{t \in S_{t'}} \mathbf{F}[t],
$$
where $S_{t'}$ is the set of frame indices in segment $t'$, $|S_{t'}|$ is the number of frames in that segment, and $\mathbf{F}' \in \mathbb{R}^{T' \times D}$ is the fixed-length pooled sequence. If $T < T'$, zero-padding is used [2507.15101].

The rationale given for average pooling is specific. It converts variable-length utterances into a fixed-length representation, preserves **global temporal structure** better than truncation, avoids discarding potentially fake regions, is less biased toward extreme peaks than max pooling, and retains both strong and weak temporal cues. The reported ablation results support this choice directly: **TDAM-AvgPool** outperforms both **TDAM-Trim-Pad** and **TDAM-MaxPool** on the PartialSpoof evaluation set [2507.15101].

This design is consequential because the method is not based on isolated transient spikes alone. Its detection signal depends on how directional changes evolve across the utterance. Average pooling therefore acts less as a generic compression operator than as a mechanism for preserving temporal variation patterns in a standardized sequence length. A plausible implication is that the pooling stage is structurally aligned with the method’s emphasis on distributed temporal irregularity rather than only boundary-local artifacts.

## 4. Supervision regime, optimization, and decision rule

TDAM-AvgPool is trained using **utterance-level labels only**. This is central to the method’s practical positioning: it does not require explicit supervision for where fake speech begins or ends, yet it still produces frame-sensitive attention through the temporal-difference pathway [2507.15101].

The training objective is **cross-entropy loss** with class weighting to address imbalance. The reported class weights are 9 for bonafide and 1 for partial deepfakes. Optimization uses **Adam** with learning rate $5 \times 10^{-4}$, weight decay $10^{-4}$, batch size 2, and 10 epochs [2507.15101].

At inference time, the model produces frame-level predictions and then averages them over time for the final utterance-level score. This architecture distinguishes between two roles: TDAM generates confidence-weighted frame emphasis, while the final averaging step converts that frame-sensitive representation into a global decision. The method therefore remains weakly supervised at the label level while still being temporally localized in its internal representation [2507.15101].

## 5. Empirical performance and ablation evidence

The reported evaluation uses **PartialSpoof (PS)** and **HAD (Half-Truth)**, with additional cross-dataset comparison involving **ASVspoof 2019 LA**. The performance metrics are **EER** on PartialSpoof and **EER and AUC** on HAD. On PartialSpoof, TDAM-AvgPool achieves **0.19% EER** on the development set and **0.59% EER** on the evaluation set. On HAD, it achieves **0.03% EER** and **99.99% AUC**. The comparison tables place these results ahead of the listed prior systems, including 0.64% EER by Zhang et al. on PartialSpoof evaluation and 0.06% EER by Cai et al. on HAD [2507.15101].

The ablation results are especially informative because they isolate the contributions of temporal differences, hierarchical modeling, directional information, and pooling choice. On the PartialSpoof evaluation set, the following results are reported:

| Variant | PS evaluation EER |
|---|---:|
| w/o TDAM | 0.87% |
| w/o $\mathbf{X}_1$ and $\mathbf{X}_2$ | 0.74% |
| w/o $\mathbf{X}_2$ | 0.80% |
| w/o directional information | 0.76% |
| TDAM-Trim-Pad | 1.03% |
| TDAM-MaxPool | 1.00% |
| TDAM-AvgPool | 0.59% |

These numbers support several conclusions stated in the paper. The full TDAM is essential; both fine- and coarse-scale difference representations contribute; directional information is not redundant; and adaptive average pooling is more effective than trimming-padding or max pooling for this task [2507.15101].

The accompanying visual analysis is consistent with these results: the model highlights not only transition boundaries but also internal fake regions, despite being trained only with utterance-level labels. This indicates that the learned representation is sensitive to broader temporal inconsistency rather than narrowly to boundary artifacts alone [2507.15101].

## 6. Scope, related terminology, and common confusions

Within the cited arXiv literature, the explicit use of **TDAM-AvgPool** as a model name corresponds to the partial deepfake speech detector described in “Frame-level Temporal Difference Learning for Partial Deepfake Speech Detection” [2507.15101]. This specificity matters because the acronym **TDAM** is used in unrelated ways in other domains.

In “DeU-Net: Deformable U-Net for 3D Cardiac MRI Video Segmentation,” **TDAM** denotes a **Temporal Deformable Aggregation Module** for deformable temporal fusion in cardiac MRI video segmentation, and the paper explicitly states that it does **not** describe a TDAM-AvgPool variant or use average pooling as TDAM’s temporal aggregation mechanism [2007.06341]. In “Temporal Pooling Strategies for Training-Free Anomalous Sound Detection with Self-Supervised Audio Embeddings,” the paper explicitly states that **TDAM-AvgPool is not mentioned** and that the closest correspondence is simply **temporal mean pooling** rather than a separate TDAM-AvgPool method [2603.04605].

A common misconception is therefore to treat TDAM-AvgPool as a generic name for any temporal average pooling module. The cited literature does not support that reading. In the deepfake-speech context, the term refers to a particular combination of temporal-difference modeling, hierarchical attention, and adaptive average pooling [2507.15101]. This suggests that TDAM-AvgPool is best understood as an application-specific architecture rather than a general-purpose pooling family.

## 7. Methodological significance

TDAM-AvgPool differs from prior partial-deepfake approaches in three stated respects. First, it does not depend on **transition artifacts** remaining visibly detectable; instead, it targets **temporal inconsistency throughout the utterance**, including within fake regions. Second, it removes the need for **dense frame-level annotations**, using utterance-level supervision while still learning frame-sensitive weighting. Third, it preserves **directional change** rather than relying solely on magnitude-based differences, and the ablation study indicates that removing directional information degrades performance [2507.15101].

The method’s significance lies in the way these components reinforce one another. The temporal-difference formulation supplies a representation of unnatural evolution in feature space; the dual-level TDAM captures that irregularity at fine and coarse scales; and adaptive average pooling preserves the relevant temporal structure when variable-length utterances are normalized. The resulting system is therefore not merely an average-pooled classifier, but a temporally structured weakly supervised detector whose decision rule is organized around the hypothesis that partial deepfakes are marked by unstable directional trajectories in acoustic embedding space [2507.15101].

Source: https://www.emergentmind.com/topics/tdam-avgpool