Audio Tamper Localization

• Tamper Localization in Audio (TLA) is defined as identifying manipulated segments by labeling audio frames or intervals, offering precise temporal recovery of edits.
• Approaches range from deep learning-based frame classification to MP3-frame analysis and proactive watermarking, each exploiting unique signal inconsistencies.
• Hierarchical and boundary-aware techniques are critical for detecting subtle, temporally limited forgeries in speech, music, and audiovisual content.

Tamper Localization in Audio (TLA) is the forensic task of determining where in time an audio signal has been manipulated, rather than only deciding whether the signal is authentic. Recent work formulates TLA as frame-level authenticity prediction for partially spoofed or partially forged speech, MP3-frame labeling of single versus multiple compression to expose temporal splicing, and interval localization in synthesized audiovisual forgeries through watermark-based recovery or verification (Xie et al., 2023, Xia et al., 26 Nov 2025, Xiang et al., 2022, Kim et al., 17 Jul 2025, Zeng et al., 27 Apr 2026). The localized output varies by formulation—binary frame masks, continuous forged intervals, MP3-frame decision sequences, or frame-aligned tamper probabilities—but the common objective is temporal recovery of manipulated regions, especially when the edited span is short, semantically critical, or embedded in otherwise plausible audio (Xia et al., 26 Nov 2025).

1. Scope and task formulations

TLA is defined operationally by the temporal unit being labeled and by the form of the output. In partially spoofed speech, the input is a sequence of framewise acoustic representations,

$$f = (f_1, f_2, \dots, f_T) \in \mathbb{R}^{D \times T},$$

and the target is a frame-label sequence

$y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$

with the best-performing configuration in TDL using $y_t=1$ for genuine frames and $y_t=0$ for spoofed frames (Xie et al., 2023). In partial audio forgery, T3-Tracer separates Partial Forgery Detection (PFD) from Temporal Forgery Localization (TFL): PFD is frame-wise binary classification, while TFL converts discrete frame predictions into continuous forged time intervals with refined temporal boundaries (Xia et al., 26 Nov 2025).

A different formulation appears in codec forensics. For MP3 splice analysis, the localization unit is the MP3 frame, each containing 1152 temporal samples, and the system outputs a binary decision sequence $\{\hat{y}_1,\hat{y}_2,\ldots,\hat{y}_L\}$ indicating whether each frame is singly or multiply compressed (Xiang et al., 2022). In proactive audiovisual integrity systems, TLA is defined as identifying time intervals $\{(t^i_\mathrm{start}, t^i_\mathrm{end})\}_{i=1}^N$ corresponding to tampered audio regions, either from recovered authentic references (Kim et al., 17 Jul 2025) or from frame-aligned watermark integrity evidence (Zeng et al., 27 Apr 2026).

The boundary of the topic is also made explicit by related work that is not yet direct TLA. The ENF spatio-temporal method performs binary audio tampering detection at the recording/file level and states that precise localization is future work, even though its ENF phase sequence and temporal modeling are localization-relevant (Zeng et al., 2022).

Formulation	Localization unit or output	Representative work
Partially spoofed speech	160 ms frame labels; frame mask	TDL (Xie et al., 2023)
Partial audio forgery	Frame forgery scores; continuous forged intervals	T3-Tracer (Xia et al., 26 Nov 2025)
MP3 splice forensics	MP3-frame binary decision sequence	Transformer-based MP3 localization (Xiang et al., 2022)
Synthesized audiovisual forgeries	Time intervals; per-time-step similarity or frame probabilities	Cross-modal watermarking (Kim et al., 17 Jul 2025), LAVA (Zeng et al., 27 Apr 2026)
ENF-based tampering analysis	File-level tampered / no tampering	ENF spatio-temporal learning (Zeng et al., 2022)

2. Threat models and forensic evidence

The most prominent speech-side threat model is partial manipulation rather than fully synthetic utterance generation. T3-Tracer studies partial audio forgery, in which attackers alter only a temporally limited span—often short and semantically important—while preserving the perceptual realism of the rest of the utterance (Xia et al., 26 Nov 2025). TDL treats the analogous partially spoofed setting as insertion of fake segments into otherwise genuine speech, with the objective of locating genuine and spoofed regions at frame level (Xie et al., 2023).

These formulations imply different evidentiary cues. In T3-Tracer, forged frames may contain abnormal spectral or channel patterns at frame level, may be globally inconsistent with the rest of the utterance in speaker traits, prosody, or background conditions at audio level, and may induce abrupt local discrepancies at the boundaries between authentic and forged content at segment level (Xia et al., 26 Nov 2025). TDL instead emphasizes discriminability of frame embeddings and local temporal consistency, using learned similarity among neighboring frames to preserve transitions between real and fake regions (Xie et al., 2023).

Codec-domain TLA relies on a different forensic premise. In MP3 splice analysis, a manipulated file is created by concatenating portions with different prior compression histories and then recompressing the result as MP3, so the localization problem becomes distinguishing single compressed from multiple compressed temporal portions at frame level (Xiang et al., 2022). Here the localized evidence is not semantic inconsistency or speaker mismatch, but compression-history inconsistency across time.

Proactive audiovisual systems replace passive forensic trace analysis with watermark integrity or recovery. In the cross-modal watermarking framework for SAVFs, the authentic audio is embedded into the visual stream before tampering; localization then compares recovered authentic audio against observed tampered audio in a semantic feature space (Kim et al., 17 Jul 2025). LAVA uses semi-fragile audio and visual watermarks, interpreting audio tampering as local degradation or absence of the embedded audio watermark over the affected interval (Zeng et al., 27 Apr 2026). ENF-based work again uses a distinct signal source: tampering is expected to create ENF phase discontinuity or inconsistency over time, although the published method stops at file-level decisions (Zeng et al., 2022).

Taken together, these works show that TLA is not tied to a single evidentiary regime. It may be driven by content inconsistency, inter-frame transition anomalies, codec traces, watermark survival, or ENF continuity, depending on the attack model and recording conditions.

3. Methodological families

A central family of TLA methods treats the problem as strongly supervised temporal authenticity segmentation. TDL uses a Frozen Wav2Vec2-XLS-R front-end, an Embedding Similarity Module (ESM), and a Temporal Convolution Operation (TCONV) (Xie et al., 2023). The ESM learns an embedding space in which genuine frames cluster with genuine frames, fake frames cluster with fake frames, and genuine–fake pairs are separated through a margin-style objective,

$$\mathcal{L}_{all} = \mathcal{L}_{BCE} + \lambda \mathcal{L}_{ESM}, \qquad \lambda=0.1.$$

TCONV then performs content-adaptive temporal convolution by reweighting neighboring frames with frame-specific local similarity. The model outputs frame-level authenticity predictions at 160 ms resolution.

T3-Tracer extends this formulation into explicitly hierarchical temporal modeling (Xia et al., 26 Nov 2025). Its pipeline is: input waveform $\rightarrow$ SSL backbone feature extraction $\rightarrow$ CNN residual enhancement $\rightarrow$ Frame–Audio Feature Aggregation Module (FA-FAM) $y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$0 Segment-level Multi-Scale Discrepancy-Aware Module (SMDAM) $y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$1 cross-attention fusion $y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$2 PRN decoder $y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$3 frame forgery scores and continuous forged intervals. FA-FAM combines Frame-level Feature Aggregation (FFA) and Audio-level Feature Aggregation (AFA), while SMDAM models both absolute local inconsistency and inter-frame differences across temporal windows 3, 5, and 7. Training is tri-supervised with frame labels, boundary labels, and a contrastive objective: $y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$4 with $y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$5, $y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$6, and $y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$7.

A second family uses codec-domain sequence labeling. The MP3 localization method parses MP3 bitstream fields, applies CNNs to mdct_coef and scalefactor, concatenates them with other codec metadata into a 300-D frame feature vector, adds positional encoding, interleaves one class token per frame, and processes the resulting token sequence with an 8-layer transformer using 15 heads (Xiang et al., 2022). Its distinguishing design is not waveform modeling but temporal inference over compression-trace features. Localization is obtained as a binary decision sequence over MP3 frames rather than as boundary regression.

A third family is proactive and cross-modal. In the SAVF recovery framework, the visual input is transformed by DWT, the authentic audio by STFT, and both are coupled by Invertible Neural Network (INN) blocks: $y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$8 TLA is then driven by the similarity score

$y = (y_1, y_2, \dots, y_T) \in \{0,1\}^{T},$9

computed between temporally aligned semantic features of tampered audio and recovered authentic audio (Kim et al., 17 Jul 2025). The method is notable because it states that the network is trained end-to-end without requiring localization annotations for tampering attacks.

LAVA also belongs to the proactive category but operates through explicit watermark detection and score-level fusion (Zeng et al., 27 Apr 2026). The audio detector outputs a per-sample watermark-presence vector $y_t=1$0, which is aggregated into a frame-aligned tamper score

$y_t=1$1

This audio score is then combined with a visual score sequence through temporal stretch correction, a visual reliability gate, temporal offset alignment, confidence-weighted fusion, and temperature scaling calibration. The resulting output is a calibrated per-frame tamper probability $y_t=1$2, with visual spatial maps only when the visual branch is reliable.

Finally, ENF-based work contributes a methodological precursor rather than a complete TLA system. It extracts a high-precision first-order ENF phase sequence $y_t=1$3 via DFT-based analysis, constructs fixed-size spatial and temporal representations, and processes them with a parallel CNN and BiLSTM plus attention (Zeng et al., 2022). Because the final classifier is file-level, its relevance to TLA lies mainly in the time-indexed ENF phase sequence and temporal modeling rather than in its published output.

4. Datasets, annotation regimes, and evaluation protocols

The major speech benchmarks are LAV-DF, ASVS2019-PS (PS / PartialSpoof), and HAD (Half-Truth Audio Dataset) for T3-Tracer (Xia et al., 26 Nov 2025), and ASVspoof2019 Partial Spoof (19PS) with cross-dataset evaluation on LAV-DF for TDL (Xie et al., 2023). TDL evaluates against frame labels at 160 ms resolution, with the 19PS maximum utterance duration of 21.03 s, W2V2 feature shape (1050, 1024), and 132 frame labels per padded utterance (Xie et al., 2023). T3-Tracer requires at least frame-level authenticity labels and boundary labels, although the paper does not spell out the exact annotation granularity of each dataset (Xia et al., 26 Nov 2025).

Codec-based TLA uses a distinct corpus. The MP3 localization paper trains and evaluates on 486,743 MP3 audio clips, synthesized from LJSpeech, GTZAN, and MAESTRO, with manipulation histories spanning single, double, and triple compression (Xiang et al., 2022). The localization unit is the MP3 frame, approximately 26.12 ms at 44.1 kHz. ENF-based tampering detection uses Carioca, New Spanish, and their combination, but only for file-level authenticity classification rather than temporal localization (Zeng et al., 2022).

Proactive audiovisual TLA uses video-centric corpora. The cross-modal watermarking framework evaluates on HDTF, with 410 talking-face videos, 98 for training and 312 for evaluation, on random 5-second segments with 25 fps video and 16 kHz audio (Kim et al., 17 Jul 2025). LAVA uses LAV-DF as its primary benchmark with frame-level temporal annotations, plus FakeAVCeleb and VoxCeleb2 for generalization and controlled async-tampering studies (Zeng et al., 27 Apr 2026).

Evaluation protocols differ sharply across formulations. T3-Tracer separates detection and localization: PFD uses EER, AUC, FNR, FPR, and F1-score, while TFL uses AP at temporal IoU thresholds $y_t=1$4, AR at proposal counts $y_t=1$5, and mAP over thresholds from $y_t=1$6 to $y_t=1$7 with step $y_t=1$8 (Xia et al., 26 Nov 2025). TDL evaluates localization only as per-frame classification, using EER, Precision, Recall, and F1 after removing padded frames (Xie et al., 2023). MP3 localization reports Jaccard score, $y_t=1$9-score, and Balanced accuracy over positive-frame indices (Xiang et al., 2022). The SAVF watermarking framework uses IoU, AP, and AUC for TLA (Kim et al., 17 Jul 2025), while LAVA reports AP, temporal IoU, and ECE, with calibration treated as part of localization reliability (Zeng et al., 27 Apr 2026).

These protocol differences are substantive rather than cosmetic. Framewise metrics emphasize per-frame discrimination, whereas AP at high IoU thresholds, Jaccard, or interval IoU put more pressure on boundary quality and contiguous segment recovery.

5. Representative empirical findings

On partially forged speech, T3-Tracer reports the best published performance across HAD, LAV-DF, and PS for both detection and localization (Xia et al., 26 Nov 2025). For PFD, it reaches EER/F1 of 0.07 / 99.95 on HAD, 0.80 / 99.57 on LAV-DF, and 7.41 / 94.04 on PS. For TFL, it achieves mAP 99.27 on HAD, 94.29 on LAV-DF, and 57.28 on PS. The gains are most pronounced on PS, where manipulations are short and dense: relative to CFPRF, T3-Tracer improves [email protected] from 66.34 to 68.74, [email protected] from 40.96 to 42.73, mAP from 55.22 to 57.28, and AR@1 from 18.48 to 21.65. Its ablations show that removing SMDAM reduces PS mAP from 57.28 to 55.34, supporting the specific role of segment-level boundary modeling.

TDL establishes a strong framewise baseline for partially spoofed speech (Xie et al., 2023). On ASVspoof2019 PS, frame-level EER falls from 8.79% without ESM to 7.04% with ESM, outperforming LCNN-BLSTM and 5gMLP baselines. In cross-dataset evaluation, training on 19PS and testing on LAV-DF, TDL reaches EER 11.23, Precision 98.73, Recall 75.42, and F1 85.51. A particularly important result is the label-formulation ablation: Boundary 1 yields EER 10.89 and F1 80.85, whereas direct authenticity prediction with real $y_t=0$0, fake $y_t=0$1 yields EER 7.04 and F1 91.54. This directly argues against the assumption that sparse boundary labeling is necessarily the best route to localization.

In codec forensics, the transformer-based MP3 method reports Jaccard 80.50, $y_t=0$2 84.43, and Balanced Accuracy 84.49, substantially above adapted prior detectors (Xiang et al., 2022). It also maintains similar recall across Single: 84.61, Double: 83.76, and Triple: 84.92 compression counts, indicating that the localization mechanism is not limited to one recompression depth.

Proactive SAVF methods report very strong interval localization when authentic references can be recovered or verified. The cross-modal watermarking framework achieves, on HDTF, IoU 97.02, AP 99.89, AUC 99.95 for Audio Swapping, and IoU 95.40, AP 98.28, AUC 98.83 for Voice Cloning (Kim et al., 17 Jul 2025). An important internal result is that direct raw comparison between recovered and tampered audio gives AP = 87.17, whereas semantic feature comparison gives AP = 98.28. LAVA, evaluated on localized short-video deepfakes, reports near-perfect detection performance (AP = 0.999) and shows large gains in temporal localization reliability under compression and asynchrony (Zeng et al., 27 Apr 2026). On LAV-DF under JPEG, Naive fusion has temporal IoU 0.750, whereas LAVA reaches 0.955; under audio stretch, raw audio watermark AP can drop to 0.131–0.148, but after correction it returns to 0.998–0.999. The paper also reports Audio-only AP = 0.999 in clean and JPEG conditions.

The ENF spatio-temporal model is not evaluated as TLA, but it provides context for forensic signals that may later support localization. It reports recording-level tampering detection accuracy of 97.62%, exceeding prior ENF-based methods by 2.12%-7.12%, and shows that temporal ENF modeling with $y_t=0$3 and BiLSTM is nearly as strong as the full spatio-temporal fusion (Zeng et al., 2022).

6. Limitations, misconceptions, and research directions

A recurrent limitation is supervision burden. TDL is explicitly strongly supervised at frame level and depends on frame labels aligned to 160 ms evaluation units (Xie et al., 2023). T3-Tracer requires frame labels, boundary labels, and contrastive pair construction, and the paper does not provide separate audio-level supervision even though it models audio-level inconsistency architecturally (Xia et al., 26 Nov 2025). This constrains scalability when precise timestamps are unavailable. By contrast, the SAVF recovery framework states that it is trained without requiring localization annotations for tampering attacks, but that advantage is coupled to a proactive watermarking assumption (Kim et al., 17 Jul 2025).

Another limitation is domain specificity. TDL is built around speech deepfake partial spoofing and is described as not directly a general-purpose audio editing tamper detector for music, environmental sound, or arbitrary splicing (Xie et al., 2023). The MP3 transformer is explicitly MP3-specific and depends on codec-domain features such as mdct_coef, scalefactor, and Huffman-table information (Xiang et al., 2022). ENF-based analysis requires recoverable ENF contamination and remains vulnerable when ENF is weak or absent (Zeng et al., 2022). Proactive audiovisual approaches require watermarking before distribution; they are therefore not retrospective tools for arbitrary pre-existing media (Kim et al., 17 Jul 2025, Zeng et al., 27 Apr 2026).

Several papers also leave parts of inference under-specified. T3-Tracer states that PRN converts frame predictions into continuous forged intervals and uses Soft-NMS, but does not provide the internal equations of PRN or the exact thresholding or segment-merging rule (Xia et al., 26 Nov 2025). TDL does not describe hysteresis, Viterbi decoding, median filtering, or segment merging, implying direct thresholded frame classification (Xie et al., 2023). The SAVF watermarking paper does not provide an explicit thresholding equation for converting similarity scores into final tamper intervals (Kim et al., 17 Jul 2025). LAVA’s strongest temporal alignment results rely on oracle offset selection using test labels, which the paper presents as an upper bound rather than a deployable procedure (Zeng et al., 27 Apr 2026).

A common misconception is that utterance-level deepfake detection and TLA are interchangeable. The literature reviewed here contradicts that view. T3-Tracer explicitly frames partial forgery as harder than utterance-level fake/real classification because the manipulation may occupy only a few frames and remain acoustically consistent with neighboring genuine content (Xia et al., 26 Nov 2025). TDL further shows that direct frame authenticity prediction can outperform explicit boundary labeling (Xie et al., 2023). Conversely, the ENF study shows that strong recording-level detection does not by itself constitute localization (Zeng et al., 2022).

A second misconception is that TLA is necessarily an audio-only problem. The proactive literature shows a different paradigm in which localization is enabled by cross-modal recovery or by fusion of audio and visual watermark evidence (Kim et al., 17 Jul 2025, Zeng et al., 27 Apr 2026). This suggests that, in some settings, temporal localization can be improved by using information outside the observed audio stream itself. A plausible implication is that future TLA systems will increasingly combine modality-specific forensic traces, explicit temporal calibration, and boundary-aware modeling, especially when manipulations are short, sparse, overlapping, or desynchronized.

Across the current literature, the strongest empirical pattern is consistent: fine-grained temporal modeling matters most when tampering is localized rather than global. Hierarchical frame/segment/audio modeling improves [email protected] and mAP on dense short-span forgeries (Xia et al., 26 Nov 2025); learned frame embeddings and similarity-aware temporal filtering improve framewise localization and cross-dataset recall (Xie et al., 2023); codec-history sequence labeling localizes splice regions at MP3-frame resolution (Xiang et al., 2022); and watermark-based systems show that temporal localization reliability depends critically on alignment, recovery quality, and calibration under realistic distortions (Kim et al., 17 Jul 2025, Zeng et al., 27 Apr 2026).