Micro-Expression-Aware Avatar Fingerprinting via Inter-Frame Feature Differencing

Abstract: Avatar fingerprinting, i.e., verifying who drives a synthetic talking-head video rather than whether it is real, is a critical safeguard for authorized use of face-reenactment technology. Existing methods rely on a fixed, non-differentiable landmark extraction stage that prevents the fingerprinting model from being optimized end-to-end from raw pixels. We propose a preprocessing-free system built on a micro-expression-aware backbone operating on raw video frames, with inter-frame feature differencing as the core design principle: consecutive feature maps are subtracted in the learned deep feature space, so that temporally stable appearance dimensions contribute zero to the output while driver-specific motion dynamics are preserved. A controlled ablation on NVFAIR confirms that temporal motion accounts for the large majority of discriminative performance, and that raw appearance features actively degrade identity separation. Both the choice of backbone and the differencing principle are essential: differencing alone is insufficient when applied to a generic encoder, as appearance-dominated features collapse to near-identical representations across adjacent frames, while the micro-expression-aware F5C backbone retains measurable motion variation that the differencing operation can exploit. Without any external preprocessing, our model achieves an overall AUC of 0.877 on NVFAIR and matches or exceeds the landmark-based baseline on the majority of cross-generator pairs.

• The paper proposes a preprocessing-free avatar fingerprinting system that isolates driver-specific micro-expression cues using inter-frame feature differencing.
• It introduces the F5C backbone to capture asymmetric micro-motion signatures with structural feature extraction, achieving superior AUC performance.
• Extensive experiments show that the method outperforms landmark-based baselines and generalizes well across various reenactment systems.

Micro-Expression-Aware Avatar Fingerprinting via Inter-Frame Feature Differencing

Introduction and Motivation

A critical challenge in digital identity security is determining the true driver behind synthetic talking-head avatar videos, a need heightened by recent advances in neural face-reenactment technologies. Traditional deepfake detection methods rely on artifact-based cues, which are invariant across drivers for a given generator and thus ineffective for avatar fingerprinting. The problem pivots from identifying real versus synthetic content to extracting person-specific temporal facial dynamics, enabling verification of the driver's identity despite the target face’s invariance. This paper proposes a preprocessing-free avatar fingerprinting system, leveraging micro-expression-aware features and inter-frame feature differencing to isolate and retain driver-specific motion dynamics while structurally suppressing target appearance.

Architectural Innovations

End-to-End Pipeline

The system operates purely on raw video frames, circumventing fixed non-differentiable landmark extraction stages used in prior works. Each clip is processed through the micro-expression-sensitive F5C backbone, generating spatially-structured feature maps for each frame. Temporal structure is introduced by differencing consecutive feature maps—effectively nullifying temporally stable appearance dimensions and capturing only the driver-specific motion signal. The resultant motion tensor is then compressed by a lightweight temporal identity head, yielding the embedding used for driver verification.

Figure 1: End-to-end pipeline for micro-expression-aware avatar fingerprinting; feature extraction and differencing yield motion tensors that encode driver identity.

Inter-Frame Feature Differencing

The differencing operation is central to appearance invariance: subtracting adjacent frame features cancels static target appearance. Mathematically, if a feature map $\mathbf{f}_t$ is decomposed as $\mathbf{f}_t = \mathbf{a} + \boldsymbol{\mu}_t$, with $\mathbf{a}$ as static appearance and $\boldsymbol{\mu}_t$ as dynamic motion, differencing leaves only $\boldsymbol{\mu}_{t+1} - \boldsymbol{\mu}_t$. This structural solution obviates the need for explicit domain supervision or auxiliary objectives and is empirically validated via controlled ablation studies.

Micro-Expression-Aware Backbone (F5C)

The F5C backbone comprises:

• ConvStack: Maps $128\times128$ grayscale frames to $128\times16\times16$ feature maps, maintaining spatial structure essential for facial dynamics modeling.
• Fully-Connected Convolution (fcc): Split 128-channel feature maps into two branches with global row/column receptive fields, capturing asymmetric micro-motion signatures characteristic to individuals.
• Channel Correspondence Convolution (ccc): Constructs $k$-NN graphs over spatial positions, aggregating correlated micro-movements, thus extending beyond the geometric scope of landmarks.

Experiments and Results

Dataset and Protocol

Evaluation uses nvfair, a benchmark featuring over 650,000 synthetic videos from 161 identities generated by three reenactment systems (Face-vid2vid, TPS, LIA). Identity-disjoint splits ensure strict generalization, with test identities unseen during training.

Method Comparison

The proposed end-to-end system achieves an AUC of 0.877 on nvfair, outperforming the landmark-based NVFAIR baseline (AUC 0.853) and matching or exceeding baseline results across most cross-generator pairs. The model omits all external preprocessing while maintaining superior verification performance and cross-generator robustness.

Controlled Ablations and Representation Analysis

Four input conditions were tested using the same F5C backbone and supervised contrastive training:

• feat (proposed): Feature-space differencing after encoding yields highest AUC (0.877).
• pixel: Differencing in pixel space (AUC 0.861), confirming the importance of differencing but showing feature-space application is superior.
• raw: Using raw features without differencing (AUC 0.649) demonstrates appearance retention degrades identity separation.
• static: Single-frame baseline (AUC 0.610).

Longer clip lengths monotonically improve AUC, peaking at 0.891 for 128 frames.

Cross-Generator Generalization

Cross-generator experiments show strong transfer capabilities, especially when using the feat representation. Training on LIA and testing across all generators achieves the highest single results (e.g., 0.902 AUC on LIA). The key outcome is consistent performance across unseen generators, which is not guaranteed by landmark-based or generic CNN approaches.

Figure 2: Cross-generator generalization comparison for feat versus landmark-based NVFAIR baseline; solid lines indicate feat, dashed lines indicate baseline, showing robust transfer.

Backbone Ablation

Replacing F5C with ResNet18 (11.6M parameters vs. 0.53M) yields inferior performance, even collapsing below static baselines for feature-space differencing. This highlights the necessity of a backbone engineered for micro-motion sensitivity, as generic appearance-oriented encoders do not adequately preserve discriminative inter-frame variation.

Theoretical and Practical Implications

The study provides strong evidence that temporal differencing is the principal mechanism for extracting identity information from avatar videos, as raw appearance features actively hinder separation. The F5C backbone's architectural design enables effective motion-specific encoding, making feature-space differencing particularly potent. These findings have practical utility: fingerprinting models can be optimized end-to-end, reducing computational complexity and removing dependencies on external landmark detectors.

The implications reach further, suggesting that behavioral biometrics for synthetic avatars are structurally robust to appearance-based attacks and pipeline shifts. The appearance-agnostic nature of the differencing operation is theoretically well-founded. Future work may extend this model to diffusion-based generators or adversarially-adapted drivers, evaluate the stability of embeddings under multi-clip aggregation, and refine the disentanglement of appearance versus temporal dynamics.

Conclusion

The paper advances avatar fingerprinting by integrating micro-expression-aware feature extraction and inter-frame feature differencing into a compact, fully end-to-end framework. Appearance invariance and driver specificity are achieved by design, validated through extensive ablation and cross-domain experiments. The findings position this approach as a principled, practical solution for biometric authentication in synthetic media, with potential for further generalization to emerging face reenactment paradigms and adversarial contexts.