AASIST3 Architecture Overview

• AASIST3 is a deepfake detection architecture that processes raw speech through convolutional encoding and hierarchical graph attention to mitigate spoofing in ASV systems.
• The integration of Kolmogorov-Arnold Network modules enables trainable nonlinear transformations and adaptive pooling, enhancing feature extraction and contextual modeling.
• A multi-branch fusion strategy combining diverse temporal and spatial representations achieves significant performance gains, including over twofold improvements on key metrics.

AASIST3 is a state-of-the-art architecture for speech deepfake detection, representing an evolution of the AASIST (Audio Anti-Spoofing using Integrated Spectro-Temporal GNNs) framework. It introduces Kolmogorov-Arnold Network (KAN) modules, advanced graph attention mechanisms, and incorporates self-supervised learning (SSL) features under open-set conditions. Designed for robustness against synthetic speech attacks—including text-to-speech (TTS) and voice conversion (VC)—AASIST3 overcomes key vulnerabilities in Automatic Speaker Verification (ASV) systems and benchmarks substantial performance gains on the ASVspoof challenge, achieving more than a twofold improvement in minDCF and up to EERs of 0.04% on some spoofing attacks (Borodin et al., 2024, Viakhirev et al., 14 Feb 2026).

1. End-to-End Architecture and Dataflow

AASIST3 processes raw 16 kHz speech waveforms through a multi-stage pipeline combining convolutional encoding, hierarchical KAN-enhanced graph attention, and advanced feature aggregation. The main dataflow consists of:

• Pre-processing: A pre-emphasis filter $x'_\ell = x_\ell - 0.97 x_{\ell-1}$ accentuates high-frequency content to facilitate discriminating spoof from bona fide speech. Open-track models apply additional augmentations (reverberation, noise, pitch, VAD).
• Front-end Feature Extraction:
  • Closed condition: SincConv (80 non-trainable band-pass filters)
  • Open condition: Wav2Vec 2.0 (XLS-R variant), producing 768-dim contextualized SSL features per frame, followed by a small linear/conv layer.
• Encoder: Six stacked 1D convolutional residual blocks with BatchNorm and SELU activation; each block optionally downsamples via max pooling.
• Feature Splitting: The encoder output tensor $\hat{h}\in \mathbb{R}^{C\times T \times F}$ is pooled across frequency or time to obtain “temporal” and “spatial” node sequences, each augmented with positional embeddings.
• KAN-Enhanced Graph Modules:
  • KAN-GAL (Graph Attention Layer): universal function approximator for node relation scoring.
  • KAN-GraphPool: node pooling with KAN-based gating.
  • KAN-HS-GAL: hierarchical, heterogeneous spatio-temporal attention.
• Four Parallel Branches: Each applies a distinct sequence of KAN-HS-GAL/GraphPool modules, parameterized differently to diversify operational capacity.
• Fusion and Aggregation: Branch outputs are summed; max and mean operations further reduce node dimensions.
• Read-out and Classification: After concatenation and dropout, a final KAN layer followed by softmax outputs the spoof probability.

This highly structured pipeline enables the model to exploit both fine-grained spectral details and broad temporal context, combining graph-based and convolutional representations throughout (Borodin et al., 2024, Viakhirev et al., 14 Feb 2026).

2. Kolmogorov-Arnold Networks (KAN): Universal Graph Nonlinearities

KAN modules are a core innovation in AASIST3, replacing conventional fixed nonlinearities in graph modules with trainable, universal 1D function approximators. The function $\phi(x)$ takes the form:

$$\phi(x) = w_b \cdot \mathrm{PReLU}(x) + w_s \cdot \left[ \sum_{i=0}^4 c_i B_i(x) \right]$$

where $B_i(x)$ are fourth-order B-spline basis functions on a grid $[-1,1]$, and parameters $w_b, w_s, a, c_i$ are Kaiming-initialized and trained jointly. A KAN layer maps $X \in \mathbb{R}^{n_l}$ to $X^+ \in \mathbb{R}^{n_{l+1}}$ via a matrix $\Phi_l$ of such scalar $\hat{h}\in \mathbb{R}^{C\times T \times F}$0 functions:

$\hat{h}\in \mathbb{R}^{C\times T \times F}$1

Within the model,

• In KAN-GAL, $\hat{h}\in \mathbb{R}^{C\times T \times F}$2 transforms pairwise node interactions pre-attention.
• In KAN-GraphPool, $\hat{h}\in \mathbb{R}^{C\times T \times F}$3 is used for sigmoid gating in node pooling.
• In KAN-HS-GAL, $\hat{h}\in \mathbb{R}^{C\times T \times F}$4 controls attention maps and all projections.

This approach allows AASIST3 to model complex, context-specific relationships in the graph attention layers while avoiding excessive parameter growth, thus enhancing expressiveness and preserving computational tractability (Borodin et al., 2024).

3. Multi-Branch Graph Attention and Fusion

The architecture’s multi-branch topology serves both as a capacity-increasing measure and as a regularization strategy:

• Four parallel branches (differing in parameters and temperature) each process the encoder output via hierarchical KAN-HS-GAL and KAN-GraphPool modules.
• Each branch produces distinct temporal and spectral representations ($\hat{h}\in \mathbb{R}^{C\times T \times F}$5 for $\hat{h}\in \mathbb{R}^{C\times T \times F}$6).
• Outputs across branches are combined by summation, then further aggregated via max, mean, and concatenation.
• The design intentionally fosters both specialization (branches may lock onto different artifacts) and consensus (all branches jointly inform the global decision).

A comparative interpretability analysis reveals distinctive operational archetypes:

• Effective Specialization (single branch dominance, low EER),
• Effective Consensus (equal contribution, low EER),
• Ineffective Consensus (equal weak features, high EER),
• Flawed Specialization (confident but wrong specialization, high EER).

This suggests that architecture-level strategies—i.e., whether the model distributes trust or specializes per attack—directly mediate empirical reliability, as evidenced by quantized EER outcomes (Viakhirev et al., 14 Feb 2026).

Branch Type	Modules	Functional Role
Per-branch (4x)	KAN-HS-GAL + Pool	Diverse artifact modeling
Global (2x)	GAT-S, GAT-T	Cross-channel consensus

4. Regularization, Training Objectives, and Losses

AASIST3 employs several regularization and objective strategies:

• Dropout: 0.2 before each KAN-based attention block; 0.5 before final fusion.
• BatchNorm: After every ConvUnit and within all KAN modules.
• Losses:
  • Closed condition: Standard cross-entropy ($\hat{h}\in \mathbb{R}^{C\times T \times F}$7) over bona fide/spoof labels.
  • Open condition: Weighted cross-entropy, focal loss $\hat{h}\in \mathbb{R}^{C\times T \times F}$8 per sample, and LibAUCM for direct AUC maximization. Total training objective:

  $\hat{h}\in \mathbb{R}^{C\times T \times F}$9

  with coefficients set per experiment. A plausible implication is that this mixture of losses mitigates class imbalance and directly optimizes for ROC sensitivity, which is critical for rare attack detection (Borodin et al., 2024).

5. Final Scoring, Fusion, and Performance

The scoring function varies by evaluation track:

• Closed track: Single model outputs spoof probability via softmax; evaluated with minDCF and EER.
• Open track: Two independently trained models, $\phi(x)$0, scan input via overlapping 4s windows and produce logit sums $\phi(x)$1 over all windows. The fused score is

$\phi(x)$2

used for thresholding and minDCF/EER metrics.

AASIST3 demonstrates empirical improvements over earlier baselines, achieving minDCF of 0.5357 (closed) and 0.1414 (open) and over twofold improvement on t-DCF compared to vanilla AASIST. These gains arise from integrating low-level feature detail, hierarchical KAN-driven attention, and rich SSL-derived contextual priors (Borodin et al., 2024).

6. Interpretability, Activation Modeling, and Failure Modes

A multi-branch architecture such as AASIST3 poses challenges for interpretability. Component-level analysis is achieved through the extraction of spectral activation signatures:

• Intermediate activations from fourteen principal components (12 branch-level, 2 global) are modeled by their empirical covariance matrices, retaining the top-K eigenvalues as a “spectral signature.”
• Concatenated signatures serve as input to a CatBoost meta-classifier, which is trained to predict attack class.
• TreeSHAP computes per-component Shapley values, yielding normalized branch contribution shares and confidence scores ($\phi(x)$3).
• EER vs. $\phi(x)$4 plots enable taxonomy into operational modes (effective/flawed specialization, effective/ineffective consensus).

This approach demonstrates that internal architectural dynamics—such as which branches are trusted under different attacks—can be quantitatively linked to detection outcomes. Notably, failure modes such as “Flawed Specialization” (overtrust in a non-informative branch) correspond with dramatic EER increases for specific attack types (Viakhirev et al., 14 Feb 2026). This suggests that robust ensemble-level or branch-interaction regularization may be crucial for resisting targeted spoofing attacks.

---

References:

• (Borodin et al., 2024) "AASIST3: KAN-Enhanced AASIST Speech Deepfake Detection using SSL Features and Additional Regularization for the ASVspoof 2024 Challenge"
• (Viakhirev et al., 14 Feb 2026) "Interpreting Multi-Branch Anti-Spoofing Architectures: Correlating Internal Strategy with Empirical Performance"