FAConformer: Frequency-Aware EEG Attention

• FAConformer is a frequency-aware CNN-Transformer framework designed for EEG-based auditory attention decoding by decomposing signals into distinct frequency bands.
• It uses independent CNN–Transformer encoders per band and a frequency-aware attention module to explicitly model interactions between spectral bands.
• The model outperforms 12 competitive baselines on DTU and KUL datasets by accurately preserving band specificity and optimizing cross-band dependencies.

Searching arXiv for FAConformer and closely related papers. FAConformer is a frequency-aware convolutional Transformer architecture for electroencephalography-based auditory attention decoding (AAD). It was introduced to address a specific limitation in prior AAD systems: although attention-related neural activity is distributed across multiple EEG frequency bands, many earlier models either relied on handcrafted band features or merged multi-band information through shallow concatenation, thereby failing to preserve band specificity or to model cross-band interactions explicitly. In its canonical form, FAConformer decomposes EEG into multiple frequency bands, applies an independent CNN–Transformer encoder to each band, fuses the resulting band-wise representations through a frequency-aware attention module, and regularizes multi-branch optimization with band-wise auxiliary supervision. On two public AAD datasets and three decision-window lengths, the model was reported to outperform 12 competitive baselines, exceeding the strongest prior model by approximately $4.9\%$ on DTU and $3.0\%$ on KUL (Wang et al., 12 Jun 2026).

1. Origin and problem setting

FAConformer was proposed for auditory attention decoding, the task of inferring which of multiple concurrent speakers a listener is attending to from neural responses alone. In the target “cocktail party” scenario, the intended downstream use is neuro-steered hearing systems, in which decoded attention can be used to select or enhance the target speaker in real time. The formulation used in the original work takes preprocessed EEG only, without speech or sound inputs, and predicts the attended direction as a binary class label such as left versus right speaker (Wang et al., 12 Jun 2026).

The architectural motivation is explicitly frequency-domain. Attention-related EEG activity is described as being spread across delta, theta, alpha, beta, and gamma ranges, with different bands reflecting different neural processes. The underlying premise is therefore twofold: a strong AAD system should preserve band specificity, and it should also model interactions between bands rather than treating them as independent channels. Prior AAD models are characterized in three broad categories of limitation: shallow multi-band usage through handcrafted features such as differential entropy or bandpower; naïve band fusion by direct concatenation; and single-stream CNN or CNN–Transformer architectures that do not explicitly represent cross-band dependencies (Wang et al., 12 Jun 2026).

The name “FAConformer” denotes a frequency-aware CNN-Transformer framework in this AAD context. A related naming usage appears in the multimodal architecture “EEG-EMG FAConformer,” where a frequency-aware Conv-Transformer is used for EEG–EMG fusion in motor pattern recognition rather than AAD. In that work, the frequency-aware component is centered on EEG band attention, but the overall objective, inputs, and architecture are different from the AAD-specific formulation (He et al., 2024).

2. Architectural organization

The canonical FAConformer pipeline is hierarchical. For a trial $\mathbf{X} \in \mathbb{R}^{C \times T}$, the model first performs multi-band decomposition,

$\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$

then applies an independent encoder $E_b$ to each band-limited signal,

$$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$

stacks the band features as tokens,

$$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$

and passes them through a frequency-aware attention stage to obtain a fused global representation, followed by a classifier

$\hat{y} = C_g(\mathbf{f}).$

In parallel, auxiliary band-specific classifiers

$\hat{y}^{b} = C_b(\mathbf{z}^{b})$

are attached to each branch to stabilize optimization (Wang et al., 12 Jun 2026).

This decomposition yields four named components. The first is multi-band decomposition. The second is Within-Band Encoding (WBE), consisting of independent CNN–Transformer encoders for each band. The third is Frequency-Aware Attention (FAA), which treats band-wise features as a sequence of tokens and performs self-attention along the band axis. The fourth is Band-Wise Auxiliary Supervision (BAS), which adds supervised losses to each band branch. The architecture is thus explicitly multi-branch and frequency-aware by design rather than by feature engineering alone (Wang et al., 12 Jun 2026).

The input–output protocol is fixed-length EEG windows of $2\,\mathrm{s}$, $3.0\%$0, or $3.0\%$1, and the output is binary attended-direction classification evaluated by accuracy. This is a subject-specific setting rather than a cross-subject generalization setting. A plausible implication is that the design emphasizes within-subject spectral structure and branch specialization over subject-independent invariance.

3. Frequency decomposition and within-band encoding

FAConformer uses eight canonical frequency bands per dataset, chosen according to sampling rate and Nyquist frequency. For the DTU dataset, downsampled to $3.0\%$2 with Nyquist $3.0\%$3, the bands are $3.0\%$4, $3.0\%$5, $3.0\%$6, $3.0\%$7, $3.0\%$8, $3.0\%$9, $\mathbf{X} \in \mathbb{R}^{C \times T}$0, and $\mathbf{X} \in \mathbb{R}^{C \times T}$1. For KUL, downsampled to $\mathbf{X} \in \mathbb{R}^{C \times T}$2 with Nyquist $\mathbf{X} \in \mathbb{R}^{C \times T}$3, the bands are $\mathbf{X} \in \mathbb{R}^{C \times T}$4, $\mathbf{X} \in \mathbb{R}^{C \times T}$5, $\mathbf{X} \in \mathbb{R}^{C \times T}$6, $\mathbf{X} \in \mathbb{R}^{C \times T}$7, $\mathbf{X} \in \mathbb{R}^{C \times T}$8, $\mathbf{X} \in \mathbb{R}^{C \times T}$9, $\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$0, and $\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$1 (Wang et al., 12 Jun 2026).

Band extraction is FFT-based. For each trial, the discrete Fourier transform $\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$2 is computed, a binary mask $\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$3 is constructed for each band, and inverse FFT reconstructs the band-limited signal: $\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$4 Each $\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$5 retains shape $\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$6 and becomes the input to an independent encoder (Wang et al., 12 Jun 2026).

Within-band encoding has a CNN front end followed by a lightweight Transformer. The CNN front end applies a grouped $\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$7D convolution for channel projection, then a depthwise temporal Conv1D with multi-scale kernels, aggregation with GELU, and a patch-wise log-power embedding that reshapes the time axis into patches of length $\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$8 to produce

$\mathbf{X} \to \{\mathbf{X}^{b}\}_{b=1}^{B_f},$9

The subsequent band-wise Transformer $E_b$0 performs self-attention along the patch or time dimension and uses standard multi-head self-attention, feed-forward layers, residual connections, and normalization, ultimately producing a band-level vector

$E_b$1

The justification for independent encoders is that different bands encode different oscillatory patterns and temporal scales; a shared encoder would force bands to share kernels and attention parameters, potentially averaging out band-specific structure (Wang et al., 12 Jun 2026).

The same frequency-aware principle appears in “EEG-EMG FAConformer,” but the mechanism differs. There, EEG is band-pass filtered with Chebyshev Type II filters into $E_b$2 bands, and a frequency band attention module learns weights $E_b$3 to produce a weighted aggregate

$E_b$4

That architecture also includes multi-scale temporal fusion, an Independent Channel-Specific Convolution Module, an EMG branch, and a multi-head-attention fuse module for multimodal fusion. This suggests that the term “FAConformer” has been used more broadly for frequency-aware Conv-Transformer designs in biosignal decoding, although the AAD model and the EEG–EMG motor model remain distinct systems (He et al., 2024).

4. Frequency-Aware Attention and Band-Wise Auxiliary Supervision

After within-band encoding, FAConformer models cross-band dependencies through Frequency-Aware Attention. The band features are stacked into the token sequence

$E_b$5

and processed by a Transformer encoder over the band axis,

$E_b$6

where $E_b$7 is specified as a 2-layer, 2-head Transformer encoder. A linear projection and flattening then produce the fused representation: $E_b$8 The self-attention operation follows the standard form

$E_b$9

Conceptually, this replaces direct band concatenation with a learned interaction mechanism in which each band token can attend to the others before classification (Wang et al., 12 Jun 2026).

The distinction from ordinary flatten-and-concatenate fusion is central to the method. Direct concatenation treats bands as independent and introduces no learned cross-band weighting prior to the classifier. FAA instead lets every band token attend to all other bands, which yields non-uniform attention patterns and exposes potentially hub-like bands in the learned attention matrices. The original experiments report that FAA is especially helpful at the shortest window length, $$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$0, where limited temporal context makes spectral context more consequential (Wang et al., 12 Jun 2026).

Band-Wise Auxiliary Supervision addresses a training pathology of multi-branch fusion systems: if the fusion module assigns low weight to some branches early in training, those branches may receive weak gradients through the global loss and become under-optimized. FAConformer therefore defines a main loss

$$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$1

band-specific auxiliary losses

$$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$2

their average

$$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$3

and the total objective

$$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$4

with $$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$5 in the experiments. The intended effect is to ensure that each band encoder learns discriminative features regardless of the current fusion weights, making subsequent adaptive weighting more meaningful (Wang et al., 12 Jun 2026).

Ablation findings reported for the AAD model are consistent with this design logic: WBE improves over a single-stream baseline; BAS on top of WBE gives further gains, particularly for longer windows and for KUL; FAA on top of WBE gives the largest gains for $$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$6 windows; and the full combination WBE+BAS+FAA gives the best or second-best accuracy across conditions (Wang et al., 12 Jun 2026).

5. Data, training protocol, and empirical performance

The AAD evaluation uses two public datasets. DTU comprises 18 subjects and 64 EEG channels, with EEG sampled at $$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$7, 60 trials per subject, and $$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$8 per trial; the task is to attend one of two spatially separated speakers at $$\mathbf{z}^{b} = E_b(\mathbf{X}^{b}) \in \mathbb{R}^{D_b},$$9 using Danish speech. KUL comprises 16 subjects and 64 EEG channels, EEG originally sampled at $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$0, 8 trials per subject of approximately 6 minutes each, and a task involving two HRTF-spatialized speakers at $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$1 using Dutch speech (Wang et al., 12 Jun 2026).

Preprocessing differs by dataset. For DTU, the pipeline uses a high-pass filter at $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$2, a notch filter at $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$3, and downsampling to $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$4. For KUL, it uses a high-pass filter at $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$5, downsampling to $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$6, and artifact removal as in Somers et al. (2018). Trials are segmented into non-overlapping windows of $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$7, $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$8, and $$\mathbf{Z} = [\mathbf{z}^{1}, \dots, \mathbf{z}^{B_f}] \in \mathbb{R}^{D_b \times B_f},$$9. Splitting is subject-specific and chronological: the first $\hat{y} = C_g(\mathbf{f}).$0 of windows form a training pool, the last $\hat{y} = C_g(\mathbf{f}).$1 form the test set, and within the training pool, $\hat{y} = C_g(\mathbf{f}).$2 is randomly taken as validation, yielding an overall $\hat{y} = C_g(\mathbf{f}).$3 train/validation/test ratio (Wang et al., 12 Jun 2026).

Training uses batch size $\hat{y} = C_g(\mathbf{f}).$4, maximum $\hat{y} = C_g(\mathbf{f}).$5 epochs with early stopping of patience $\hat{y} = C_g(\mathbf{f}).$6, learning rate $\hat{y} = C_g(\mathbf{f}).$7, weight decay $\hat{y} = C_g(\mathbf{f}).$8, $\hat{y} = C_g(\mathbf{f}).$9, FAA depth $\hat{y}^{b} = C_b(\mathbf{z}^{b})$0, FAA head count $\hat{y}^{b} = C_b(\mathbf{z}^{b})$1, and five repeated runs with seeds $\hat{y}^{b} = C_b(\mathbf{z}^{b})$2. The evaluation metric is discrete classification accuracy: $\hat{y}^{b} = C_b(\mathbf{z}^{b})$3 No AUC or correlation-based metrics are used in that paper (Wang et al., 12 Jun 2026).

The comparative study includes 12 baselines: EEGNet, SCNN, IFNet, DBPNet, DARNet, DHGCN, CTNet, TMSA-Net, EEGConformer, MSCFormer, MSVTNet, and DBConformer. The average accuracies across the three window lengths are summarized below.

Dataset	Best baseline	FAConformer
DTU	IFNet, 79.40%	84.38%
KUL	DBPNet, 90.91%	93.92%

These averages correspond to improvements of $\hat{y}^{b} = C_b(\mathbf{z}^{b})$4 on DTU and $\hat{y}^{b} = C_b(\mathbf{z}^{b})$5 on KUL over the strongest baseline. Example per-window results include DTU $\hat{y}^{b} = C_b(\mathbf{z}^{b})$6: IFNet $\hat{y}^{b} = C_b(\mathbf{z}^{b})$7 versus FAConformer $\hat{y}^{b} = C_b(\mathbf{z}^{b})$8; DTU $\hat{y}^{b} = C_b(\mathbf{z}^{b})$9: IFNet $2\,\mathrm{s}$0 versus FAConformer $2\,\mathrm{s}$1; and KUL $2\,\mathrm{s}$2: IFNet $2\,\mathrm{s}$3 versus FAConformer $2\,\mathrm{s}$4 (Wang et al., 12 Jun 2026).

The related multimodal “EEG-EMG FAConformer” was evaluated on the Jeong2020 dataset with 25 participants, 60-channel EEG, 6-channel EMG, and 5-fold cross-validation. It reported the best or tied-best accuracy across multigrasp MI, twist MI, multigrasp real-move, reaching real-move, and twist real-move tasks, with particularly large gains in motor imagery settings. Because those experiments concern EEG–EMG fusion for motor pattern recognition rather than EEG-only AAD, they are best understood as a separate application family sharing the FAConformer naming and frequency-aware Conv-Transformer motif (He et al., 2024).

6. Interpretability, robustness, limitations, and scope of the term

FAConformer includes explicit interpretability analyses through FAA attention maps. The reported procedure averages self-attention matrices over all test samples, layers, and heads for each subject, yielding $2\,\mathrm{s}$5 matrices that show how query bands distribute attention over key bands. Three observations are emphasized: attention is non-uniform, indicating selective weighting rather than uniform fusion; dominant bands differ across subjects, indicating subject-specific spectral signatures; and some bands behave as hubs, especially higher-frequency bands such as $2\,\mathrm{s}$6, $2\,\mathrm{s}$7, and sometimes $2\,\mathrm{s}$8 on KUL (Wang et al., 12 Jun 2026).

Feature-space analysis using t-SNE compares embeddings from EEGNet, IFNet, DBConformer, DBPNet, and FAConformer. The reported result is that FAConformer yields the cleanest separation between attended and unattended classes, with tighter intra-class clusters and larger inter-class gaps across both datasets and all three window lengths. Parameter sensitivity studies further indicate stability with respect to the BAS weight $2\,\mathrm{s}$9, FAA depth $3.0\%$00, and number of attention heads $3.0\%$01: performance is described as stable over reasonable ranges, with only slight degradation for overly deep FAA on KUL (Wang et al., 12 Jun 2026).

The main limitations identified for the AAD formulation are increased architectural complexity, subject-specific training, and EEG-only input. The model uses eight independent CNN–Transformer encoders, an FAA Transformer, and multiple classifiers, which increases parameter count and training time relative to lighter CNNs. The experiments do not study cross-subject generalization. Although some AAD systems incorporate audio or speech-envelope inputs, FAConformer as evaluated uses EEG only. Future directions suggested by the authors include lighter frequency-aware architectures, extension to cross-subject and cross-dataset settings, and multimodal variants combining EEG with audio or video (Wang et al., 12 Jun 2026).

The term FAConformer is therefore best understood in a narrow and a broad sense. In the narrow sense, it denotes the AAD architecture introduced in 2026: a frequency-aware convolutional Transformer with independent band encoders, cross-band attention, and auxiliary supervision (Wang et al., 12 Jun 2026). In a broader biosignal-decoding sense, the same label has also been used for the frequency-aware component within “EEG-EMG FAConformer,” where band attention, convolutional temporal modeling, and Transformer-style fusion are applied to multimodal motor pattern recognition (He et al., 2024). This suggests a family resemblance centered on explicit frequency-aware representation learning, but the AAD model remains the primary referent of the term in the supplied literature.