Amplitude and Phase Attention in Signal Processing

• Amplitude and Phase Attention is a framework that models signal components separately yet interactively for enhanced inference in time-frequency tasks.
• It employs joint fusion and specialized attention mechanisms, such as Transformers and gating, to preserve structural fidelity and robustness.
• Empirical results in speech separation, deepfake detection, SAR, and RGB–event tracking confirm its superior performance over magnitude-only methods.

Searching arXiv for papers on amplitude and phase attention across modalities. Amplitude and phase attention denotes a class of modeling strategies in which amplitude and phase are treated as distinct but interacting signal components, and downstream inference is guided by their joint structure rather than by magnitude-only processing or undifferentiated real-valued features. In current work, this idea appears most explicitly in time–frequency speech separation, frequency-domain deepfake detection, SAR detection, and RGB–event tracking, where the underlying representation is a complex spectrum $X = A e^{i\phi}$ and the model either fuses $A$ and $\phi$ before attention or constructs separate amplitude and phase branches with learned interactions (Liu et al., 17 Sep 2025, Lin et al., 9 Jan 2026, Ni et al., 12 Aug 2025, Wang et al., 3 Jan 2026). Related but broader formulations occur in functional data analysis, oscillator modeling, holography, and complex-valued energy-based models, where “phase” may mean time warping, collective oscillator angle, or optical phase rather than Fourier angle (Marron et al., 2015, Fagerholm et al., 2019, Xiang et al., 2022, Li et al., 2020).

1. Conceptual scope and domain-specific meanings

In spectral signal processing, amplitude and phase arise from the decomposition

$X(u,v)=A(u,v)e^{i\phi(u,v)}$

for images, or

$$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$

for speech (Lin et al., 9 Jan 2026, Liu et al., 17 Sep 2025). In these settings, amplitude attention typically emphasizes frequency energy, contrast, or texture strength, whereas phase attention emphasizes alignment, edges, contours, or waveform reconstruction fidelity. Phase4DFD explicitly frames magnitude as encoding power and global frequency statistics, and phase as preserving edges, contours, object boundaries, and geometric structure; APSS frames explicit phase estimation as necessary because phase errors bound separation quality even with strong magnitude estimation (Lin et al., 9 Jan 2026, Liu et al., 17 Sep 2025).

The same terminology has non-spectral meanings in other literatures. In functional data analysis, amplitude variation is vertical variability in aligned curves, while phase variation is horizontal variability represented by warping functions $h$, with $x_i(t)=y_i(h_i(t))$ after alignment (Marron et al., 2015). In generative models for functional data, amplitude is the aligned function and phase is the reparameterization $\gamma$, modeled separately and then recomposed (Tucker et al., 2012). In oscillator models, phase is motion along the limit cycle and amplitude is deviation transverse to it; dynamic causal modeling therefore introduces separate phase and amplitude connectivity matrices, while phase–amplitude reduction introduces collective phase $\theta$ and amplitude coordinates $r_m$ associated with Floquet modes (Fagerholm et al., 2019, Mircheski et al., 2023).

A recurring misconception is that amplitude and phase are universally “magnitude” and “angle” of a Fourier coefficient. The cited literature shows that the pair can also mean “what happens” versus “when it happens” in functional data, or tangential versus radial dynamics in oscillatory systems (Marron et al., 2015, Tucker et al., 2012, Fagerholm et al., 2019).

2. Core architectural patterns

Recent neural architectures converge on a small set of design patterns: explicit amplitude/phase extraction, a joint or cross-modal interaction stage, and specialized decoders or decision heads.

System	Amplitude/phase representation	Attention or coupling mechanism
APSS	STFT amplitude $A$0 and phase $A$1 stacked as $A$2	Time–frequency Transformers over fused features, then parallel amplitude and phase separators
Phase4DFD	FFT magnitude $A$3 and normalized phase $A$4 from grayscale input	Input-level phase-aware attention map $A$5 that reweights RGB, FFT magnitude, and LBP
DenoDet V2	2D DFT amplitude $A$6 and phase $A$7 of feature maps	Band-wise self-attention plus phase–amplitude token exchange
APMTrack	RGB and event FFT amplitude and phase	Decoupled amplitude attention and phase attention before spatial-domain backbone

APSS exemplifies a fused-encoder design. It extracts mixture amplitude and phase, stacks them,

$A$8

maps them with a feature combiner $A$9 to $\phi$0 with $\phi$1, processes $\phi$2 by $\phi$3 TF-blocks, and then feeds the resulting representation $\phi$4 to parallel amplitude and phase separators (Liu et al., 17 Sep 2025). The feature combiner is the first site of amplitude–phase fusion: its first 2D convolution mixes the two channels immediately, and its DenseNet expands temporal receptive field before Transformer processing.

Phase4DFD uses a gating design rather than Transformer attention. It computes grayscale FFT magnitude

$\phi$5

and phase

$\phi$6

extracts $\phi$7 and $\phi$8 with separate $\phi$9 conv-BN-ReLU branches, concatenates them, and produces an input attention map

$X(u,v)=A(u,v)e^{i\phi(u,v)}$0

which reweights the five-channel augmented input $X(u,v)=A(u,v)e^{i\phi(u,v)}$1 before backbone feature extraction (Lin et al., 9 Jan 2026). Here phase does not propagate as an explicit feature through the backbone; it modulates where the model should look.

DenoDet V2 and APMTrack separate amplitude and phase more explicitly. DenoDet V2 computes

$X(u,v)=A(u,v)e^{i\phi(u,v)}$2

then uses band-wise spectral attention and symmetric cross-attention between amplitude and phase within each frequency band (Ni et al., 12 Aug 2025). APMTrack transforms RGB and event features into the frequency domain, refines $X(u,v)=A(u,v)e^{i\phi(u,v)}$3 with Conv–LeakyReLU–Conv blocks, and fuses high-frequency event information into RGB through separate amplitude and phase attention branches before inverse FFT and tokenization (Wang et al., 3 Jan 2026).

3. Attention mechanisms and phase parameterization

The cited systems use several distinct notions of “attention.” In APSS, the attention operator itself is standard multi-head self-attention. The novelty is not a specialized complex-valued attention rule, but the fact that the queries, keys, and values are functions of fused amplitude–phase features. Each TF-block applies frequency attention at fixed time and time attention at fixed frequency,

$X(u,v)=A(u,v)e^{i\phi(u,v)}$4

with 8 heads in both frequency and time Transformers (Liu et al., 17 Sep 2025). Amplitude and phase therefore affect the content of $X(u,v)=A(u,v)e^{i\phi(u,v)}$5, but not the attention mechanism itself.

Phase4DFD uses convolutional gating as attention. Its effective score function is

$X(u,v)=A(u,v)e^{i\phi(u,v)}$6

with separate magnitude and phase branches followed by nonlinear $X(u,v)=A(u,v)e^{i\phi(u,v)}$7 fusion (Lin et al., 9 Jan 2026). This is not query–key–value attention, but it still learns a spatially varying, channel-specific weighting conditioned jointly on magnitude and phase. The paper emphasizes phase discontinuities introduced by synthetic generation as the cue that guides this modulation.

DenoDet V2 implements the most explicit phase–amplitude cross-attention among the cited vision systems. After band partitioning of the spectrum, Phase–Amplitude Token Exchange builds one cross-attention with amplitude queries and phase keys/values, and another with phase queries and amplitude keys/values. In band $X(u,v)=A(u,v)e^{i\phi(u,v)}$8,

$X(u,v)=A(u,v)e^{i\phi(u,v)}$9

and symmetrically for phase queries attending to amplitude tokens (Ni et al., 12 Aug 2025). This makes phase a structural prior for amplitude denoising and amplitude a reciprocal refinement signal for phase.

APMTrack uses a lighter-weight cross-modal formulation. Its amplitude branch computes

$$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$0

and its phase branch uses the analogous update

$$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$1

after L2 normalization along the channel dimension (Wang et al., 3 Jan 2026). Here event high-frequency information selectively boosts RGB amplitude and phase where the two modalities agree.

Phase parameterization is a central technical issue because direct angle regression is discontinuous at $$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$2. APSS addresses this with its Parallel Estimation Architecture: for each speaker and time–frequency bin it predicts pseudo-real and pseudo-imaginary components $$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$3 and recovers phase by

$$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$4

which avoids naive phase-regression wrap-around problems (Liu et al., 17 Sep 2025). DenoDet V2 similarly replaces direct angle usage with $$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$5 and $$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$6, and reports 55.6 mAP for no split, 56.1 mAP for angle split, and 56.2 mAP for split plus alignment (Ni et al., 12 Aug 2025). This suggests that phase-aware attention mechanisms often require a representation that respects circular geometry rather than treating phase as an unconstrained scalar.

4. Theoretical antecedents and broader formulations

Several cited works do not present an attention layer but provide the mathematical foundations for amplitude–phase factorization. In functional data analysis, the core problem is that phase variability inflates variance and distorts means and principal components if one aligns curves by naive $$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$7 matching. The Fisher–Rao/SRVF framework instead defines

$$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$8

and uses an amplitude distance obtained by minimizing over warping functions, while phase is represented by diffeomorphisms or by $$X(t,f)=\mathcal{STFT}\{x(n)\}(t,f),\quad A(t,f)=|X(t,f)|,\quad P(t,f)=\arg(X(t,f))$$9 on the unit sphere (Marron et al., 2015). Generative modeling then performs separate fPCAs for aligned amplitude and for phase warpings, followed by a joint probability model on their coefficients (Tucker et al., 2012). This is not neural attention, but it formalizes the same principle: amplitude and phase should be modeled separately because they live in different geometries.

Oscillator modeling reaches a similar conclusion from a dynamical-systems direction. Dynamic causal modeling of oscillators introduces a complex state $h$0 and separate phase and amplitude connectivity matrices $h$1 and $h$2, with coupling terms weighted by amplitude similarity or phase alignment (Fagerholm et al., 2019). Phase–amplitude reduction for collective oscillations defines a collective phase $h$3 and amplitudes $h$4 associated with Floquet modes, with reduced equations

$h$5

where $h$6 are phase sensitivity functions and $h$7 are isostable sensitivity functions (Mircheski et al., 2023). In that literature, “attention” corresponds naturally to the relative weighting of perturbation directions that change phase versus those that excite transverse amplitude modes.

Complex-valued learning models make the same decomposition explicit. The Complex Amplitude-Phase Boltzmann Machine uses

$h$8

so that phase-sensitive complex coupling $h$9 and phase-agnostic amplitude coupling $x_i(t)=y_i(h_i(t))$0 are separate terms (Li et al., 2020). The paper’s main claim is that the new amplitude–amplitude coupling term is necessary because the magnitude of the complex input sum would otherwise have to control both activation probability and phase concentration simultaneously.

Related optical and inverse-problem work shows that amplitude–phase decomposition is also meaningful in physics-aware imaging. The THz holography paper models the object field as

$x_i(t)=y_i(h_i(t))$1

and recovers amplitude and phase jointly with supervised and unsupervised dual-head networks, although without explicit attention layers (Xiang et al., 2022). Dielectric metasurfaces achieve complete and independent control of the optical field

$x_i(t)=y_i(h_i(t))$2

by combining form birefringence and rotation angle, and RIS-based transmission schemes similarly treat ON/OFF patterns as amplitude control and phase shifts as coherent focusing (Overvig et al., 2019, Li et al., 2023). These formulations suggest that amplitude and phase attention can be understood as a hardware or model-space allocation of degrees of freedom, not only as a software attention block.

5. Empirical evidence

The empirical record across modalities consistently indicates that explicit amplitude–phase modeling is useful when the task depends on structural fidelity, coherent reconstruction, or noise-robust discrimination.

In speech separation, APSS reports SI-SNRi/SDRi of 21.3 / 21.5 on WSJ0-2Mix, compared with 21.1 / 21.3 for TFPSNet, and 17.1 / 17.6 on Libri2Mix, above the cited time-domain baselines at approximately 16.6–16.9 / 17.2–17.4 (Liu et al., 17 Sep 2025). Its ablations are especially diagnostic: removing the feature combiner drops performance from 21.3 / 21.5 to 17.3 / 17.5; removing PEA yields 19.9 / 20.0; removing amplitude masks yields 20.0 / 20.1. The paper therefore attributes substantial gains to joint amplitude–phase fusion and phase-specific parameterization.

In deepfake detection, magnitude-only augmentation is weak: on DFFD, RGB only gives 99.23% accuracy, RGB + FFT magnitude gives 99.26%, RGB + LBP gives 99.24%, and RGB + FFT + LBP without phase modeling drops to 99.13% (Lin et al., 9 Jan 2026). The best configuration is RGB + FFT + LBP + Phase-Aware, at 99.46%. The same framework achieves 98.62 accuracy and 99.88 AUC on CIFAKE, versus 97.35 and 99.62 for the BNext-M baseline, and 99.46 accuracy and 99.95 AUC on DFFD, versus 98.75 and 99.92 for the same backbone.

In SAR detection, DenoDet V2 reports 56.71 mAP on SARDet-100K, up from 55.88 for DenoDet V1 and 55.01 for the GFL baseline, while reducing parameters from 65.78M to 32.60M (Ni et al., 12 Aug 2025). The amplitude-versus-phase ablation is notable: no spectral refine gives 55.0, only amplitude refine 55.6, only phase refine 56.2, and both together 56.4. Token exchange further raises performance to 56.7. The paper therefore argues that phase is the more robust structural cue under coherent noise, but the best performance still requires amplitude–phase complementarity.

In RGB–event tracking, APMTrack reports SR 68.0 / PR 83.3 on COESOT, with FLOPs reduced from 1167.4G for the concatenation baseline to 701.0G for the full model (Wang et al., 3 Jan 2026). Its component ablation shows amplitude attention alone at SR 66.8 / PR 81.7, phase attention alone at 66.5 / 81.3, and both together at 67.2 / 82.4. This is a direct empirical argument for decoupled amplitude and phase attention: each branch alone is insufficient, while joint use improves both accuracy and efficiency.

6. Limitations, misconceptions, and open directions

A first limitation is terminological. There is no single canonical module called “Amplitude and Phase Attention.” APSS explicitly states that there is no module named exactly this, even though its fused feature combiner and shared TF-Transformer stack can be interpreted as amplitude–phase-aware attention (Liu et al., 17 Sep 2025). The concept therefore spans standard self-attention on fused features, convolutional gating, band-wise cross-attention, and even non-neural geometric or physical decompositions.

A second limitation is representation-specific robustness. Phase4DFD does not systematically explore robustness to heavy JPEG compression or cross-dataset generalization beyond CIFAKE and DFFD (Lin et al., 9 Jan 2026). DenoDet V2 is strongly motivated by SAR speckle statistics and only studies 2D DFT rather than alternative transforms such as wavelets or learned bases (Ni et al., 12 Aug 2025). APMTrack uses Fourier-domain early fusion and motion-guided sparsification effectively for RGB–event tracking, but its design is tightly coupled to high-frequency event information and motion-sensitive token selection (Wang et al., 3 Jan 2026).

A third issue is that explicit phase supervision is not always present. APSS uses only waveform-level SI-SNR loss, even though amplitude and phase are explicitly parameterized (Liu et al., 17 Sep 2025). The paper argues that this can suffice because reconstruction passes through $x_i(t)=y_i(h_i(t))$3 and iSTFT, but it also suggests that auxiliary spectral losses such as amplitude $x_i(t)=y_i(h_i(t))$4, cosine phase loss, or complex $x_i(t)=y_i(h_i(t))$5 could be added. This suggests an open design question rather than a settled prescription.

Across the literature, a plausible synthesis is that amplitude and phase attention is most valuable when the task depends on structural consistency that magnitude-only models tend to underuse: explicit phase in monaural separation, phase discontinuities in synthetic images, phase stability under coherent SAR noise, and high-frequency event cues in multimodal tracking (Liu et al., 17 Sep 2025, Lin et al., 9 Jan 2026, Ni et al., 12 Aug 2025, Wang et al., 3 Jan 2026). The strongest open direction follows directly from APSS and related formulations: maintain amplitude and phase as separate modalities, introduce cross-attention between them, preserve phase-aware parameterizations such as $x_i(t)=y_i(h_i(t))$6 or $x_i(t)=y_i(h_i(t))$7, and let shared context encoders determine how much each component should influence the task at hand (Liu et al., 17 Sep 2025, Ni et al., 12 Aug 2025).