Phase-Aware Wave-MLP Overview

• Phase-Aware Wave-MLP is a family of MLP architectures that dynamically modulates token mixing using amplitude and phase.
• It redefines token representation by treating tokens as complex wave functions, enabling adaptive, input-dependent aggregation.
• Empirical results demonstrate improved performance on image classification and dense prediction tasks through efficient local-window token interactions.

Searching arXiv for the named papers to ground the article with current paper metadata and citations. Searching arXiv for "An Image Patch is a Wave: Phase-Aware Vision MLP" and closely related phase-aware wave modeling papers. Phase-Aware Wave-MLP denotes a class of MLP-style architectures in which tokens are treated as wave-like entities with amplitude and phase, so that token mixing is modulated by phase rather than performed only through fixed real-valued weights. In the literature, the canonical realization is Wave-MLP, introduced in “An Image Patch is a Wave: Phase-Aware Vision MLP” (Tang et al., 2021), which represents each token as a complex-valued wave function and uses dynamic phase terms to modulate token aggregation. Related work extends the same general theme into handwriting synthesis, where a phase-aware Wave-MLP is embedded inside a GAN generator (Khoa et al., 28 Aug 2025), and into phase-native spectral sequence modeling, where hidden states live on the unit circle and global coupling is implemented by a Discrete Fourier Transform rather than attention (Sigdel, 18 Mar 2026). Taken together, these works define a broader technical perspective in which phase is treated as a first-class representational variable for structured token interaction rather than as an incidental byproduct of complex arithmetic.

1. Definition and conceptual basis

Phase-Aware Wave-MLP emerged from a specific limitation identified in early pure-MLP vision models: token aggregation is performed with fixed learned weights, independent of the semantic content of the current input. In the formulation given for conventional token mixing, if $Z=[z_1,z_2,\dots,z_n]$ with $z_j\in\mathbb{R}^d$, then a standard token-mixing fully connected layer computes

$$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$

Because $W^t$ is shared across all inputs, the same aggregation pattern is applied regardless of the image or sequence content (Tang et al., 2021).

Wave-MLP addresses this by changing the representation of each token. Instead of treating a token as only a real-valued feature vector, it models the token as a wave with an amplitude and a phase. In the original formulation, the token $z_j$ is lifted into a complex-valued wave-like representation

$$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$

where $|z_j|$ is the amplitude and $\theta_j$ is the phase vector (Tang et al., 2021). The amplitude corresponds to the token’s original feature content, while the phase modulates how that token should interact with others during mixing. This phase is not static; it is generated dynamically from input content, so the effective token relationships become input-dependent.

The conceptual motivation is wave superposition. The original analysis considers the superposition of two wave-like tokens and shows that the resulting amplitude depends on the phase difference: $$|z_r| = \sqrt{|z_i|^2 + |z_j|^2 + 2 |z_i|\odot |z_j|\odot \cos(\theta_j-\theta_i)},$$ with phase

$$\theta_r= \theta_i + {\rm atan2}\!\Big( |z_j|\odot \sin(\theta_j-\theta_i),\; |z_i|+|z_j|\odot \cos(\theta_j-\theta_i) \Big).$$

This makes the role of phase explicit: if phases are aligned, aggregation is strengthened; if phases are opposed, aggregation is weakened (Tang et al., 2021).

A related formulation appears in FW-GAN, where the generator’s phase-aware Wave-MLP models each spatial token $z_j\in\mathbb{R}^d$0 as a complex wave signal

$z_j\in\mathbb{R}^d$1

and interprets the amplitude as token content and the phase as a learned modulation controlling relational or directional behavior (Khoa et al., 28 Aug 2025). This suggests a common representational principle across domains: phase-aware Wave-MLP is not defined by a single task, but by the use of amplitude-phase tokenization to make mixing adaptive.

2. Mathematical formulation of phase-aware token mixing

The practical implementation of Wave-MLP avoids general complex-valued neural-network machinery. Using Euler’s formula, the complex token is decomposed as

$z_j\in\mathbb{R}^d$2

Thus each token is represented by real and imaginary components parameterized by trigonometric modulation (Tang et al., 2021).

In the vision formulation, amplitude is generated by a standard channel projection,

$z_j\in\mathbb{R}^d$3

and phase is estimated dynamically from the input by

$z_j\in\mathbb{R}^d$4

where $z_j\in\mathbb{R}^d$5 is a lightweight phase estimation module (Tang et al., 2021). The mixed output is then computed by combining cosine- and sine-modulated components: $z_j\in\mathbb{R}^d$6 This equation is the core phase-aware token-mixing rule. Relative to standard token-FC, it keeps fixed weights $z_j\in\mathbb{R}^d$7 and $z_j\in\mathbb{R}^d$8, but the effective token contribution is modulated by $z_j\in\mathbb{R}^d$9 and $$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$0, which depend on the input content (Tang et al., 2021).

FW-GAN uses the same general mechanism but presents it in a generator context. After wave-like token construction,

$$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$1

the token set $$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$2 is aggregated through Token-FC: $$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$3 and projected back to the real domain as

$$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$4

The notation differs slightly, but the computational idea is the same: token mixing is phase-modulated rather than purely linear in the real-valued activations (Khoa et al., 28 Aug 2025).

A broader generalization is provided by the Phasor Transformer, which is not literally an MLP architecture but occupies a closely related design space. There, sequence states are represented on the unit-circle manifold $$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$5 as

$$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$6

with admissible manifold

$$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$7

The trainable operation is a diagonal phase-shift gate

$$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$8

and global coupling is performed by a normalized DFT matrix $$\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.$$9, yielding blocks of the form

$W^t$0

This is not an MLP mixer in the usual sense, but it provides a phase-native formalism for deterministic global token mixing (Sigdel, 18 Mar 2026).

3. Architectural structure and implementation variants

The canonical Wave-MLP backbone is a hierarchical pure-MLP vision model that preserves $W^t$1 feature maps and stacks four stages of Wave blocks (Tang et al., 2021). Each Wave block contains two major submodules: a Phase-Aware Token Mixing Module and a channel-mixing MLP. The token-mixing module includes horizontal and vertical branches operating along height and width respectively, together with a direct branch that preserves original information. The horizontal and vertical branches are phase-aware; the direct branch is a channel-FC shortcut (Tang et al., 2021).

A crucial implementation choice is that token-FC is restricted to a local window rather than applied globally across all tokens. The paper uses a window size of 7 in the main experiments. This makes the operator compatible with varying image sizes and dense prediction tasks, unlike global token-FC layers whose dimensions are tied to a fixed number of tokens (Tang et al., 2021). This local-window design also suggests that phase-aware Wave-MLP is not simply a complex-valued reformulation of global mixing, but a structured spatial mixer.

The baseline architecture family is summarized by four main scales. Wave-MLP-T has stages with dimensions $W^t$2, repeated $W^t$3 times respectively, with 17M / 2.4G parameters/FLOPs. Wave-MLP-S uses repeats $W^t$4 and has 30M / 4.5G. Wave-MLP-M uses dimensions $W^t$5, expansions $W^t$6, repeats $W^t$7, and has 44M / 7.9G. Wave-MLP-B uses dimensions $W^t$8, repeats $W^t$9, and has 63M / 10.2G (Tang et al., 2021). An efficiency variant, Wave-MLP-T*, replaces the phase estimator with depth-wise convolution and has 15M / 2.1G (Tang et al., 2021).

In FW-GAN, the phase-aware Wave-MLP is not a standalone backbone but is embedded inside the generator’s WaveGBlocks. The generator maps a character sequence

$z_j$0

and a style code

$z_j$1

to an initialized feature map

$z_j$2

which is progressively refined by WaveGBlocks with conditional batch normalization (Khoa et al., 28 Aug 2025). Within each WaveGBlock, the Wave-MLP contains Token Mixing and Channel MLP Mixing, with two directional PATM branches operating along height and width and adaptive branch reweighting inspired by CycleMLP (Khoa et al., 28 Aug 2025). The paper does not provide the exact fusion formula, normalization scheme, or exact activation specification inside the Wave-MLP itself.

The Phasor Transformer offers a different architectural path. It does not use residual streams, LayerNorm, or standard feed-forward MLPs; instead, it alternates complex spectral interference with a pull-back normalization

$z_j$3

which maps raw angular coordinates into a bounded principal interval and is inserted between stacked phase blocks (Sigdel, 18 Mar 2026). This suggests that phase-aware wave architectures can depart substantially from standard MLP-Mixer recipes while preserving the central idea of phase-native token propagation.

4. Dynamic phase estimation, geometry, and representations

A defining property of phase-aware Wave-MLP is that the phase term is generated from input content rather than being a static positional code. In Wave-MLP, the phase estimator $z_j$4 is intentionally lightweight, and the paper studies three variants: identity projection, depth-wise convolution, and channel-FC. The main model uses channel-FC, while the efficient Wave-MLP-T* uses depth-wise convolution (Tang et al., 2021). This reflects a design preference for minimal overhead while preserving input-conditioned modulation.

The representation can be understood in two distinct but related ways. In Wave-MLP and FW-GAN, tokens are amplitude-phase entities, but the implementation remains real-valued through $z_j$5 and $z_j$6 modulation (Tang et al., 2021, Khoa et al., 28 Aug 2025). In the Phasor Transformer, by contrast, the state is explicitly geometric: each token is a unit phasor on $z_j$7, and a length-$z_j$8 sequence lives on the torus $z_j$9 (Sigdel, 18 Mar 2026). That formulation makes phase the native state variable rather than a modulation attached to a real-valued feature.

The FW-GAN formulation introduces a useful distinction between phase awareness and frequency modeling. Inside the Wave-MLP, phase awareness is a feature-mixing mechanism based on amplitude/phase modulation, whereas the broader framework’s explicit frequency modeling is handled by a Frequency Distribution Loss and a high-frequency discriminator (Khoa et al., 28 Aug 2025). The paper states that the Wave-MLP is not presented as using an FFT in its token mixing, wavelets inside the Wave-MLP, or explicit complex convolutions over Fourier coefficients; its wave parameterization is instead based on complex-valued token representation, sinusoidal decomposition via Euler’s formula, and phase-modulated token aggregation (Khoa et al., 28 Aug 2025).

The Phasor Transformer pushes the geometric interpretation further. Inputs are amplitude-normalized and mapped into bounded phases by

$$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$0

then lifted to complex phasors

$$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$1

This deliberately suppresses free amplitude variation in the internal state and makes the method especially appropriate for oscillatory or phase-synchronization-dominated time series (Sigdel, 18 Mar 2026). A plausible implication is that phase-aware Wave-MLP designs can vary in whether they treat amplitude and phase symmetrically or prioritize phase as the primary latent variable.

5. Relations to other phase-aware and wave-based models

Wave-MLP sits within a wider landscape of phase-aware modeling, but its relation to nearby methods is architectural rather than terminological. The Phasor Transformer is the closest mathematical reference for a phase-native global mixer. It replaces attention with a parameter-free DFT token mixer and trainable pre/post phase shifts, with per-block complexity $$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$2 rather than attention’s $$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$3, and trainable parameter count $$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$4 for depth $$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$5 (Sigdel, 18 Mar 2026). It is not a token-mixing MLP, but conceptually it occupies the same design space: deterministic global mixing, lightweight trainable local operations, and explicit oscillatory inductive bias.

For long-horizon wave prediction, “Harnessing Loss Decomposition for Long-Horizon Wave Predictions via Deep Neural Networks” (Deo et al., 2024) contributes a phase-sensitive training principle rather than a phase-aware backbone. It decomposes prediction error into dissipation and dispersion components,

$$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$6

$$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$7

and uses them in a weighted objective. The paper does not define phase explicitly via Fourier variables or complex outputs; instead, correlation in physical space serves as a surrogate for phase alignment (Deo et al., 2024). This suggests that a phase-aware Wave-MLP need not always represent phase in the latent state; it may also become phase-aware through its loss design.

“A Phase Shift Deep Neural Network for High Frequency Approximation and Wave Problems” (Cai et al., 2019) provides a different MLP-based route. Its PhaseDNN represents high-frequency functions as sums of fixed sinusoidal carriers multiplied by MLP-generated envelopes,

$$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$8

so that each subnetwork learns a demodulated low-frequency envelope (Cai et al., 2019). This is phase-aware and MLP-based, but unlike Wave-MLP it uses prescribed frequency channels rather than dynamically estimated token phases.

APEX introduces a still broader view of phase-aware wave modeling by explicitly factorizing complex fields as

$$\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,$$9

and arguing that amplitude remains comparatively stable across frequency shifts while phase deteriorates rapidly (Sun et al., 26 May 2026). Its higher-frequency enhancer represents targets as

$|z_j|$0

and conditions generation on a coarse amplitude anchor and a Green’s-function-inspired phase prior (Sun et al., 26 May 2026). This is not a Wave-MLP architecture, but it reinforces the broader principle that explicit amplitude/phase factorization can be preferable to undifferentiated real-valued processing in oscillatory domains.

6. Empirical results, ablations, and limitations

Wave-MLP was evaluated extensively on ImageNet-1K, COCO, and ADE20K. On ImageNet, Wave-MLP-T* achieves 80.1% with 15M parameters and 2.1G FLOPs; Wave-MLP-T achieves 80.6% with 17M / 2.4G; Wave-MLP-S achieves 82.6% with 30M / 4.5G; Wave-MLP-M achieves 83.4% with 44M / 7.9G; and Wave-MLP-B achieves 83.6% with 63M / 10.2G (Tang et al., 2021). The paper compares these against prior MLP, CNN, and transformer backbones and reports a favorable accuracy-efficiency tradeoff. For example, Wave-MLP-S outperforms Swin-T at the same 4.5G FLOPs, and Wave-MLP-B slightly exceeds Swin-B with fewer parameters and FLOPs (Tang et al., 2021).

On COCO with RetinaNet 1×, the reported AP values are 40.4 for Wave-MLP-T, 43.4 for Wave-MLP-S, 44.8 for Wave-MLP-M, and 44.2 for Wave-MLP-B (Tang et al., 2021). On Mask R-CNN 1×, Wave-MLP-T reports 41.5 box AP / 38.2 mask AP, Wave-MLP-S 44.0 / 40.0, Wave-MLP-M 45.3 / 41.0, and Wave-MLP-B 45.7 box AP, with a formatting corruption in the B-row mask AP entry (Tang et al., 2021). On ADE20K with Semantic FPN, the reported mIoU values are 41.2 for Wave-MLP-T, 44.4 for Wave-MLP-S, and 46.8 for Wave-MLP-M (Tang et al., 2021). These results establish that phase-aware Wave-MLP functions not only as an image-classification model but also as a dense-prediction backbone.

The ablation studies directly test the role of phase. In Table 5, removing phase yields 78.8%, using static phase yields 79.3%, and using dynamic phase yields 80.1% on Wave-MLP-T* (Tang et al., 2021). This is the clearest evidence that the benefit is not merely the introduction of extra parameters; the phase must be input-dependent. A related ablation on phase estimation modules reports 78.8% for the baseline, 79.3% for identity, 80.1% for depth-wise convolution, and 80.6% for channel-FC (Tang et al., 2021). Window-size ablations report 79.7% for size 3, 79.8% for size 5, 80.1% for size 7, and 80.0% for all tokens, supporting the local-window choice (Tang et al., 2021).

FW-GAN provides task-specific evidence in handwriting synthesis. The ablation comparing the Base BigGAN backbone with and without the Wave-Modulated Generator shows the transition from (B) Base + FDL to (C) (B) + Wave-Modulated Generator improves FID from 10.20 to 6.89, CER from 10.74 to 10.32, NED from 10.50 to 10.12, and WER from 29.05 to 28.81 (Khoa et al., 28 Aug 2025). Because this comparison isolates the effect of the Wave-Modulated Generator, it is the most direct evidence for the usefulness of the phase-aware Wave-MLP inside FW-GAN.

At the same time, the available evidence places clear limits on what can be claimed. FW-GAN does not specify the number of WaveGBlocks, exact channels per stage, MLP hidden expansion ratio, exact PATM equations for height and width branches, exact branch fusion weights, exact normalization inside Wave-MLP, or exact activation function names inside the module (Khoa et al., 28 Aug 2025). In Wave-MLP itself, not every branch-fusion detail is written as an explicit equation in the main text (Tang et al., 2021). The Phasor Transformer, while highly relevant conceptually, is only a partial architectural match because it omits standard feed-forward MLPs and uses DFT-based global coupling instead (Sigdel, 18 Mar 2026). These omissions matter for historical and conceptual classification: “Phase-Aware Wave-MLP” is best understood as a family of designs organized by phase-modulated token interaction, not as a single fully standardized block.

A common misconception is to equate phase-aware Wave-MLP with generic complex-valued neural networks. The primary literature does not support that equation. Wave-MLP is implemented with real-valued trigonometric modulation rather than full complex-valued learning (Tang et al., 2021), and FW-GAN likewise frames its Wave-MLP as wave-inspired token mixing rather than as FFT-based or wavelet-based processing (Khoa et al., 28 Aug 2025). Another misconception is that phase-awareness alone guarantees superior performance. The ablations show that dynamic phase helps relative to no phase or static phase (Tang et al., 2021), but the gains depend on the phase estimator, window design, and surrounding architecture. This suggests that phase-aware Wave-MLP is a specific inductive bias, not a universal replacement for all token-mixing schemes.