---
title: Phase-Aware Wave-MLP Overview
url: https://www.emergentmind.com/topics/phase-aware-wave-mlp
type: topic
---

# Phase-Aware Wave-MLP Overview

Searching arXiv for the named papers to ground the article with current paper metadata and citations.
Searching arXiv for "An Image Patch is a Wave: Phase-Aware Vision MLP" and closely related phase-aware wave modeling papers.
Phase-Aware Wave-MLP denotes a class of MLP-style architectures in which tokens are treated as wave-like entities with amplitude and phase, so that token mixing is modulated by phase rather than performed only through fixed real-valued weights. In the literature, the canonical realization is Wave-MLP, introduced in “An Image Patch is a Wave: Phase-Aware Vision MLP” [2111.12294], which represents each token as a complex-valued wave function and uses dynamic phase terms to modulate token aggregation. Related work extends the same general theme into handwriting synthesis, where a phase-aware Wave-MLP is embedded inside a GAN generator [2508.21040], and into phase-native spectral sequence modeling, where hidden states live on the unit circle and global coupling is implemented by a Discrete Fourier Transform rather than attention [2603.17433]. Taken together, these works define a broader technical perspective in which phase is treated as a first-class representational variable for structured token interaction rather than as an incidental byproduct of complex arithmetic.

## 1. Definition and conceptual basis

Phase-Aware Wave-MLP emerged from a specific limitation identified in early pure-MLP vision models: token aggregation is performed with fixed learned weights, independent of the semantic content of the current input. In the formulation given for conventional token mixing, if \(Z=[z_1,z_2,\dots,z_n]\) with \(z_j\in\mathbb{R}^d\), then a standard token-mixing fully connected layer computes
\[
\text{Token-FC}(Z, W^t)_j = \sum_k W^t_{jk} \odot z_k.
\]
Because \(W^t\) is shared across all inputs, the same aggregation pattern is applied regardless of the image or sequence content [2111.12294].

Wave-MLP addresses this by changing the representation of each token. Instead of treating a token as only a real-valued feature vector, it models the token as a wave with an amplitude and a phase. In the original formulation, the token \(z_j\) is lifted into a complex-valued wave-like representation
\[
\tilde z_j = |z_j| \odot e^{i\theta_j}, \qquad j=1,2,\cdots,n,
\]
where \(|z_j|\) is the amplitude and \(\theta_j\) is the phase vector [2111.12294]. The amplitude corresponds to the token’s original feature content, while the phase modulates how that token should interact with others during mixing. This phase is not static; it is generated dynamically from input content, so the effective token relationships become input-dependent.

The conceptual motivation is wave superposition. The original analysis considers the superposition of two wave-like tokens and shows that the resulting amplitude depends on the phase difference:
\[
|z_r| = \sqrt{|z_i|^2 + |z_j|^2 + 2 |z_i|\odot |z_j|\odot \cos(\theta_j-\theta_i)},
\]
with phase
\[
\theta_r= \theta_i + {\rm atan2}\!\Big( |z_j|\odot \sin(\theta_j-\theta_i),\; |z_i|+|z_j|\odot \cos(\theta_j-\theta_i) \Big).
\]
This makes the role of phase explicit: if phases are aligned, aggregation is strengthened; if phases are opposed, aggregation is weakened [2111.12294].

A related formulation appears in FW-GAN, where the generator’s phase-aware Wave-MLP models each spatial token \(\mathbf{f}_j\) as a complex wave signal
\[
\tilde{\mathbf{f}_j = |\mathbf{f}_j| \odot e^{i\theta_j}, \quad j = 1, 2, 3,\dots,n,
\]
and interprets the amplitude as token content and the phase as a learned modulation controlling relational or directional behavior [2508.21040]. This suggests a common representational principle across domains: phase-aware Wave-MLP is not defined by a single task, but by the use of amplitude-phase tokenization to make mixing adaptive.

## 2. Mathematical formulation of phase-aware token mixing

The practical implementation of Wave-MLP avoids general complex-valued neural-network machinery. Using Euler’s formula, the complex token is decomposed as
\[
\tilde z_j = |z_j|\odot \cos\theta_j+i|z_j|\odot \sin\theta_j,\qquad j=1,2,\cdots,n.
\]
Thus each token is represented by real and imaginary components parameterized by trigonometric modulation [2111.12294].

In the vision formulation, amplitude is generated by a standard channel projection,
\[
z_j=\text{Channel-FC}(x_j,W^c), \qquad j=1,2,\cdots,n,
\]
and phase is estimated dynamically from the input by
\[
\theta_j=\Theta(x_j, W^\theta),
\]
where \(\Theta\) is a lightweight phase estimation module [2111.12294]. The mixed output is then computed by combining cosine- and sine-modulated components:
\[
o_j = \sum_{k}W^t_{jk} z_k\odot \cos\theta_k + W^i_{jk} z_k\odot \sin\theta_k, \qquad j=1,2,\cdots, n.
\]
This equation is the core phase-aware token-mixing rule. Relative to standard token-FC, it keeps fixed weights \(W^t\) and \(W^i\), but the effective token contribution is modulated by \(\cos\theta_k\) and \(\sin\theta_k\), which depend on the input content [2111.12294].

FW-GAN uses the same general mechanism but presents it in a generator context. After wave-like token construction,
\[
\tilde{z}_j = |z_j| \odot \cos \theta_j + i |z_j| \odot \sin \theta_j, \quad j = 1, 2, \cdots, n,
\]
the token set \(\tilde F=[\tilde f_1,\tilde f_2,\cdots,\tilde f_n]\) is aggregated through Token-FC:
\[
\tilde{o}_j = \text{Token-FC}(\tilde{F}, W^t)_j, \quad j = 1, 2, \cdots, n,
\]
and projected back to the real domain as
\[
o_j = \sum_k W^t_{jk} f_k \odot \cos \theta_k + W^i_{jk} f_k \odot \sin \theta_k, \quad j = 1, 2, \cdots, n.
\]
The notation differs slightly, but the computational idea is the same: token mixing is phase-modulated rather than purely linear in the real-valued activations [2508.21040].

A broader generalization is provided by the Phasor Transformer, which is not literally an MLP architecture but occupies a closely related design space. There, sequence states are represented on the unit-circle manifold \(S^1\) as
\[
\boldsymbol{z}=\left(e^{i\phi_1},\dots,e^{i\phi_N}\right)^\top\in\mathbb{T}^N\subset\mathbb{C}^N,
\]
with admissible manifold
\[
\mathcal{M}_{\mathrm{LPM}=\mathbb{T}^N=\{\boldsymbol{z}\in\mathbb{C}^N:\ |z_t|=1,\ t=1,\dots,N\}.
\]
The trainable operation is a diagonal phase-shift gate
\[
S(\boldsymbol{\theta})=\mathrm{diag}\!\left(e^{i\theta_1},\dots,e^{i\theta_T}\right),
\]
and global coupling is performed by a normalized DFT matrix \(F_T\), yielding blocks of the form
\[
\mathcal{B}(\boldsymbol{\theta})=S(\boldsymbol{\theta}^{\mathrm{post})F_TS(\boldsymbol{\theta}^{\mathrm{pre}).
\]
This is not an MLP mixer in the usual sense, but it provides a phase-native formalism for deterministic global token mixing [2603.17433].

## 3. Architectural structure and implementation variants

The canonical Wave-MLP backbone is a hierarchical pure-MLP vision model that preserves \(H\times W\times C\) feature maps and stacks four stages of Wave blocks [2111.12294]. Each Wave block contains two major submodules: a Phase-Aware Token Mixing Module and a channel-mixing MLP. The token-mixing module includes horizontal and vertical branches operating along height and width respectively, together with a direct branch that preserves original information. The horizontal and vertical branches are phase-aware; the direct branch is a channel-FC shortcut [2111.12294].

A crucial implementation choice is that token-FC is restricted to a local window rather than applied globally across all tokens. The paper uses a window size of 7 in the main experiments. This makes the operator compatible with varying image sizes and dense prediction tasks, unlike global token-FC layers whose dimensions are tied to a fixed number of tokens [2111.12294]. This local-window design also suggests that phase-aware Wave-MLP is not simply a complex-valued reformulation of global mixing, but a structured spatial mixer.

The baseline architecture family is summarized by four main scales. Wave-MLP-T has stages with dimensions \(64,128,320,512\), repeated \(2,2,4,2\) times respectively, with **17M / 2.4G** parameters/FLOPs. Wave-MLP-S uses repeats \(2,3,10,3\) and has **30M / 4.5G**. Wave-MLP-M uses dimensions \(64,128,320,512\), expansions \(8,8,4,4\), repeats \(3,4,18,3\), and has **44M / 7.9G**. Wave-MLP-B uses dimensions \(96,192,384,768\), repeats \(2,2,18,2\), and has **63M / 10.2G** [2111.12294]. An efficiency variant, Wave-MLP-T\*, replaces the phase estimator with depth-wise convolution and has **15M / 2.1G** [2111.12294].

In FW-GAN, the phase-aware Wave-MLP is not a standalone backbone but is embedded inside the generator’s WaveGBlocks. The generator maps a character sequence
\[
\mathbf{y} \in \{0,1\}^{n \times L}
\]
and a style code
\[
\mathbf{z} \in \mathbb{R}^d
\]
to an initialized feature map
\[
\mathbf{F}_0 \in \mathbb{R}^{C_0 \times H_0 \times W_0},
\]
which is progressively refined by WaveGBlocks with conditional batch normalization [2508.21040]. Within each WaveGBlock, the Wave-MLP contains Token Mixing and Channel MLP Mixing, with two directional PATM branches operating along height and width and adaptive branch reweighting inspired by CycleMLP [2508.21040]. The paper does not provide the exact fusion formula, normalization scheme, or exact activation specification inside the Wave-MLP itself.

The Phasor Transformer offers a different architectural path. It does not use residual streams, LayerNorm, or standard feed-forward MLPs; instead, it alternates complex spectral interference with a pull-back normalization
\[
\Phi_{\mathrm{norm}(\phi_{\mathrm{raw})=\arcsin(\sin(\phi_{\mathrm{raw})),
\]
which maps raw angular coordinates into a bounded principal interval and is inserted between stacked phase blocks [2603.17433]. This suggests that phase-aware wave architectures can depart substantially from standard MLP-Mixer recipes while preserving the central idea of phase-native token propagation.

## 4. Dynamic phase estimation, geometry, and representations

A defining property of phase-aware Wave-MLP is that the phase term is generated from input content rather than being a static positional code. In Wave-MLP, the phase estimator \(\Theta\) is intentionally lightweight, and the paper studies three variants: identity projection, depth-wise convolution, and channel-FC. The main model uses channel-FC, while the efficient Wave-MLP-T\* uses depth-wise convolution [2111.12294]. This reflects a design preference for minimal overhead while preserving input-conditioned modulation.

The representation can be understood in two distinct but related ways. In Wave-MLP and FW-GAN, tokens are amplitude-phase entities, but the implementation remains real-valued through \(\cos\theta\) and \(\sin\theta\) modulation [2111.12294; 2508.21040]. In the Phasor Transformer, by contrast, the state is explicitly geometric: each token is a unit phasor on \(S^1\), and a length-\(N\) sequence lives on the torus \(\mathbb{T}^N\subset\mathbb{C}^N\) [2603.17433]. That formulation makes phase the native state variable rather than a modulation attached to a real-valued feature.

The FW-GAN formulation introduces a useful distinction between phase awareness and frequency modeling. Inside the Wave-MLP, phase awareness is a feature-mixing mechanism based on amplitude/phase modulation, whereas the broader framework’s explicit frequency modeling is handled by a Frequency Distribution Loss and a high-frequency discriminator [2508.21040]. The paper states that the Wave-MLP is not presented as using an FFT in its token mixing, wavelets inside the Wave-MLP, or explicit complex convolutions over Fourier coefficients; its wave parameterization is instead based on complex-valued token representation, sinusoidal decomposition via Euler’s formula, and phase-modulated token aggregation [2508.21040].

The Phasor Transformer pushes the geometric interpretation further. Inputs are amplitude-normalized and mapped into bounded phases by
\[
\phi_t=\frac{x_t}{\max|\boldsymbol{x}|}\cdot\frac{\pi}{2},\qquad t=1,\ldots,T,
\]
then lifted to complex phasors
\[
\boldsymbol{z}_{\mathrm{in}=(e^{i\phi_1},\ldots,e^{i\phi_T})^\top\in\mathbb{T}^N,\quad N=T.
\]
This deliberately suppresses free amplitude variation in the internal state and makes the method especially appropriate for oscillatory or phase-synchronization-dominated time series [2603.17433]. A plausible implication is that phase-aware Wave-MLP designs can vary in whether they treat amplitude and phase symmetrically or prioritize phase as the primary latent variable.

## 5. Relations to other phase-aware and wave-based models

Wave-MLP sits within a wider landscape of phase-aware modeling, but its relation to nearby methods is architectural rather than terminological. The Phasor Transformer is the closest mathematical reference for a phase-native global mixer. It replaces attention with a parameter-free DFT token mixer and trainable pre/post phase shifts, with per-block complexity \(\mathcal{O}(T\log T)\) rather than attention’s \(\mathcal{O}(T^2)\), and trainable parameter count \((2D+1)T\) for depth \(D\) [2603.17433]. It is not a token-mixing MLP, but conceptually it occupies the same design space: deterministic global mixing, lightweight trainable local operations, and explicit oscillatory inductive bias.

For long-horizon wave prediction, “Harnessing Loss Decomposition for Long-Horizon Wave Predictions via Deep Neural Networks” [2412.02924] contributes a phase-sensitive training principle rather than a phase-aware backbone. It decomposes prediction error into dissipation and dispersion components,
\[
\tau_{\text{DISS} = \left[\sigma\left(u_a\right)-\sigma\left(u_d\right)\right]^2 + \left(\bar{u}_a-\bar{u}_d\right)^2,
\]
\[
\tau_{\text{DISP} = 2(1-\rho)\sigma\left(u_a\right)\sigma\left(u_d\right),
\]
and uses them in a weighted objective. The paper does not define phase explicitly via Fourier variables or complex outputs; instead, correlation in physical space serves as a surrogate for phase alignment [2412.02924]. This suggests that a phase-aware Wave-MLP need not always represent phase in the latent state; it may also become phase-aware through its loss design.

“A Phase Shift Deep Neural Network for High Frequency Approximation and Wave Problems” [1909.11759] provides a different MLP-based route. Its PhaseDNN represents high-frequency functions as sums of fixed sinusoidal carriers multiplied by MLP-generated envelopes,
\[
T(x)=\sum_{m=1}^M e^{i\omega_m x}T_m(x)
\quad\text{or}\quad
T(x)=\sum_{m=1}^M A_m(x)\cos(\omega_m x)+B_m(x)\sin(\omega_m x),
\]
so that each subnetwork learns a demodulated low-frequency envelope [1909.11759]. This is phase-aware and MLP-based, but unlike Wave-MLP it uses prescribed frequency channels rather than dynamically estimated token phases.

APEX introduces a still broader view of phase-aware wave modeling by explicitly factorizing complex fields as
\[
u(\mathbf r,\nu)=A(\mathbf r;\nu)\exp\bigl(i\phi(\mathbf r;\nu)\bigr)
\]
and arguing that amplitude remains comparatively stable across frequency shifts while phase deteriorates rapidly [2605.26732]. Its higher-frequency enhancer represents targets as
\[
x_1= \bigl[\log (|u|+\epsilon),\; \sin\phi,\; \cos\phi\bigr]
\]
and conditions generation on a coarse amplitude anchor and a Green’s-function-inspired phase prior [2605.26732]. This is not a Wave-MLP architecture, but it reinforces the broader principle that explicit amplitude/phase factorization can be preferable to undifferentiated real-valued processing in oscillatory domains.

## 6. Empirical results, ablations, and limitations

Wave-MLP was evaluated extensively on ImageNet-1K, COCO, and ADE20K. On ImageNet, Wave-MLP-T\* achieves **80.1%** with **15M** parameters and **2.1G** FLOPs; Wave-MLP-T achieves **80.6%** with **17M / 2.4G**; Wave-MLP-S achieves **82.6%** with **30M / 4.5G**; Wave-MLP-M achieves **83.4%** with **44M / 7.9G**; and Wave-MLP-B achieves **83.6%** with **63M / 10.2G** [2111.12294]. The paper compares these against prior MLP, CNN, and transformer backbones and reports a favorable accuracy-efficiency tradeoff. For example, Wave-MLP-S outperforms Swin-T at the same **4.5G** FLOPs, and Wave-MLP-B slightly exceeds Swin-B with fewer parameters and FLOPs [2111.12294].

On COCO with RetinaNet 1×, the reported AP values are **40.4** for Wave-MLP-T, **43.4** for Wave-MLP-S, **44.8** for Wave-MLP-M, and **44.2** for Wave-MLP-B [2111.12294]. On Mask R-CNN 1×, Wave-MLP-T reports **41.5** box AP / **38.2** mask AP, Wave-MLP-S **44.0 / 40.0**, Wave-MLP-M **45.3 / 41.0**, and Wave-MLP-B **45.7** box AP, with a formatting corruption in the B-row mask AP entry [2111.12294]. On ADE20K with Semantic FPN, the reported mIoU values are **41.2** for Wave-MLP-T, **44.4** for Wave-MLP-S, and **46.8** for Wave-MLP-M [2111.12294]. These results establish that phase-aware Wave-MLP functions not only as an image-classification model but also as a dense-prediction backbone.

The ablation studies directly test the role of phase. In Table 5, removing phase yields **78.8%**, using static phase yields **79.3%**, and using dynamic phase yields **80.1%** on Wave-MLP-T\* [2111.12294]. This is the clearest evidence that the benefit is not merely the introduction of extra parameters; the phase must be input-dependent. A related ablation on phase estimation modules reports **78.8%** for the baseline, **79.3%** for identity, **80.1%** for depth-wise convolution, and **80.6%** for channel-FC [2111.12294]. Window-size ablations report **79.7%** for size 3, **79.8%** for size 5, **80.1%** for size 7, and **80.0%** for all tokens, supporting the local-window choice [2111.12294].

FW-GAN provides task-specific evidence in handwriting synthesis. The ablation comparing the Base BigGAN backbone with and without the Wave-Modulated Generator shows the transition from **(B)** Base + FDL to **(C)** (B) + Wave-Modulated Generator improves FID from **10.20** to **6.89**, CER from **10.74** to **10.32**, NED from **10.50** to **10.12**, and WER from **29.05** to **28.81** [2508.21040]. Because this comparison isolates the effect of the Wave-Modulated Generator, it is the most direct evidence for the usefulness of the phase-aware Wave-MLP inside FW-GAN.

At the same time, the available evidence places clear limits on what can be claimed. FW-GAN does not specify the number of WaveGBlocks, exact channels per stage, MLP hidden expansion ratio, exact PATM equations for height and width branches, exact branch fusion weights, exact normalization inside Wave-MLP, or exact activation function names inside the module [2508.21040]. In Wave-MLP itself, not every branch-fusion detail is written as an explicit equation in the main text [2111.12294]. The Phasor Transformer, while highly relevant conceptually, is only a partial architectural match because it omits standard feed-forward MLPs and uses DFT-based global coupling instead [2603.17433]. These omissions matter for historical and conceptual classification: “Phase-Aware Wave-MLP” is best understood as a family of designs organized by phase-modulated token interaction, not as a single fully standardized block.

A common misconception is to equate phase-aware Wave-MLP with generic complex-valued neural networks. The primary literature does not support that equation. Wave-MLP is implemented with real-valued trigonometric modulation rather than full complex-valued learning [2111.12294], and FW-GAN likewise frames its Wave-MLP as wave-inspired token mixing rather than as FFT-based or wavelet-based processing [2508.21040]. Another misconception is that phase-awareness alone guarantees superior performance. The ablations show that dynamic phase helps relative to no phase or static phase [2111.12294], but the gains depend on the phase estimator, window design, and surrounding architecture. This suggests that phase-aware Wave-MLP is a specific inductive bias, not a universal replacement for all token-mixing schemes.

Source: https://www.emergentmind.com/topics/phase-aware-wave-mlp