Phase-Coherent Transformer (PCT)

• Phase-Coherent Transformer (PCT) is a family of models that preserve relative phase information using complex-valued attention and deterministic gating mechanisms.
• The design replaces conventional softmax attention with smooth real gates or DFT-based mixing to maintain phase fidelity and avoid token competition.
• PCT applications range from spectral sequence modeling and deep reinforcement learning for trajectory control to phase-aligned circuit crossovers.

Phase-Coherent Transformer (PCT) denotes a family of transformer formulations organized around the preservation of phase relations during computation. In the most direct usage, PCT is the complex-valued architecture introduced in "Complex-Valued Phase-Coherent Transformer," which replaces row-normalized softmax attention with token-non-competing attention generated by a smooth real gate applied to L2-normalized complex query-key similarities (Hioki, 11 May 2026). In adjacent usage, the label is also mapped to the phase-native Phasor Transformer and its stacked Large Phasor Model (LPM), which represent sequence states on the unit circle $S^1$ and replace explicit attention maps with deterministic Discrete Fourier Transform (DFT) token coupling (Sigdel, 18 Mar 2026); to a Gated Transformer-XL (GTrXL) reinforcement-learning policy that maintains coherent memory across mission phases in spacecraft trajectory optimization (Jain et al., 14 Nov 2025); and, in a circuit-theoretic sense, to the Resonant Transformer Router (RTR), a transformer-based lossless crossover whose low-frequency and high-frequency branches are phase-aligned and linearly complementary (Li et al., 10 Sep 2025). Across these usages, phase coherence refers to preserving relative phase, temporal continuity across regime changes, or constructive phase alignment under recombination.

1. Terminology and scope

The term does not denote a single universally standardized architecture. Its most specific meaning is the complex-valued attention mechanism of the 2026 PCT paper, but other recent works use the phrase interpretively to describe architectures whose central inductive bias is coherent phase evolution rather than conventional token competition or manually segmented control logic.

Usage of PCT	Defining mechanism	Representative source
Complex-valued PCT	Smooth real gate on L2-normalized complex similarities; token-non-competing attention	(Hioki, 11 May 2026)
Phase-native PCT	$S^1$ states with trainable phase shifts and DFT token coupling	(Sigdel, 18 Mar 2026)
Control-oriented PCT	GTrXL-PPO with segment-level recurrence across mission phases	(Jain et al., 14 Nov 2025)
Circuit-level PCT	Complementary LF/HF transformer crossover with matched phase response	(Li et al., 10 Sep 2025)

This multiplicity of usage is important for interpretation. A common misconception is that PCT necessarily refers to a complex-valued neural attention layer. The literature summarized here shows a broader pattern: the phrase is also applied to phase-native spectral sequence models, to phase-coherent policy memory in multi-regime control, and to transformer hardware whose branches exhibit identical phase kernels. A plausible implication is that "phase-coherent transformer" functions partly as a design principle and partly as a specific model name, depending on context.

2. Complex-valued attention and the formal PCT architecture

In its canonical neural-network form, PCT is a complex-valued Transformer whose attention mechanism is explicitly designed to preserve phase information across layers. The motivation is that softmax attention introduces row-wise normalization, forcing weights within each query row to sum to one and thereby creating token competition. In complex-valued models, where values carry phase information, this competition can suppress negatively aligned components and interfere with phase superposition. PCT therefore replaces softmax with a smooth, real, element-independent gate applied to normalized complex similarities (Hioki, 11 May 2026).

The token representation is complex: $x_i \in \mathbb{C}^d$, with complex projections

$$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$

where $W_q, W_k, W_v \in \mathbb{C}^{d\times d}$. Queries and keys are L2-normalized at the vector level,

$$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$

and scored using the real part of the Hermitian inner product,

$$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$

Attention weights are then produced by

$\alpha_{ij} = \sigma(s_{ij}+b),$

where $\sigma$ is the logistic sigmoid and $b$ is initialized to $S^1$0. Aggregation is non-competing:

$S^1$1

Because $S^1$2 is real and the value path remains complex-linear, phases in $S^1$3 are preserved and can superpose without a softmax denominator coupling tokens within a row.

The paper distinguishes two notions of coherence. L1, or per-layer phase coherence, requires that each attention layer preserve phase relationships without cross-token coupling from row normalization. L2, or all-layer cascade phase stability, requires that stacked layers preserve phase information without depth-dependent accumulation of noise. These properties are tied to four gate conditions: C1 real-valued output, C2 boundedness on the operating range $S^1$4, C3 smoothness with nonzero gradient on the operating range, and C4 element independence. Under C1 and C4, Theorem 1 states per-layer phase coherence; under L1, C2, C3, L2 normalization of $S^1$5, and a non-expansive Transformer substrate, Theorem 2 gives an $S^1$6-independent Lipschitz bound for per-token phase perturbations (Hioki, 11 May 2026).

Architecturally, PCT uses native complex linear layers for $S^1$7, $S^1$8, $S^1$9, and output, complex RMSNorm in pre-norm, ModReLU in the feedforward network, and rotary positional encodings applied to $x_i \in \mathbb{C}^d$0 and $x_i \in \mathbb{C}^d$1 in every layer. Its computational complexity is identical in asymptotic time and space to softmax attention, namely $x_i \in \mathbb{C}^d$2 for full attention. The novelty is therefore not subquadratic scaling but a different attention law: PCT replaces row-normalized token competition with token-non-competing gating while keeping the standard Transformer scaffold in complex arithmetic.

The comparison with other attention mechanisms is structurally precise. Softmax violates C4 because its denominator couples tokens. ReLU-like gates violate C3 because they have zero gradient for $x_i \in \mathbb{C}^d$3 and delete anti-phase information. Unbounded gates violate C2 by amplifying per-layer effects excessively. PCT’s design is thus presented not as a heuristic but as an operating-range-constrained mechanism whose phase-preserving behavior is both mathematically characterized and experimentally stress-tested.

3. Phase-native sequence modeling on $x_i \in \mathbb{C}^d$4: the Phasor Transformer and LPM

A distinct but related line of work realizes phase coherence by making phase itself the primary state variable. The Phasor Transformer represents token states on the unit-circle manifold

$x_i \in \mathbb{C}^d$5

with a single token written as $x_i \in \mathbb{C}^d$6 and a sequence of length $x_i \in \mathbb{C}^d$7 represented on the $x_i \in \mathbb{C}^d$8-torus

$x_i \in \mathbb{C}^d$9

Given an input window $$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$0, the model encodes bounded angles by

$$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$1

and lifts them to phasors

$$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$2

This ensures unit-modulus coordinates at network input (Sigdel, 18 Mar 2026).

The core block combines trainable phase-shift gates with parameter-free DFT mixing. The trainable gates are diagonal unit-modulus rotations

$$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$3

and the global mixer is the length-$$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$4 DFT

$$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$5

implemented in $$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$6 by FFT. A single block is

$$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$7

so that, for a phasor state $$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$8, one obtains $$q_i = W_q x_i,\quad k_j = W_k x_j,\quad v_j = W_v x_j,$$9. In angle form the trainable phase update is $W_q, W_k, W_v \in \mathbb{C}^{d\times d}$0, and in complex form

$W_q, W_k, W_v \in \mathbb{C}^{d\times d}$1

Only the pre- and post-shifts are trainable, contributing $W_q, W_k, W_v \in \mathbb{C}^{d\times d}$2 parameters per block.

The defining claim of this architecture is that it discards learned pairwise attention maps and replaces them with deterministic frequency-domain coupling. Because $W_q, W_k, W_v \in \mathbb{C}^{d\times d}$3 is parameter-free and unitary, it provides exact global coupling at $W_q, W_k, W_v \in \mathbb{C}^{d\times d}$4 complexity and avoids explicit $W_q, W_k, W_v \in \mathbb{C}^{d\times d}$5 attention maps. The paper explicitly contrasts this with conventional self-attention, which computes $W_q, W_k, W_v \in \mathbb{C}^{d\times d}$6 and incurs $W_q, W_k, W_v \in \mathbb{C}^{d\times d}$7 pairwise interaction and attention-map storage. The Phasor Transformer therefore occupies a different efficiency-performance point: fewer trainable parameters, deterministic spectral mixing, and geometry-aligned phase computation rather than content-dependent pairwise scoring (Sigdel, 18 Mar 2026).

Deep composition is realized in the Large Phasor Model (LPM), which stacks $W_q, W_k, W_v \in \mathbb{C}^{d\times d}$8 such blocks and inserts an inter-block pull-back normalization

$W_q, W_k, W_v \in \mathbb{C}^{d\times d}$9

This normalization re-folds raw angles into a bounded principal interval after intermediate states leave $$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$0 under linear mixing. The resulting composition is

$$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$1

followed by angle pull-back before the next block. Per block, trainable parameters equal $$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$2; for depth $$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$3, parameters scale as $$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$4 when including a readout phase projection. Memory footprint avoids $$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$5 attention maps and favors $$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$6-$$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$7 structures.

The paper also proposes a phase-locking value

$$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$8

with $$\tilde q_i = \frac{q_i}{\|q_i\|_2},\qquad \tilde k_j = \frac{k_j}{\|k_j\|_2},$$9, as a coherence metric over tokens. This suggests a direct quantitative bridge between geometric state constraints and synchronization-oriented analysis in oscillatory time-series. The broader conceptual point is that phase coherence here is structural: state geometry on $$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$0, globally coherent interference under a unitary mixer, and bounded inter-block re-embedding all make relative phase a first-class inductive bias rather than a latent by-product of learned real-valued embeddings.

4. Phase coherence as persistent memory in multi-phase control

In the spacecraft trajectory-optimization literature, the phrase is used in a different but related sense. "Multi-Phase Spacecraft Trajectory Optimization via Transformer-Based Reinforcement Learning" does not coin the term PCT, but its GTrXL-PPO policy is explicitly described as achieving phase coherence across launch, ascent, stage separation, and orbit insertion by maintaining stable policy behavior and persistent memory as dynamics, objectives, and constraints change across regimes (Jain et al., 14 Nov 2025).

The backbone is an encoder-only GTrXL serving both actor and critic. The policy maintains a sliding window $$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$1 of past hidden states with length $$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$2, attends over $$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$3, and updates memory by

$$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$4

Transformer-XL recurrence thus carries memory across segments instead of compressing all history into a single recurrent state. Attention scores incorporate relative positions,

$$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$5

so the policy can reason about how long ago events occurred and maintain coherence over variable-length phases. To stabilize reinforcement learning, residuals are replaced by learnable gates,

$$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$6

with positively initialized $$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$7 to bias early training toward identity mappings.

The policy is trained with PPO. The paper gives the clipped surrogate objective

$$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$8

with total loss

$$s_{ij} = \Re\left(\tilde q_i^{\ast\top} \tilde k_j\right)\sqrt{d}\in[-\sqrt{d},\sqrt{d}].$$9

Advantages use GAE with $\alpha_{ij} = \sigma(s_{ij}+b),$0 and $\alpha_{ij} = \sigma(s_{ij}+b),$1. The observation design is part of the coherence mechanism: normalized global time or time remaining, and when available a phase index $\alpha_{ij} = \sigma(s_{ij}+b),$2, are included so that attention can bind regime information to trajectory history without manual phase switching.

Empirically, this interpretation of PCT is tied to long-horizon control rather than complex arithmetic. Memory length is $\alpha_{ij} = \sigma(s_{ij}+b),$3 in single-phase tasks and $\alpha_{ij} = \sigma(s_{ij}+b),$4 in rocket ascent, which at $\alpha_{ij} = \sigma(s_{ij}+b),$5 s corresponds to approximately $\alpha_{ij} = \sigma(s_{ij}+b),$6 minutes of context. In the four-phase rocket-ascent problem, PPO trained for $\alpha_{ij} = \sigma(s_{ij}+b),$7 updates, training reward reached approximately $\alpha_{ij} = \sigma(s_{ij}+b),$8, and final orbital parameters were within approximately $\alpha_{ij} = \sigma(s_{ij}+b),$9 of targets, with $\sigma$0 error approximately $\sigma$1, $\sigma$2 error approximately $\sigma$3, and $\sigma$4 error approximately $\sigma$5 (Jain et al., 14 Nov 2025). The paper treats the absence of manual phase switching, the continuity of steering across staging events, and the retention of pre-separation guidance patterns as evidence of phase coherence. A plausible implication is that, in control, phase coherence refers less to harmonic phase and more to temporally coherent policy state across discontinuous dynamics.

5. Circuit-level phase-coherent transformers: the Resonant Transformer Router

A fourth usage appears outside neural sequence modeling. RTR is a transformer-based, passive, near-lossless crossover whose low-frequency and high-frequency outputs satisfy

$\sigma$6

so that the original signal can be perfectly reconstructed by linear summation of the two channels (Li et al., 10 Sep 2025). Although the paper does not use the term PCT, it explicitly frames RTR as implementing the same principle because the LF and HF outputs share the same phase response and can be recombined with $\sigma$7 phase alignment at the crossover frequency.

In the idealized derivation, a source excites a series combination of capacitor $\sigma$8 and primary magnetizing inductance $\sigma$9. The LF output is taken across the capacitor and the HF output from the transformer secondary, which under unity turns ratio equals the inductor branch voltage. By Kirchhoff’s voltage law,

$b$0

hence

$b$1

and therefore $b$2. With series resistance $b$3,

$b$4

the transfer functions are

$b$5

In the lossless case $b$6, their sum is exactly $b$7; for $b$8, the deviation is

$b$9

Phase coherence follows from the shared denominator. Evaluated at $S^1$00,

$S^1$01

$S^1$02

With appropriate polarity choice, both outputs have phase

$S^1$03

so $S^1$04. Their group delays are therefore equal. The crossover frequency is approximately

$S^1$05

The paper reports complementarity within numerical precision, insertion loss near $S^1$06 dB, near-perfect phase alignment for RTR, and phase deviations below $S^1$07 under $S^1$08 component variations, contrasting this with conventional LC crossover behavior (Li et al., 10 Sep 2025).

This usage broadens the meaning of "transformer" from attention architecture to electromagnetic device. It also shows that phase coherence can be construed operationally rather than representationally: two branches are coherent not because a learned model preserves latent phases, but because circuit transfer functions share a common phase kernel and support exact reconstruction by linear recombination. The conceptual continuity with neural PCTs lies in the rejection of destructive competition in favor of additive, phase-consistent superposition.

6. Empirical profile, comparisons, and limitations

Across the neural formulations, PCT-type designs are defined as much by their failure modes as by their positive results. In the complex-valued PCT, under parameter-fair comparison, the model consistently outperforms both the standard softmax Transformer and its direct complex-valued counterpart across tasks including long-range memory, hierarchical long-range reasoning, positional retrieval, phase-based memory and superposition, and image classification (Hioki, 11 May 2026). The headline numbers are explicit: Copy $S^1$09 gives PCT $S^1$10 versus real softmax $S^1$11 and complex softmax $S^1$12; Copy $S^1$13 gives $S^1$14 versus $S^1$15 and $S^1$16; NIAH $S^1$17 gives $S^1$18 while both real and complex softmax baselines give $S^1$19; LRA-ListOps mid $S^1$20 gives $S^1$21 versus $S^1$22 and $S^1$23. PCT is also reported as uniquely robust across learning rates $S^1$24, $S^1$25, $S^1$26 and batches $S^1$27, $S^1$28, $S^1$29 on Copy, and it shows no accuracy collapse across depths $S^1$30-$S^1$31.

The ablations are central to interpretation. Gates that violate C3 by deleting anti-phase information, such as ReLU and clamped ReLU, collapse on phase-sensitive and long-range tasks; the paper reports clamped ReLU at approximately $S^1$32 on Copy $S^1$33-$S^1$34 and ReLU at approximately $S^1$35 on Copy $S^1$36. Gates that violate C2 by becoming excessively large, such as the cubic gate, partially collapse, achieving $S^1$37 on Copy $S^1$38. By contrast, softplus, which violates C2 on $S^1$39 but remains bounded on the operating range under L2 normalization, reaches $S^1$40 on Copy $S^1$41. The paper therefore identifies anti-phase deletion as the dominant failure mode and unbounded magnitudes as a secondary but independent degradation source (Hioki, 11 May 2026).

The phase-native Phasor Transformer presents a different empirical frontier. On a short-context synthetic multi-frequency autoregressive benchmark with $S^1$42, a single phasor block with $S^1$43 trainable angles achieved test MSE approximately $S^1$44, while training MSE decreased from $S^1$45 to $S^1$46 over $S^1$47 epochs. A PyTorch self-attention baseline achieved approximately $S^1$48 test MSE with more than $S^1$49 parameters and $S^1$50 mixing. On a longer-context benchmark with $S^1$51, the phasor model achieved MAE approximately $S^1$52 with $S^1$53 trainable angles, while a $S^1$54-head self-attention baseline achieved approximately $S^1$55 MAE with $S^1$56 parameters; the parameter ratio was approximately $S^1$57 in favor of the phasor design. In a deep-stack study with $S^1$58 and $S^1$59, inserting $S^1$60 reduced MSE from $S^1$61 to $S^1$62, and $S^1$63-step autoregressive rollouts were stable and tracked target dynamics qualitatively (Sigdel, 18 Mar 2026). These results support the paper’s stated efficiency-performance frontier but also delimit applicability: learned attention remains more accurate on the reported synthetic benchmarks.

The control-oriented formulation likewise states its limitations explicitly. It reports near-optimal behavior on single-phase double-integrator and Van der Pol benchmarks, seamless multi-phase waypoint navigation, and stable convergence on four-phase rocket ascent, but it also notes that inference latency is not reported, training cost details are not provided, no ablation results are reported, and sensitivity to attention horizon remains unresolved for very long missions (Jain et al., 14 Nov 2025). The circuit formulation has its own practical constraints: complementarity degrades if branches are loaded rather than high-impedance buffered, reactive peaking near resonance must be managed through $S^1$64 control, and careful transformer design is required to maintain high coupling and low parasitics (Li et al., 10 Sep 2025).

Taken together, these works support several nontrivial distinctions. First, PCT is not inherently synonymous with subquadratic attention: the complex-valued PCT retains full $S^1$65 attention, whereas the phasor variant achieves $S^1$66 through FFT-based mixing. Second, phase coherence is not tied to a single mathematical object: it may mean preservation of negatively aligned complex components across attention layers, bounded phase evolution on $S^1$67, memory continuity across regime transitions, or exact phase-aligned recombination in analog hardware. Third, the term names both a specific architecture and a broader design principle. A plausible implication is that future work will continue to treat phase coherence less as a domain-specific trick than as a general criterion for stable superposition, long-range temporal consistency, and non-destructive aggregation across layers or branches.