Phase-Aware Input Masking Overview

• Phase-aware input masking is a set of techniques that exploit phase-sensitive information rather than relying solely on magnitude data across multiple domains.
• In speech enhancement, methods like complex ratio mask estimation and two-stream networks demonstrate improved perceptual quality and signal recovery.
• Applications in language models and optical imaging reveal that phase-aware strategies dynamically adjust inference and hardware design to enhance overall system performance.

Searching arXiv for the provided topic and papers to ground the article in current arXiv records. Phase-aware input masking denotes masking strategies in which the mask is designed to preserve, estimate, or exploit phase-related structure rather than relying exclusively on magnitude or amplitude information. In the arXiv literature represented here, the term has distinct technical meanings across domains. In speech enhancement and separation, it typically refers to masking in the complex short-time Fourier transform (STFT) domain so that both magnitude and phase are corrected, rather than reusing the noisy phase (Goswami et al., 2020). In LLMs, it refers to masking that changes with the operational phase of inference, specifically the prefill phase versus autoregressive generation (Katz et al., 2024). In optical imaging, it refers to pupil-plane phase masks implemented with liquid-crystal geometric phase patterns for multiplexed holographic aperture masking (Doelman et al., 2018). This diversity of usage makes the topic less a single algorithm than a family of masking paradigms organized around phase sensitivity.

1. Conceptual scope and terminology

A recurring premise in the cited work is that magnitude-only masking is often inadequate when phase carries task-relevant structure. In speech enhancement, this premise appears as a critique of methods that estimate only the magnitude spectrum of clean speech and then reuse the noisy phase during waveform synthesis. The alternative is to estimate a complex mask, such as the Complex Ratio Mask (CRM) or complex ideal ratio mask (cIRM), so that the real and imaginary parts of the STFT are both modified (Goswami et al., 2020). Related work further argues that phase recovery is important for perceptual quality, and that explicitly estimating phase offsets or complex targets can outperform magnitude-only formulations (Zhang et al., 2023).

A second usage appears in transformer language modeling. There, “phase-aware” does not refer to Fourier phase. Instead, it refers to the distinction between the prompt-processing “prefill” phase and the token-by-token generation phase. Segment-Based Attention Masking treats prompt segments as units during prefill, allowing within-segment bidirectional attention, while restoring conventional causal masking during generation (Katz et al., 2024). A plausible implication is that the phrase “phase-aware” is best interpreted contextually: in speech it usually concerns signal phase, whereas in transformers it concerns inference phase.

A third usage appears in multiplexed holographic aperture masking, where “phase-aware” refers to explicit control of optical phase by liquid-crystal geometric phase masks. The phase shift is given by

$\phi = \pm 2\theta,$

where $\theta(x,y)$ is the local orientation of the fast axis of the retarder, with the sign determined by circular polarization (Doelman et al., 2018).

Domain	Masked quantity	Representative formulation
Speech enhancement and separation	Complex STFT, cIRM, phase correction	CRM estimation, normalized cIRM, two-stream amplitude/phase masking
LLMs	Attention matrix during prefill and generation	Segment-based masking with within-segment bidirectional attention in prefill
Optical imaging	Pupil-plane phase pattern	Geometric phase masks for multiplexed holographic aperture masking

2. Complex masks and phase-sensitive targets in speech enhancement

The canonical formulation in phase-aware speech enhancement is the complex mask applied to STFT coefficients. For noisy speech $x[n] = s[n] + v[n]$, with STFTs $X[n,k]$ and $S[n,k]$, the CRM is defined as

$M[n,k] = \frac{S[n,k]}{X[n,k]}.$

Its real and imaginary parts are

$$M_r[n,k] = \frac{X_r[n,k] S_r[n,k] + X_i[n,k] S_i[n,k]}{X_r^2[n,k] + X_i^2[n,k]},$$

$$M_i[n,k] = \frac{X_r[n,k] S_i[n,k] - X_i[n,k] S_r[n,k]}{X_r^2[n,k] + X_i^2[n,k]}.$$

To avoid unbounded behavior when $X[n,k] \approx 0$, the mask can be compressed as

$B[n,k] = \tanh(M_r[n,k]) + j\tanh(M_i[n,k]),$

so that both components lie in $\theta(x,y)$0 (Goswami et al., 2020).

Later work reformulates the same objective in polar coordinates. For

$\theta(x,y)$1

the cIRM can be written as

$\theta(x,y)$2

This enables a decoupled prediction strategy: estimate a magnitude mask $\theta(x,y)$3 for $\theta(x,y)$4, and estimate a normalized cIRM $\theta(x,y)$5 through its cosine and sine components. In MPCRN, the last decoder output has three channels: one channel, after sigmoid, gives the magnitude mask $\theta(x,y)$6; two channels, after $\theta(x,y)$7, estimate $\theta(x,y)$8 and $\theta(x,y)$9 (Zhang et al., 2023). Because these two outputs may not satisfy unit norm exactly, triangle correction is applied before reconstructing enhanced magnitude and phase.

PHASEN adopts a related but architecturally different factorization. Rather than directly regressing a cIRM, it predicts a real-valued amplitude mask $x[n] = s[n] + v[n]$0 and a unit-modulus complex phase correction $x[n] = s[n] + v[n]$1, and forms the enhanced spectrogram as

$x[n] = s[n] + v[n]$2

with $x[n] = s[n] + v[n]$3 and $x[n] = s[n] + v[n]$4 (Yin et al., 2019). This formulation makes amplitude scaling and phase rotation explicit. A common misconception addressed by these papers is that phase awareness necessarily requires direct Cartesian regression of real and imaginary mask components. The cited results suggest that normalized phase targets, unit-modulus phase rotation, and explicit amplitude/phase separation are all viable alternatives.

3. Architectural realizations: complex recurrence, parallel sequence modeling, and two-stream networks

The paper “Phase Aware Speech Enhancement using Realisation of Complex-valued LSTM” introduces a realization of a complex-valued LSTM using a pair of real-valued LSTMs interconnected to model complex arithmetic. For a complex input sequence $x[n] = s[n] + v[n]$5, the outputs are

$x[n] = s[n] + v[n]$6

$x[n] = s[n] + v[n]$7

This preserves dependencies between the real and imaginary parts of the CRM. The architecture uses a context window of 21 frames, stacks two RCLSTM layers, with 64 units in the first layer and 257 units in the second layer, and uses a complex dense layer with $x[n] = s[n] + v[n]$8 activation to estimate the bounded CRM (Goswami et al., 2020). The motivation is explicit: feed-forward neural networks had been used for complex ratio masking, but they cannot capture the sequential information considered essential for phase estimation.

MPCRN reaches a similar phase-aware endpoint without complex-valued recurrence. Its central architectural addition is the Parallel Sequence Modeling (PSM) block, which replaces the standard recurrent component in a CRN-based speech enhancement model. The temporal branch uses a GRU along the time dimension, while the spectral branch uses a BiGRU along the frequency dimension; both branches use LayerNorm and PReLU, and their outputs are fused by addition and a $x[n] = s[n] + v[n]$9 convolution (Zhang et al., 2023). The paper’s stated rationale is that standard CRN architectures model only the time axis and neglect possible correlations along frequency.

PHASEN addresses phase prediction through a two-stream design rather than a complex-valued layer or a single real-valued branch. An amplitude stream predicts an amplitude mask, a phase stream predicts a complex phase correction, and the two streams exchange information through attention-like cross-stream communication. In each two-stream block,

$X[n,k]$0

with

$X[n,k]$1

PHASEN also inserts Frequency Transformation Blocks (FTBs), which apply a learned frequency transformation matrix across all frequencies at each time step to capture long-range harmonic correlations (Yin et al., 2019). The paper reports that the learned transformation matrix spontaneously captures harmonic correlation.

Taken together, these architectures define three distinct responses to the same difficulty: native complex recurrence for real/imaginary coupling, parallel real-valued sequence modeling for time and frequency dependencies, and amplitude/phase stream factorization with explicit cross-stream communication.

4. Consistency constraints and learning objectives

A major technical objection to unconstrained complex masking is that not every complex-valued matrix corresponds to the STFT of a real time-domain signal. The consistency formulation in “End-to-End Model for Speech Enhancement by Consistent Spectrogram Masking” states

$X[n,k]$2

unless the spectrogram is consistent (Du et al., 2019). The paper argues that inconsistent spectrograms enlarge the solution space and cause unintended artifacts. Its proposed Consistency Spectrogram Masking (CSM) predicts separate masks for real and imaginary parts,

$X[n,k]$3

but optimizes the model in the time domain:

$X[n,k]$4

This enforces consistency by construction and is reported to accelerate training and improve speech quality.

RCLSTM uses a complex-domain mean-squared objective between estimated and target bounded masks,

$X[n,k]$5

where $X[n,k]$6 denotes complex conjugate (Goswami et al., 2020). MPCRN uses a composite objective consisting of a magnitude loss and a complex-valued loss:

$X[n,k]$7

with $X[n,k]$8 typically (Zhang et al., 2023). PHASEN uses

$X[n,k]$9

where $S[n,k]$0 is an MSE on amplitude-compressed magnitudes and $S[n,k]$1 is an MSE on amplitude-compressed complex STFTs (Yin et al., 2019).

Speaker separation introduces an additional complication: the permutation ambiguity between output streams and target speakers. “Mask-dependent Phase Estimation for Monaural Speaker Separation” proposes mask-dependent permutation invariant training (PIT), where the permutation is selected using only the mask loss,

$S[n,k]$2

and then used for phase training (Ni et al., 2019). The same paper proposes weighted phase losses, including Inverse Magnitude Weighted Loss and a joint weighted loss, to emphasize T-F regions where phase estimation is harder or more consequential. This suggests that phase-aware masking is not only a question of target parameterization but also of where the optimization allocates statistical emphasis.

5. Empirical performance across speech enhancement, ASR augmentation, and separation

On noisy speech mixtures formed from the Voice-Bank corpus and DEMAND database, RCLSTM was compared with SMM, CIRM, and RILSTM. The reported PESQ scores were 2.62 for RCLSTM, 2.51 for SMM, 2.49 for CIRM, and 2.55 for RILSTM. The paper states that RCLSTM improves PESQ by over 4.3% relative to real-value-based masking methods, with relative improvement most pronounced at low SNRs, reaching up to 43% at 2.5 dB SNR. It also reports the highest PESQ among Wiener, SEGAN, MMSE-GAN, and RCLSTM, and leading performance in SSNR and MOS measures including CSIG, CBAK, and CMOS (Goswami et al., 2020).

MPCRN reports strong performance on VoiceBank+DEMAND in a causal setup, with WB-PESQ 2.96, COVL 3.56, and CBAK 3.50, using 2.09M parameters. The paper attributes the result to the combination of magnitude-and-phase-aware masking and the PSM block, and reports a real-time factor of 0.12 with low computational cost, stated as less than 3M parameters (Zhang et al., 2023). PHASEN reports a 1.76 dB SDR improvement on the AVSpeech + AudioSet dataset and significant gains over Google’s network on that dataset; on Voice Bank + DEMAND it is reported to outperform previous methods on four metrics (Yin et al., 2019).

Phase-aware masking is also used for data augmentation rather than enhancement. PhasePerturbation applies randomization, frequency masking, and temporal masking directly to the phase spectrum,

$S[n,k]$3

while keeping the original amplitude spectrum for resynthesis (Lei et al., 2023). On wav2vec 2.0 fine-tuning with the TIMIT corpus, the paper reports a 10.9% relative reduction in WER compared with no augmentation, and additional improvements of 12.9% and 15.9% in WER when complementing VTLP and SpecAug. In the reported table, the LARGE LV-60K baseline improves from 20.1 WER to 17.9 with PhasePerturbation and to 16.9 with PhasePerturbation+SpecAug (Lei et al., 2023).

For monaural speaker separation on WSJ0-2mix, direct phase prediction on top of a chimera++ mask estimator reaches 13.6 dB SDR with mask-dependent PIT, joint weighted loss, and curriculum learning, compared with a 10.5 dB SDR baseline using mixture phase (Ni et al., 2019). Across these studies, a consistent empirical pattern is that phase-aware masking is most useful when phase is either explicitly predicted, explicitly regularized, or explicitly perturbed for robustness.

6. Cross-domain reinterpretations: transformer attention phases and optical phase masks

In transformer LLMs, phase-aware masking is reformulated as attention masking that depends on the known operational phase. Segment-Based Attention Masking defines a prefill-phase mask

$S[n,k]$4

so that tokens within the same segment can attend to each other in a fully bidirectional manner during prompt processing. Once generation begins, the model returns to the standard causal mask

$S[n,k]$5

The paper states that this Segment-by-Segment scheme entails no additional computational overhead, can be integrated into models such as Llama and Qwen, and consistently achieves state-of-the-art performance on a suite of commonsense reasoning benchmarks (Katz et al., 2024). A common ambiguity is terminological: unlike speech enhancement, the “phase” here is the inference phase, not the phase of a complex signal.

Optical masking provides a different generalization again. In holographic aperture masking, phase-aware masking is realized through geometric phase control in liquid-crystal devices. The holographic phase for multiplexing is given by

$S[n,k]$6

where scaled holograms are summed to create multiple point-spread-function copies (Doelman et al., 2018). The cited results emphasize increased throughput, expanded uv-coverage, and improved calibration relative to sparse aperture masking, with the tradeoff shifted to detector space. Design examples include 18/36 subapertures yielding 18 baselines and 6 closure phases, and 31/36 subapertures yielding 33 baselines and 12 closure phases. Broadband operation up to about 30% bandwidth is reported without overlapping PSFs, while small leakage below 0.2% from imperfect half-wave retardance is noted (Doelman et al., 2018).

These cross-domain cases clarify that “phase-aware input masking” is not a single standardized term. In the cited literature, it names a broader design principle: the mask is conditioned on phase-sensitive structure that standard magnitude-only or uniformly causal masking ignores. In speech that structure is the complex STFT phase; in GPTs it is the distinction between prefill and decoding; in holographic imaging it is the geometric phase of polarized light.