Multi-Resolution STFT Loss

• Multi-Resolution STFT Loss is a training objective that leverages multiple STFT analyses at varying resolutions to capture both transient details and slow spectral structures.
• It mitigates the intrinsic time–frequency tradeoff by combining spectral convergence and log-magnitude losses, among other formulations, to enhance waveform fidelity.
• Empirical studies in speech enhancement and ECG reconstruction show that multi-resolution supervision often outperforms single-resolution methods in perceptual quality and performance metrics.

Multi-Resolution STFT Loss denotes a family of training objectives that compare a generated, enhanced, or reconstructed waveform with a target through several short-time Fourier transform analyses computed at different resolutions. Its central purpose is to mitigate the intrinsic time–frequency tradeoff of any single STFT: short windows better resolve transients, whereas long windows better resolve slower spectral structure. In the speech and audio literature, a common formulation averages several STFT-magnitude losses composed of spectral convergence and log-magnitude terms, but the phrase does not designate a single canonical formula. Other domains use simpler direct $L_1$ discrepancies between STFT representations, and some variants supervise only selected components of the STFT rather than the usual magnitude spectrogram (Yamamoto et al., 2019).

1. Conceptual basis

The rationale for multi-resolution supervision is rooted in the limitations of a single STFT parameterization. Several papers state the same core tradeoff in closely related language: increasing window size improves frequency resolution but reduces temporal resolution, whereas smaller windows improve temporal precision but reduce frequency selectivity. A single STFT therefore exposes only one operating point in the time–frequency plane, which can be insufficient for signals whose relevant structure spans multiple scales (Yamamoto et al., 2019).

This issue is especially explicit in applications where morphology is intrinsically multi-scale. In radar-to-ECG reconstruction, the argument is that ECG contains sharp, high-frequency QRS complexes together with slower, lower-frequency P and T waves and baseline structure; accordingly, one STFT window cannot simultaneously constrain all clinically relevant morphology (Kundu et al., 31 Oct 2025). In speech enhancement, the same logic is expressed as a need to preserve clarity, intelligibility, and speech content after denoising while avoiding overfitting to one fixed STFT representation (Wan et al., 2023).

A plausible unifying interpretation is that multi-resolution STFT loss provides several complementary spectral “views” of the same waveform. Short analyses emphasize local temporal events, longer analyses emphasize harmonic or slowly varying structure, and their aggregation constrains waveform synthesis more broadly than either pointwise time-domain error or a single spectrogram loss alone. This interpretation is explicit in waveform generation and speech enhancement papers that present multi-resolution loss as a way to better learn the time-frequency characteristics of the signal and avoid suboptimal waveform-domain behavior under a single fixed analysis (Yamamoto et al., 2019).

2. Canonical formulations and major variants

In the now-standard speech waveform formulation, each resolution contributes two terms computed on STFT magnitudes: spectral convergence and log STFT magnitude loss. In "Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram" (Yamamoto et al., 2019), the single-resolution loss is

$$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$

with

$$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$

and

$$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$

The multi-resolution auxiliary term is then

$$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$

with $M=3$, a Hanning window, and three explicit analysis settings: $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$ where each tuple is $$(\text{FFT size},\text{window size},\text{frame shift})$$ (Yamamoto et al., 2019).

A closely related formulation appears in "Multi-Loss Convolutional Network with Time-Frequency Attention for Speech Enhancement" (Wan et al., 2023). There, the per-resolution loss is likewise

$L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$

with the same spectral convergence and log-magnitude definitions on clean and estimated waveforms, and the multi-resolution combination is

$L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$

The paper is explicit that the varying parameters are FFT size, window size, and frame shift, but it does not provide the actual numeric configurations, the window type, or even the numerical value of $$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$0 (Wan et al., 2023).

Other applications depart materially from this canonical magnitude-spectrogram definition. In "LifWavNet: Lifting Wavelet-based Network for Non-contact ECG Reconstruction from Radar" (Kundu et al., 31 Oct 2025), the waveform term is

$$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$1

and the multi-resolution STFT term is

$$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$2

with Hanning windows $$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$3, $$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$4, and $$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$5. This formulation does not decompose the loss into spectral convergence, log-magnitude, power, or an explicit complex-domain term. The STFT is described as a complex-valued matrix containing magnitude and phase, but the exact treatment of complex values is not further specified (Kundu et al., 31 Oct 2025).

A different nonstandard variant appears in "A High-Fidelity Speech Super Resolution Network using a Complex Global Attention Module with Spectro-Temporal Loss" (Tamiti et al., 30 Jun 2025). The paper explicitly uses a real-valued multiresolution STFT loss,

$$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$6

with $$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$7 resolutions

$$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$8

where the tuples are $$L_{\mathrm{s}}(G)=\mathbb{E}\left[L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})+L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})\right],$$9. The crucial paper-specific detail is that the final method applies this loss only on the real part of the STFT, not on STFT magnitudes in the usual sense, and not on the full complex spectrum (Tamiti et al., 30 Jun 2025).

Formulation family	Per-resolution term	Explicit configurations
Parallel WaveGAN-style	$$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$0 on STFT magnitudes	Three resolutions with Hanning window
MNTFA-style	$$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$1 on waveforms via STFT	Parameters vary, but numeric settings omitted
LifWavNet-style	Direct $$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$2 between STFT representations	Hanning windows 800, 400, 200
CTFT-Net-style	Real-part spectral convergence + log-magnitude	Three resolutions, real-part-only

This range of formulations shows that “MR-STFT loss” is best understood as a design pattern rather than a single fixed objective. In some papers it means multi-resolution magnitude-spectrogram supervision; in others it means direct multi-window STFT matching or even real-part-only supervision.

3. Precursors in single-resolution STFT spectral losses

The multi-resolution literature rests on earlier single-resolution STFT spectral-loss formulations. "STFT spectral loss for training a neural speech waveform model" (Takaki et al., 2018) defines a waveform model trained not by pointwise sample MSE alone, but through STFT amplitude and phase losses. With complex STFT coefficient $$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$3, amplitude $$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$4, and phase $$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$5, the paper uses

$$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$6

and the circular phase loss

$$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$7

The overall spectral loss is

$$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$8

This is not a multi-resolution objective: the paper uses one STFT configuration with frame shift $$L_{\mathrm{sc}}(\mathbf{x},\hat{\mathbf{x}})= \frac{\left\| \left|\mathrm{STFT}(\mathbf{x})\right|-\left|\mathrm{STFT}(\hat{\mathbf{x}})\right| \right\|_{F}} {\left\| \left|\mathrm{STFT}(\mathbf{x})\right| \right\|_{F}},$$9, frame length $$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$0, and FFT size $$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$1 (Takaki et al., 2018).

"Training a Neural Speech Waveform Model using Spectral Losses of Short-Time Fourier Transform and Continuous Wavelet Transform" (Takaki et al., 2019) generalizes this view by expressing both STFT and CWT as a common complex linear transform $$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$2. It retains the same amplitude and phase loss structure and explicitly states that a special case of the CWT matrix includes a transformation to perform STFT. This paper also remains single-resolution on the STFT side, again with frame shift $$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$3, frame length $$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$4, and FFT size $$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$5, but it makes the broader point that complementary time-frequency resolutions can be useful even when obtained through different transforms rather than multiple STFT parameter sets (Takaki et al., 2019).

These papers matter because they supply the mathematical basis later reused by MR-STFT losses: differentiable spectral losses on STFT-derived quantities, explicit gradients back to waveform samples, and the idea that waveform training is better served by structured time-frequency supervision than by samplewise regression alone. The 2018 paper further gives a maximum-likelihood interpretation, viewing amplitude loss as Gaussian and phase loss as von Mises, which clarifies that STFT-domain objectives need not be treated as purely heuristic (Takaki et al., 2018).

4. Integration into broader training objectives

MR-STFT loss is rarely used in isolation. In "Parallel WaveGAN" (Yamamoto et al., 2019), it is combined with a least-squares GAN generator loss,

$$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$6

with $$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$7. The paper is explicit that the STFT term is added to improve the stability and efficiency of adversarial training, and the training schedule freezes the discriminator for the first $$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$8k steps before joint training (Yamamoto et al., 2019).

"Improved parallel WaveGAN vocoder with perceptually weighted spectrogram loss" (Song et al., 2021) keeps the same overall MR-STFT-plus-adversarial structure, but modifies both spectral convergence and log-magnitude components by a frequency-dependent weighting $$L_{\mathrm{mag}}(\mathbf{x},\hat{\mathbf{x}})= \frac{1}{N} \left\|\log \left|\mathrm{STFT}(\mathbf{x})\right|-\log \left|\mathrm{STFT}(\hat{\mathbf{x}})\right|\right\|_{1}.$$9 derived from averaged line spectral frequencies. The weight is created from a time-invariant masking filter

$$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$0

with $$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$1, and its magnitude response is normalized to the range $$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$2 to $$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$3 before being applied to the MR-STFT loss (Song et al., 2021).

In speech enhancement, the composite objectives differ. "Multi-Loss Convolutional Network with Time-Frequency Attention for Speech Enhancement" (Wan et al., 2023) uses

$$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$4

where $$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$5 is a log of the sum of MSE terms on real part, imaginary part, and magnitude spectrograms, $$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$6 is the multi-resolution STFT auxiliary loss, and $$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$7 is a KL divergence between clean and enhanced WavLM embeddings. The paper explicitly states equal weighting across the three terms (Wan et al., 2023).

"LifWavNet" (Kundu et al., 31 Oct 2025) uses a much simpler composition,

$$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$8

with no adversarial loss, no perceptual loss, no physiological loss, no feature-matching loss, no $$L_{\mathrm{aux}}(G)=\frac{1}{M}\sum_{m=1}^{M}L_{\mathrm{s}}^{(m)}(G),$$9 waveform loss in the final model, and no additional subband supervision. The loss is imposed only on the final reconstructed ECG waveform $M=3$0 versus the ground-truth ECG $M=3$1, not on intermediate wavelet coefficients or hidden features (Kundu et al., 31 Oct 2025).

In speech super-resolution, "A High-Fidelity Speech Super Resolution Network using a Complex Global Attention Module with Spectro-Temporal Loss" (Tamiti et al., 30 Jun 2025) combines the real-valued MR-STFT loss with time-domain SI-SDR through an unweighted sum,

$M=3$2

This gives a cross-domain objective: waveform fidelity from SI-SDR and spectral supervision from MR-STFT, even though the spectral supervision is only on the real part of the STFT (Tamiti et al., 30 Jun 2025).

A related but architecturally distinct use appears in "Time-domain Speech Enhancement Assisted by Multi-resolution Frequency Encoder and Decoder" (Shi et al., 2023). There, DEMUCS-style training uses

$M=3$3

with $M=3$4, and the contribution is to redesign the encoder and decoder so that multi-resolution frequency information is better aligned with the network structure rather than merely attached as several losses to one output waveform (Shi et al., 2023).

5. Empirical behavior across tasks

The strongest recurring empirical pattern is that multi-resolution supervision usually outperforms single-resolution supervision when the target signal genuinely contains multiple temporal and spectral scales. In "Parallel WaveGAN" (Yamamoto et al., 2019), the comparison is direct: for ClariNet, moving from single-resolution STFT to multi-resolution STFT improves MOS from $M=3$5 to $M=3$6, and for Parallel WaveGAN the single-resolution adversarial version scores only $M=3$7 while the multi-resolution version reaches $M=3$8. The paper explicitly states that multi-resolution STFT loss-based models showed higher perceptual quality than single STFT loss-based ones (Yamamoto et al., 2019).

In radar-to-ECG reconstruction, the ablation in "LifWavNet" (Kundu et al., 31 Oct 2025) shows a more nuanced result. On the Med-Radar dataset, temporal-only loss yields $M=3$9, MRE $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$0, $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$1, and $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$2; the proposed multi-resolution objective yields $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$3, MRE $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$4, $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$5, and $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$6. One single-window variant attains the lowest MRE, but the proposed loss gives the best overall performance, especially for downstream heart rate and heart-rate-variability estimation (Kundu et al., 31 Oct 2025).

In speech enhancement, "Multi-Loss Convolutional Network with Time-Frequency Attention for Speech Enhancement" (Wan et al., 2023) shows that adding the multi-resolution STFT loss to $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$7 improves PESQ from $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$8 to $$(1024,600,120),\quad (2048,1200,240),\quad (512,240,50),$$9 and STOI from $$(\text{FFT size},\text{window size},\text{frame shift})$$0 to $$(\text{FFT size},\text{window size},\text{frame shift})$$1, but worsens WER from $$(\text{FFT size},\text{window size},\text{frame shift})$$2 to $$(\text{FFT size},\text{window size},\text{frame shift})$$3. When added on top of the WavLM loss, the combined model improves PESQ from $$(\text{FFT size},\text{window size},\text{frame shift})$$4 to $$(\text{FFT size},\text{window size},\text{frame shift})$$5, STOI from $$(\text{FFT size},\text{window size},\text{frame shift})$$6 to $$(\text{FFT size},\text{window size},\text{frame shift})$$7, and WER from $$(\text{FFT size},\text{window size},\text{frame shift})$$8 to $$(\text{FFT size},\text{window size},\text{frame shift})$$9. The paper therefore treats MR-STFT as beneficial for enhancement quality, but not sufficient by itself for ASR robustness (Wan et al., 2023).

In speech super-resolution, "A High-Fidelity Speech Super Resolution Network using a Complex Global Attention Module with Spectro-Temporal Loss" (Tamiti et al., 30 Jun 2025) reports a direct MR-versus-SR comparison. The SR-STFT baseline gives LSD $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$0, STOI $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$1, PESQ $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$2, SI-SDR $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$3, and NISQA-MOS $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$4, whereas the main CTFT-Net configuration using MR-STFT plus SI-SDR gives LSD $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$5, STOI $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$6, PESQ $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$7, SI-SDR $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$8, and NISQA-MOS $L_s=L_{sc}(s,\hat{s})+L_{mag}(s,\hat{s}),$9. The paper summarizes this by stating that the real-valued MR-STFT loss is always better than the real-valued SR-STFT loss for CTFT-Net because MR-STFT captures fine and coarse-grained details from different resolutions (Tamiti et al., 30 Jun 2025).

The DEMUCS-oriented critique in "Time-domain Speech Enhancement Assisted by Multi-resolution Frequency Encoder and Decoder" (Shi et al., 2023) complicates a simple “more resolutions are always better” narrative. Replacing DEMUCS’s standard $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$0 ms STFT-loss resolutions with only stationary $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$1 ms resolutions reduces performance from STOI $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$2, PESQ $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$3 to STOI $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$4, PESQ $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$5. At the same time, the proposed architecture that injects stationary multi-resolution frequency features into the encoder and aligns decoder outputs with specific resolution losses reaches PESQ $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$6, a $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$7 improvement over baseline DEMUCS. The evidence therefore favors multi-resolution supervision, but also suggests that architecture–loss alignment matters (Shi et al., 2023).

6. Ambiguities, misconceptions, and adjacent directions

A common misconception is that MR-STFT loss always means the specific speech-audio formulation of spectral convergence plus log-magnitude on STFT magnitudes. The surveyed papers show otherwise. In some works, that description is exact; in others, the loss is a direct $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$8 difference between STFT representations at several window lengths, and in still others it is a real-part-only objective. The phrase is therefore semantically broader than any one implementation (Kundu et al., 31 Oct 2025).

A second misconception is that MR-STFT loss is necessarily phase-aware. Many practical formulations are not. "Parallel WaveGAN" (Yamamoto et al., 2019) defines its auxiliary loss only on STFT magnitudes and explicitly notes future work such as introducing phase-related loss. "Multi-Loss Convolutional Network with Time-Frequency Attention for Speech Enhancement" (Wan et al., 2023) also uses magnitude-based spectral convergence and log-magnitude terms. By contrast, "LifWavNet" leaves the treatment of complex-valued STFT differences unspecified, and "CTFT-Net" formally uses only the real part of the STFT in its final method (Kundu et al., 31 Oct 2025).

A third misconception is that multi-resolution supervision necessarily dominates single-resolution or purely time-domain criteria on every metric. The available evidence is more specific. In LifWavNet, the proposed loss gives the best overall physiological fidelity, but one single-window loss achieves the lowest MRE (Kundu et al., 31 Oct 2025). In MNTFA, MR-STFT improves PESQ and STOI, yet does not automatically reduce WER unless paired with the WavLM loss (Wan et al., 2023). In the DEMUCS study, simply changing the resolution set is not sufficient; the architecture must be redesigned to exploit the supervision effectively (Shi et al., 2023).

The literature also contains adjacent STFT-domain objectives that are not MR-STFT losses but illuminate its boundaries. "An Explicit Consistency-Preserving Loss Function for Phase Reconstruction and Speech Enhancement" (Ku et al., 2024) does not compare predicted and target spectra across multiple resolutions. Instead, it penalizes whether a predicted complex spectrogram is a consistent STFT representation of a real signal: $L_{aux}=\frac{1}{M}\sum_{m=1}^{M}L_s^{(m)}.$9 This is a single-resolution validity constraint rather than a reference-matching loss. A plausible implication is that MR-STFT and explicit consistency losses are complementary: the former asks what the output should resemble across several scales, while the latter asks whether the predicted complex STFT is even realizable as a waveform (Ku et al., 2024).

Taken together, these papers establish Multi-Resolution STFT Loss as a flexible class of time-frequency supervision strategies rather than a single universally standardized objective. Its enduring appeal is the same across domains: to constrain waveform learning simultaneously at several temporal and spectral scales. Its exact algebra, however, remains application-dependent, and the most consequential design choices often concern not only the loss formula itself but also its interaction with phase handling, auxiliary objectives, and model architecture.