Multi-Scale Recurrence Discriminator (MS-RD)

• The paper introduces MS-RD as a discriminator that leverages multi-scale amplitude-based recurrence plots and depthwise-separable 2-D CNNs to capture self-similarity dynamics in speech.
• MS-RD operates on downsampled waveform views to assess recurrence geometry, complementing discriminators that focus on Lyapunov, amplitude, and phase statistics.
• Empirical results indicate that integrating MS-RD can reduce parameter counts by up to 45× while maintaining or improving spectral fidelity and perceptual quality in bandwidth extension.

Searching arXiv for the cited papers and closely related context. The Multi-Scale Recurrence Discriminator (MS-RD, or MSRD) is a discriminator architecture for adversarial speech bandwidth extension that uses recurrence representations to model self-similarity dynamics across multiple temporal scales. In the 2025 speech-bandwidth-extension literature, MSRD is introduced within nonlinear dynamical-systems-inspired GAN frameworks as a critic that asks whether a generated waveform revisits similar states with the same geometry, density, and multi-scale structure as real speech. Its defining mechanism is the construction of binary recurrence plots from downsampled waveform views, followed by lightweight depthwise-separable 2-D CNN processing. In this literature, MSRD is positioned as a compact alternative to heavy waveform-domain periodic critics such as the Multi-Period Discriminator (MPD), while remaining complementary to discriminators that target Lyapunov behavior, amplitude statistics, and phase statistics (Tamiti et al., 21 Jul 2025, Tamiti et al., 1 Oct 2025).

1. Conceptual origin and research status

MSRD emerges from the claim that speech production is best viewed as a nonlinear dynamical system. The motivating observations are that the vocal folds form a driven, self-sustained, viscous–elastic oscillator with aerodynamic coupling; glottal flow exhibits quasi-periodicity around $f_0$ plus irregular, turbulent, and chaotic components; and perceptually relevant cues such as micro-jitter, harmonic fine structure, aperiodic bursts, and co-articulation transitions can be interpreted as manifestations of deterministic chaos and multi-scale recurrence (Tamiti et al., 1 Oct 2025).

Within this framing, MSRD is the discriminator specialized for recurrence geometry: it targets when and how the signal revisits similar states. That function differentiates it from the Multi-Resolution Lyapunov Discriminator (MRLD), which targets local divergence and sensitivity to initial conditions; from MSDFA, which targets long-range fractal scaling; and from MRAD/MRPD, which operate on magnitude and phase statistics. The intended role of MSRD is therefore neither generic periodicity detection nor generic spectro-temporal realism, but explicit supervision of multi-scale self-similar recurrence structure in generated speech (Tamiti et al., 21 Jul 2025).

The 2025 preprints present two distinct publication states. In "Nonlinear Framework for Speech Bandwidth Extension" (Tamiti et al., 21 Jul 2025), MS-RD is described as conceptually central to the nonlinear part of NDSI-BWE, but the final deployed system uses MRLD, MSDFA, MRAD, and MRPD rather than MS-RD. In "NLDSI-BWE: Non Linear Dynamical Systems-Inspired Multi Resolution Discriminators for Speech Bandwidth Extension" (Tamiti et al., 1 Oct 2025), MSRD is formalized as one of the two new nonlinear dynamical systems-inspired discriminators, used jointly with MRLD to replace AP-BWE’s MPD.

2. Recurrence representation and dynamical formulation

The dynamical basis of MSRD is the recurrence plot. In classical nonlinear dynamics, for a trajectory $\{\mathbf{x}_i\}_{i=1}^{N}$, the recurrence plot is

$$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$

where $\Theta(\cdot)$ is the Heaviside step function and $\varepsilon$ is a threshold. Structured textures in such plots encode periodicity, quasi-periodicity, laminar states, drifts, and chaotic transients (Tamiti et al., 1 Oct 2025).

MSRD deliberately adopts a cheaper, simplified recurrence construction. Rather than computing distances in a delay-embedded state space, it uses an amplitude-based recurrence plot on a downsampled waveform. For a scale $s$, the waveform is downsampled by selecting every $s$-th sample, with scale set

$\mathcal{S} = \{1,2,4,8,16\}.$

If the resulting sequence exceeds $L_{\max}=256$, it is uniformly subsampled to length 256. For the resulting $\tilde{x}^{(s)} \in \mathbb{R}^{L}$, MSRD computes

$\{\mathbf{x}_i\}_{i=1}^{N}$0

followed by a data-adaptive threshold

$\{\mathbf{x}_i\}_{i=1}^{N}$1

and then the binary recurrence plot

$\{\mathbf{x}_i\}_{i=1}^{N}$2

This construction is intended to capture when samples along the 1-D time series are similar in amplitude, which the papers associate with periodic structure, repeated motifs, and more irregular or chaotic behavior (Tamiti et al., 1 Oct 2025).

The “multi-scale” component is not incidental. The fine scales are described as being sensitive to pitch-period structure, micro-structure, and jitter, whereas coarser scales are meant to expose broader phonetic, syllabic, prosodic, and longer-range repetition patterns. In the NDSI-BWE description, this multi-scale downsampling is explicitly presented as the mechanism by which MS-RD accesses both fine-scale and increasingly coarse-scale temporal resolutions (Tamiti et al., 21 Jul 2025).

3. Architecture and computational organization

Two closely related architectural descriptions appear in the 2025 speech papers. In the NDSI-BWE manuscript, recurrence plots from the five scales are resized to a common $\{\mathbf{x}_i\}_{i=1}^{N}$3 resolution and stacked along the channel dimension, yielding

$\{\mathbf{x}_i\}_{i=1}^{N}$4

which is then passed to a 5-layer depthwise-separable 2-D CNN producing a scalar real/fake score $\{\mathbf{x}_i\}_{i=1}^{N}$5. The pseudocode explicitly states: downsample at scales $\{\mathbf{x}_i\}_{i=1}^{N}$6, compute pairwise distances, threshold at the mean distance, resize each recurrence plot to $\{\mathbf{x}_i\}_{i=1}^{N}$7, stack the plots, and apply CNN_MSRD(R) (Tamiti et al., 21 Jul 2025).

In the later NLDSI-BWE paper, MSRD is presented as a multi-branch discriminator over the same scale set. Each scale feeds a Single-Scale Recurrence Discriminator (SSRD), and each SSRD is a 2-D depthwise-separable CNN that outputs patch logits $\{\mathbf{x}_i\}_{i=1}^{N}$8 and intermediate feature maps $\{\mathbf{x}_i\}_{i=1}^{N}$9. The paper does not provide a full layer-by-layer SSRD table, but it explicitly states that MSRD overall has about 247.7k parameters and uses depthwise-separable convolution throughout for efficiency (Tamiti et al., 1 Oct 2025).

The architectural constant across these descriptions is the use of depthwise-separable 2-D convolutions rather than dense 2-D convolution. The stated rationale is parameter reduction: standard convolution with kernel size $$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$0, input channels $$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$1, and output channels $$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$2 uses $$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$3 parameters, whereas depthwise-separable convolution uses $$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$4. In the broader NDSI-BWE framework, the authors associate depthwise convolution at the core of the discriminator blocks with large parameter savings; in the later paper, MSRD and MRLD together are reported to replace MPD with approximately 45\times fewer parameters (Tamiti et al., 21 Jul 2025, Tamiti et al., 1 Oct 2025).

A technical consequence of this design is that MSRD operates on recurrence images, not directly on raw waveform slices or STFT tensors. This means that the learned filters are tuned to visual structures such as diagonals, blocks, and texture-like recurrence motifs, rather than to local waveform oscillations alone. This suggests a discriminator prior centered on pairwise temporal similarity geometry rather than on marginal waveform or spectrogram statistics.

4. Adversarial role within bandwidth-extension systems

In the speech BWE setting, MSRD is not a standalone model but one discriminator in a multi-discriminator GAN. In the NDSI-BWE description, MS-RD is presented as the missing recurrence-oriented component beside MRLD, MSDFA, MR-PPD, MRAD, MRPD, and MPD. The text explicitly states that MS-RD would be used in parallel with other discriminators, each providing its own adversarial score, with the generator receiving a sum of adversarial losses from all active critics (Tamiti et al., 21 Jul 2025).

The later NLDSI-BWE paper specifies the adversarial training more concretely. The generator is the AP-BWE generator: a ConvNeXt-based, dual amplitude/phase stream model. The discriminator set is MRAD + MRPD + MRLD + MSRD, and these jointly replace AP-BWE’s MPD. The discriminator objective uses hinge loss. For a discriminator branch $$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$5,

$$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$6

The overall discriminator loss is the sum of branch losses from MRLD, MSRD, MRAD, and MRPD, with MSRD contributing one hinge-loss term per recurrence scale (Tamiti et al., 1 Oct 2025).

The generator objective is reported as

$$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$7

Here, $$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$8 is a feature-matching loss over discriminator feature maps, including the MSRD feature maps $$R_{ij} = \Theta\big(\varepsilon - \|\mathbf{x}_i - \mathbf{x}_j\|\big),$$9, and $\Theta(\cdot)$0 includes adversarial terms from all active MSRD scales (Tamiti et al., 1 Oct 2025). In this formulation, MSRD influences training in two ways: directly through real/fake classification on recurrence structure, and indirectly through feature matching against the intermediate recurrence-image representations of real speech.

Conceptually, this positions MSRD as complementary to the other discriminators. MRLD focuses on the rate of divergence of nearby trajectories; MRAD and MRPD focus on magnitude and phase statistics; MSRD focuses on the pattern of returns. The intended composite effect is that generated wideband speech should exhibit realistic amplitude and phase behavior while also matching the recurrence geometry and chaotic dynamics of natural voiced sound production (Tamiti et al., 1 Oct 2025).

5. Empirical behavior, efficiency, and ablation evidence

The empirical record for MSRD differs sharply between the two 2025 speech papers. In the NDSI-BWE manuscript, the final ablation table does not include MS-RD in the deployed discriminator set, and the paper states that the final system uses MRLD, MSDFA, MRAD, and MRPD. Nevertheless, the manuscript reports that the authors considered MS-RD a lightweight alternative to MPD and explicitly state that “MSRD, which captures recurrence patterns similar to the periodic patterns captured by the MPD, can replace MPD with a fraction of parameters due to the presence of the depth-wise separable convolution as the core block” (Tamiti et al., 21 Jul 2025).

The later NLDSI-BWE paper provides the decisive quantitative evidence. It reports the following discriminator parameter counts: MPD $\Theta(\cdot)$1M, MRLD $\Theta(\cdot)$2k, and MSRD $\Theta(\cdot)$3k. Thus MRLD + MSRD together amount to approximately 483.2k parameters, corresponding to a reduction of roughly 45\times relative to MPD. For the full discriminator stack, MRAD + MRPD + MRLD + MSRD $\Theta(\cdot)$4M versus MPD + MRAD + MRPD $\Theta(\cdot)$5M, a reduction of approximately 13.7\times. At whole-system scale, AP-BWE is reported as 72M parameters and NLDSI-BWE as 51.75M, or about 28% fewer parameters overall (Tamiti et al., 1 Oct 2025).

Ablation on the 2→16 kHz task further clarifies MSRD’s functional role. The paper reports that MSRD-only yields LSD = 1.2618, STOI = 0.8832, SI-SNR = 9.21, and MOS drops to $\Theta(\cdot)$6, indicating that recurrence supervision alone is insufficient. With MRAD + MRPD, the reported values are LSD = 1.1261, STOI = 0.8663, SI-SNR = 7.68, and MOS = 4.07. Adding MSRD gives MRAD + MRPD + MSRD with LSD = 1.1221, SI-SNR = 8.24, and MOS = 4.19, which the paper interprets as an improvement in spectral fidelity and perceptual quality. The final quartet MRAD + MRPD + MRLD + MSRD yields LSD = 1.1112, STOI = 0.8669, SI-SNR = 7.63, and NISQA-MOS = 4.13, whereas the AP-BWE stack with MPD gives LSD = 1.1101, STOI = 0.8537, SI-SNR = 6.67, and MOS = 4.11 (Tamiti et al., 1 Oct 2025).

Across full-task comparisons, the same paper reports that replacing MPD with MRLD + MSRD maintains or improves quality. For NISQA-MOS, AP-BWE versus NLDSI-BWE is reported as 3.86 vs 4.11 for 4–16 kHz, 3.97 vs 4.29 for 8–16 kHz, and 4.49 vs 4.50 for 16–48 kHz. STOI is reported as equal at 0.94, 0.99, and 0.99 across the three ranges. PESQ is said to be essentially unchanged, and LSD roughly similar, with NLDSI-BWE slightly better or comparable depending on range. At 16–48 kHz, SI-SDR and SI-SNR are slightly lower for NLDSI-BWE (19.49 and 19.44) than for AP-BWE (20.86 and 20.74) (Tamiti et al., 1 Oct 2025).

These results support a narrow but important conclusion. MSRD is not sufficient as a sole discriminator, but in combination with amplitude, phase, and Lyapunov critics it is reported to improve or preserve perceptual quality and spectral fidelity while dramatically reducing discriminator size.

6. Interpretation, distinctions, and broader relevance

A common source of confusion is the word “recurrence.” In the speech-bandwidth-extension papers, “recurrence” refers to recurrence plots and the geometry of returns to similar states in a nonlinear dynamical system. It does not refer to recurrent neural networks, and it is distinct from recursive parameter sharing across scales. This distinction matters because an earlier image-super-resolution paper describes a “multi-scale and recursive discriminator” with shared modules across pyramid levels and a VN+SC front-end, and a secondary interpretation in the supplied material treats that architecture as something one “would mean by a Multi-Scale Recurrence Discriminator.” Architecturally, however, that image model is a different object: it is recursive over scales rather than recurrence-plot based (Michelini et al., 2018).

Within the speech literature itself, another point of potential misunderstanding is whether MSRD became part of the final deployed system immediately. The record is mixed by publication stage rather than by concept. In (Tamiti et al., 21 Jul 2025), MS-RD is described in detail but omitted from the final NDSI-BWE discriminator set; in (Tamiti et al., 1 Oct 2025), MSRD is formalized, optimized, and used in the final NLDSI-BWE stack. This suggests a progression from conceptual proposal to stabilized deployment.

The reported limitations are also specific. MSRD uses amplitude-only recurrence rather than multi-dimensional delay embeddings; it therefore avoids embedding dimension $\Theta(\cdot)$7 and delay $\Theta(\cdot)$8, but still depends on the choice of scale set $\Theta(\cdot)$9, length cap $\varepsilon$0, and global-mean thresholding. The evaluations are reported only on VCTK (v0.92) in noise-free conditions, and the introduction of recurrence plots and Lyapunov exponents is said to increase training time relative to AP-BWE, although inference cost is unaffected because discriminators are not used at inference (Tamiti et al., 1 Oct 2025).

The same paper identifies several broader directions. It states that recurrence-aware, multi-scale discriminators could extend to speech enhancement, dereverberation, neural vocoders, TTS, speaker verification, voice pathology detection, and music or general audio restoration. A plausible implication is that MSRD’s core contribution is not tied to bandwidth extension alone, but to a more general discriminator design principle: using compact 2-D CNNs over pairwise similarity structure across time as a third supervisory axis beside waveform-domain and time–frequency-domain critics (Tamiti et al., 1 Oct 2025).

In that sense, MSRD occupies a precise methodological niche. It is a discriminator for self-similarity dynamics, implemented through multi-scale recurrence plots, intended to supervise generated speech by the recurrence geometry of real acoustic trajectories. The 2025 results present it not as a universal replacement for other critics, but as an efficient component whose value is clearest in combination with Lyapunov, amplitude, and phase discriminators.