Formant Modification in Speech Processing

• Formant modification (FM) is the deliberate alteration of vocal tract resonances by shifting or scaling formant frequencies to change speaker timbre while preserving linguistic content.
• Techniques range from discrete scaling of specific formants (e.g., F1–F3) to global LPC-based envelope warping, offering precise control in various speech applications.
• Applications include speaker anonymization, neural speech synthesis, and ASR data augmentation, balancing privacy, intelligibility, and naturalness trade-offs.

Searching arXiv for recent and foundational papers on formant modification and related methods. Formant modification (FM) denotes the deliberate alteration of vocal-tract resonance structure, typically by shifting or scaling formant frequencies, in order to change speaker-dependent timbral properties while preserving other dimensions of speech such as linguistic content, timing, or, in some implementations, the glottal excitation. In the source–filter model, speech is represented as $x(t) = s(t) * h(t)$, with the glottal source $s(t)$ exciting a vocal-tract filter $h(t)$ whose spectral envelope exhibits formant peaks at frequencies $F_k$. FM operates on these resonant frequencies, or on an equivalent all-pole spectral-envelope representation, and is used in speaker anonymization, de-identification, neural speech synthesis, and data augmentation for automatic speech recognition. Across contemporary formulations, FM ranges from training-free signal processing with constant scaling of $F_1$–$F_3$ to neural source–filter systems that treat formants as explicit, differentiable control variables (Yao et al., 2022, Tavi et al., 2022, Juvela et al., 2024, Zarazaga et al., 2023, Sapkota et al., 18 Jun 2026).

1. Theoretical basis in the source–filter model

FM is most naturally defined within the source–filter decomposition of speech production. In this model, the glottal source generates a quasi-periodic excitation characterized by its fundamental frequency $f_0$, while the vocal tract acts as a linear filter whose spectral envelope contains peaks at the formant frequencies $F_k$. In frequency domain form, $X(\omega) = S(\omega)\,H(\omega)$, and the formant peaks occur at $\omega_k = 2\pi F_k$ (Yao et al., 2022).

The central rationale for FM is that speaker identity is encoded strongly in the absolute positions of formants and in pitch range and dynamics, whereas linguistic information depends more heavily on time-varying articulatory patterns and relative formant trajectories. Consequently, modifying $s(t)$0, often together with $s(t)$1, can suppress identity-bearing cues while retaining intelligibility if the perturbation remains within plausible acoustic bounds (Yao et al., 2022).

A common abstraction is uniform formant scaling. If $s(t)$2, then a scaled representation is defined by

$s(t)$3

with an analogous scaling for pitch,

$s(t)$4

This corresponds conceptually to a frequency-axis dilation of the spectral envelope,

$s(t)$5

which preserves relative formant ratios while shifting their absolute positions (Yao et al., 2022).

An alternative, widely used in classical formant synthesis and in neural formant-control literature, expresses the vocal-tract transfer function as a cascade of resonators. With formant frequency $s(t)$6, bandwidth $s(t)$7, and sampling rate $s(t)$8,

$s(t)$9

and

$h(t)$0

This parameterization makes explicit that modifying formant frequencies changes the pole locations of the filter, hence the spectral envelope and perceived vowel quality or timbre (Zarazaga et al., 2023).

2. Classical signal-processing formulations

The most direct FM procedures are training-free and operate through signal processing. In the de-identification setting studied for Finnish speech, FM is implemented as a simple constant upward scaling of the first three formants: $h(t)$1 with $h(t)$2. The method uses Praat’s Vocal Toolkit “Change formant,” which relies on Burg LPC analysis to estimate formants and then shift them. Only $h(t)$3–$h(t)$4 are modified; higher formants remain unchanged (Tavi et al., 2022).

This formulation reflects a long-standing view of FM as a spectral-envelope perturbation that degrades automatic speaker verification by altering timbre-related cues while leaving the utterance otherwise intact. In that study, the method required no training data and no learned model, which made it practical for under-resourced settings where x-vector anonymization or neural resynthesis might be unavailable (Tavi et al., 2022).

A distinct but related approach appears in dysarthric ASR augmentation, where FM is described as LPC-based global pole warping. Rather than explicitly tracking $h(t)$5–$h(t)$6, LPC poles are warped using a scalar factor $h(t)$7, producing a global shift of the envelope followed by resynthesis from the modified all-pole filter. The paper reports a sweep over $h(t)$8 in steps of 0.05 and finds that small negative values near $h(t)$9 or $F_k$0 are most effective (Sapkota et al., 18 Jun 2026).

The literature synthesis accompanying that work notes that a typical all-pass formulation uses the substitution

$F_k$1

in the LPC filter, inducing a frequency mapping $F_k$2. The paper does not present this formula as part of its own explicit implementation, but it is consistent with the LPC-warping paradigm cited there (Sapkota et al., 18 Jun 2026).

A key distinction therefore separates two classical FM families. One family explicitly edits tracked formants, usually $F_k$3–$F_k$4, via multiplicative scaling. The other warps the entire all-pole envelope through LPC-domain transformations. This suggests a useful taxonomy: “discrete-formant FM” for direct $F_k$5 manipulation and “envelope-warp FM” for LPC-pole warping. The latter phrase is an Editor’s term.

3. FM in speaker anonymization and de-identification

The most detailed recent formulation of FM as an anonymization mechanism is the framework that directly represents speaker identity with formant and $F_k$6 trajectories extracted from source speech and then uniformly scales them frame-wise (Yao et al., 2022). The framework contrasts with x-vector-based anonymization by avoiding external speaker embeddings entirely. This prevents the introduction of other speakers’ characteristics, which had previously degraded speaker distinguishability in anonymized speech (Yao et al., 2022).

Two scaling strategies are defined. In gender-independent uniform scaling, the same factor $F_k$7 is applied to both formants and $F_k$8. In gender-dependent scaling, opposite-direction scaling is used to reflect typical male–female differences: $F_k$9 With $F_1$0, this enlarges male formants and $F_1$1 and reduces female formants and $F_1$2, keeping both within plausible ranges (Yao et al., 2022).

Although the equations describe frequency-domain scaling, the implemented system does not explicitly warp spectra. Instead, scaled trajectories condition a neural acoustic model that predicts anonymized Mel-spectrograms consistent with the scaled attributes, after which a multi-band HiFi-GAN vocoder reconstructs the waveform (Yao et al., 2022). The practical consequence is that FM becomes a parametric control signal for neural resynthesis rather than a direct waveform transform.

The reported privacy–utility trade-off is central. For gender-independent scaling with $F_1$3, privacy as measured by EER increases as $F_1$4 moves farther from 1.0, but intelligibility and distinctiveness degrade beyond a moderate range. A workable range is reported as $F_1$5. For gender-dependent scaling, the best trade-off is reported at $F_1$6 (Yao et al., 2022).

Under that final gender-dependent configuration, the system reports EER 34.91% on Libri-test and 35.82% on VCTK-test, with WER 4.34% and 9.29%, pitch correlation $F_1$7 0.79 and 0.84, and voice-distinctiveness metric $F_1$8 and $F_1$9, respectively. The prior NWPU-ASLP anonymization framework had $F_3$0, indicating substantially worse distinctiveness (Yao et al., 2022). The significance of this result is not merely stronger privacy; it is the claim that FM can suppress identity without collapsing different anonymized speakers toward the same timbral region.

A related but simpler line of work augments FM with functional data analysis of $F_3$1 trajectories. There, FM alone uses constant upward scaling of $F_3$2–$F_3$3, while pitch is manipulated separately by reconstructing continuous $F_3$4 curves using functional principal components: $F_3$5 Replacing the dominant score $F_3$6 with values derived from intended child or cross-sex speech improves de-identification relative to FM-only baselines (Tavi et al., 2022). In that study, the best result combines cross-sex FDA $F_3$7 manipulation with FM +20%, reaching EER 16.94% for female speech and 17.24% for male speech. The paper characterizes the female result as an approximately 25% relative improvement over FM-only, from 13.55% to 16.94% (Tavi et al., 2022).

4. Neural and differentiable realizations

Recent work repositions FM from a post hoc perturbation technique to a primary control interface for neural synthesis. In speaker-independent neural formant synthesis, a compact parameter vector per frame includes VUV, log $F_3$8, $F_3$9–$f_0$0, spectral tilt, spectral centroid, and energy. A non-causal WaveNet-style gated dilated CNN maps these parameters to Mel-spectrograms, and a pretrained universal vocoder, especially HiFi-GAN V1, renders waveforms (Zarazaga et al., 2023).

Within this framework, FM is realized by editing the formant trajectories prior to synthesis. The paper explicitly defines FM as frame-synchronous manipulation of vocal-tract resonances, primarily $f_0$1–$f_0$2, while preserving $f_0$3, voicing, timing, and broader prosody (Zarazaga et al., 2023). The intended use cases are experimental: stimulus creation for cue-weighting studies, categorical perception, coarticulation, and parametric prosody experiments.

The architecture does not impose an explicit differentiable DSP layer; rather, the CNN learns a mapping from phonetically meaningful features to Mel representations. Control fidelity is then verified by re-extracting parameters from the synthesized output. In copy synthesis on held-out speakers from VCTK, median MSE per z-scored feature typically falls between $f_0$4 and $f_0$5, with lowest errors for log $f_0$6, $f_0$7, and spectral centroid, and larger deviations for higher formants such as $f_0$8 (Zarazaga et al., 2023).

A more explicitly source–filter-based realization appears in HiFi-Glot, which couples a neural glottal source with a differentiable all-pole filter (Juvela et al., 2024). Here the feature-mapping network outputs “LARs + gain,” transformed into reflection coefficients by $f_0$9, ensuring $F_k$0, and gain by $F_k$1. A direct-form all-pole polynomial $F_k$2 is then computed via forward Levinson recursion, yielding

$F_k$3

With excitation STFT $F_k$4, output synthesis is

$F_k$5

This provides direct, differentiable control over formant resonances at synthesis time (Juvela et al., 2024).

Unlike explicit per-formant biquad cascades, HiFi-Glot does not instantiate each formant as a separate resonator. Instead, $F_k$6–$F_k$7 are inputs to a learned mapping that predicts a stable all-pole envelope. The system is trained end-to-end with HiFi-GAN losses plus an envelope log-spectral-distance objective,

$F_k$8

anchoring the predicted envelope to LPC-derived ground truth (Juvela et al., 2024).

The FM evaluation in HiFi-Glot applies formant scaling in the range $F_k$9 to $X(\omega) = S(\omega)\,H(\omega)$0–$X(\omega) = S(\omega)\,H(\omega)$1. For $X(\omega) = S(\omega)\,H(\omega)$2 and $X(\omega) = S(\omega)\,H(\omega)$3, the model achieves median absolute errors under 50 Hz and under 150 Hz, respectively, and all neural systems in the study outperform Praat in perceptual quality in a MUSHRA-like listening test, although no statistical difference among the neural models is found (Juvela et al., 2024). The broader significance is methodological: FM becomes an input-conditioned control mechanism within a fully differentiable synthesis stack.

5. Estimation, conditioning, and implementation practice

FM depends critically on accurate estimation of formants and, where relevant, $X(\omega) = S(\omega)\,H(\omega)$4. The anonymization framework based on uniform formant and $X(\omega) = S(\omega)\,H(\omega)$5 scaling uses PRAAT to extract per-frame $X(\omega) = S(\omega)\,H(\omega)$6–$X(\omega) = S(\omega)\,H(\omega)$7 and $X(\omega) = S(\omega)\,H(\omega)$8, with PRAAT’s standard LPC-based formant tracking and pitch estimation pipeline. Practical settings listed for implementation guidance include frame length 25–40 ms, hop 10 ms, Hann window, LPC order 10–14 for 16 kHz speech, formant range constraints such as 200–5500 Hz, and $X(\omega) = S(\omega)\,H(\omega)$9 in unvoiced frames (Yao et al., 2022).

That same framework also recommends optional temporal smoothing, including median filters and Kalman smoothing, to reduce tracking noise and spurious formant jumps or crossings, and explicit monotonicity constraints such as $\omega_k = 2\pi F_k$0 enforced by small local adjustments (Yao et al., 2022). These details matter because FM is sensitive to tracker failures: incorrect trajectories can generate implausible resynthesis or negate the intended privacy effect.

In the FDA-based de-identification study, $\omega_k = 2\pi F_k$1 extraction uses Praat’s autocorrelation method, with female floor/ceiling 140 Hz / 520 Hz and male 65 Hz / 380 Hz. Unvoiced segments are interpolated, and $\omega_k = 2\pi F_k$2 is converted to semitones before fitting continuous curves with a B-spline basis of order 4 and 202 basis functions, with smoothing parameter $\omega_k = 2\pi F_k$3. No landmark registration or DTW is used (Tavi et al., 2022). The paper attributes major variation in $\omega_k = 2\pi F_k$4 trajectories to the first principal component, which primarily shifts vertical position, and the second, which alters tilt or slope (Tavi et al., 2022).

Neural formant-control systems require somewhat different preprocessing. Speaker-independent neural formant synthesis uses Praat Burg formants with a 25 ms window, pre-emphasis over 50 Hz, up to 5 formants, ceilings 5.0 kHz for male and 5.5 kHz for female, and an 11.6 ms hop derived from 1024-sample windows and 256-sample hops at 22,050 Hz. Missing formant values are linearly interpolated, and all continuous parameters are mean-variance normalized (Zarazaga et al., 2023). HiFi-Glot instead extracts $\omega_k = 2\pi F_k$5–$\omega_k = 2\pi F_k$6 via differentiable root-finding from a 10th-order LPC polynomial, uses WORLD for log $\omega_k = 2\pi F_k$7, and median-filters $\omega_k = 2\pi F_k$8 trajectories with kernel size 3 (Juvela et al., 2024).

Across these systems, a recurring practical pattern is the separation of linguistic and identity-bearing conditioning. In the anonymization framework, speaker-invariant bottleneck features are extracted by WeNet ASR and combined with scaled formant and pitch trajectories in a CBHG-based acoustic model, whose formant and $\omega_k = 2\pi F_k$9 encoders each contain two CNN layers with kernel size 3 and dropout, trained using an L1 loss on Mel-spectrograms (Yao et al., 2022). This suggests that FM is most robust when embedded within an overview architecture that can absorb imperfect analysis through learned priors.

6. Applications, trade-offs, and limitations

FM serves at least three technically distinct application domains: privacy protection, controllable speech synthesis, and ASR data augmentation.

In privacy protection, the central trade-off is between speaker concealment and preservation of intelligibility and distinctiveness. The uniform-scaling anonymization results show that extreme scaling such as $s(t)$00 or $s(t)$01 increases EER but degrades WER and prosodic naturalness, while moderate ranges such as $s(t)$02 or gender-dependent $s(t)$03 provide a more balanced outcome (Yao et al., 2022). In training-free de-identification, stronger FM magnitudes improve EER but reduce STOI: FM-only at +10% yields STOI 0.82 for female and 0.80 for male speech, while +20% lowers these to 0.76 and 0.69 (Tavi et al., 2022). Combining FM with FDA-based $s(t)$04 manipulation improves EER further but lowers STOI again, with $s(t)$05 yielding 0.71 for female and 0.64 for male speech (Tavi et al., 2022).

In neural synthesis, the trade-off is between precision of acoustic control and naturalness. Speaker-independent neural formant synthesis reports that manipulation accuracy degrades as scaling approaches the extremes of $s(t)$06, and that cross-feature coupling can cause deviations in unmanipulated features, although the method often achieves lower formant error than Praat and better preservation of spectral centroid (Zarazaga et al., 2023). HiFi-Glot similarly notes that errors grow for larger manipulations and higher formants, particularly $s(t)$07–$s(t)$08, and that bandwidths and anti-formants are not explicitly modeled in the all-pole input set (Juvela et al., 2024).

In dysarthric ASR augmentation, FM is not the best augmentation overall, but it provides consistent gains. On the TORGO corpus, baseline best-case WERs without augmentation are 12.89% for low severity, 45.72% for medium, and 65.24% for high. FM achieves 9.65% on low severity and 40.34% on medium severity when training on high-severity speech with $s(t)$09, and 60.05% on high severity when training on medium-severity speech with $s(t)$10 (Sapkota et al., 18 Jun 2026). SRM is أفضل for low and medium severity, and PM is best for high severity, which indicates that spectral-envelope variability alone does not capture the full range of dysarthric distortions, especially where dysprosody and timing disruptions dominate (Sapkota et al., 18 Jun 2026).

Several limitations recur across the literature. Formant tracking errors in rapid transitions, high-$s(t)$11 speech, or consonantal regions can produce spurious jumps or formant crossing (Yao et al., 2022). Excessive shifts can degrade naturalness, intelligibility, or filter stability (Yao et al., 2022, Sapkota et al., 18 Jun 2026). All-pole methods do not model anti-formants, limiting their adequacy for nasal or heavily antiresonant speech (Juvela et al., 2024, Sapkota et al., 18 Jun 2026). Neural methods improve perceptual quality but may require substantial training data and computational infrastructure, whereas classical FM remains lightweight and transparent (Tavi et al., 2022).

A common misconception is that FM is equivalent to pitch modification or to vocal-tract length perturbation. The cited work treats these as distinct operations. FM targets the spectral envelope or formant poles; PM targets $s(t)$12 and prosody; SRM changes timing; VTLP applies a global spectral warp associated with vocal-tract length (Sapkota et al., 18 Jun 2026). Another misconception is that FM necessarily destroys speaker distinctiveness. The anonymization results based on source-derived formant and $s(t)$13 scaling argue the opposite: when external speaker characteristics are not introduced, FM can preserve inter-speaker distinctiveness more effectively than embedding-based anonymization (Yao et al., 2022).

Taken together, the current literature presents FM as a spectrum of methods unified by control over resonant structure. At one end, it is a simple multiplicative perturbation of $s(t)$14–$s(t)$15 implemented with Burg LPC in Praat; at the other, it is an explicit input modality to differentiable source–filter neural generators. The technical continuity across these regimes lies in the same core premise: modifying formant structure changes identity, timbre, and vowel quality in systematic ways that can be exploited for anonymization, experimental control, and robustness enhancement (Yao et al., 2022, Tavi et al., 2022, Juvela et al., 2024, Zarazaga et al., 2023, Sapkota et al., 18 Jun 2026).