Bone-Conduction Diffusion (BCDM-DC-L)

• The paper introduces a multimodal conditional diffusion framework that leverages bone-conducted inputs with air-conducted signals for robust speech enhancement under extreme noise.
• It details a mathematically grounded approach using forward and reverse stochastic differential equations, with an Euler–Maruyama scheme for efficient inference.
• Empirical results demonstrate superior performance over baselines, showing consistent improvements in PESQ, POLQA, and ESTOI across varying noise levels.

The Bone-conduction Diffusion model, large variant (BCDM-DC-L), is a multimodal speech enhancement architecture that integrates air- and bone-conducted microphone inputs via a conditional diffusion generative framework. Designed to address severe performance degradation in single-channel speech enhancement systems under extreme noise, BCDM-DC-L explicitly leverages the complementary, noise-robust characteristics of bone-conducted speech to guide the denoising process in the spectral domain through a mathematically grounded stochastic differential equation (SDE) mechanism (Khanagha et al., 18 Jan 2026).

1. Mathematical Foundation

BCDM-DC-L models the clean speech short-time Fourier transform (STFT) slice $x_0 \in \mathbb R^D$, the noisy air-conducted STFT $y$, and the bone-conducted STFT $y_c$. The generative process is defined as a continuous-time SDE—specifically, a drift-diffusion process:

$$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$

where the drift term

$$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$

pulls the process toward the noisy air-conducted mixture, and the diffusion scheduling is

$$g(t) = \sigma_{\min} \left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)^{t} \sqrt{2\ln\left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)} \tag{3}$$

with $\sigma_{\min} = 0.05$, $\sigma_{\max} = 0.5$. $W_t$ is standard Wiener noise.

The corresponding reverse-time SDE is

$$dx_t = \big[ -f(x_t, y) + g(t)^2 \nabla_{x_t} \log p_t(x_t \mid y) \big] dt + g(t) d\overline{W}_t \tag{4}$$

where the "score" $y$0 is approximated by a neural network $y$1. Discretization over $y$2 steps yields the forward transition:

$y$3

and the reverse (denoising) process is modeled as

$y$4

where $y$5 is constructed via an Euler–Maruyama step utilizing the learned score $y$6.

Training minimizes the (conditional) Fisher divergence between true and model score:

$y$7

In practice, the equivalent "noise-prediction" parameterization is adopted:

$y$8

No additional variational bounds are used.

2. Model Architecture

The BCDM-DC-L architecture is based on a multi-resolution NCSN++ U-Net, comprising symmetric encoder-decoder paths. The "large" (L) variant adopts channel widths $y$9 and ResNet blocks of depth 2, with approximately 67.4 million parameters.

Bone-conduction conditioning (DC) is realized via a separate BigGAN-style ResNet downsampling encoder for $y_c$0 featuring depth 2 and matching channel sizes. Decoder-stage integration concatenates: (i) main decoder skip-connection features, (ii) the previous decoder output, and (iii) the bone-conduction encoder's features; a $y_c$1 convolution reduces channel dimensionality post-concatenation.

Data flow outline:

• $y_c$2 noisy mixture spectrogram
• $y_c$3 forward SDE state sampled from $y_c$4
• For $y_c$5:
  • $y_c$6
  • $y_c$7
  • $y_c$8
• Output $y_c$9 (complex STFT of enhanced speech)

3. Training Methodology

The model is trained with the loss

$$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$0

where $$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$1 with $$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$2 provided directly from SDE analysis.

Optimization employs Adam (learning rate $$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$3; $$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$4), batch size 8, with exponential moving average parameter tracking (decay 0.999) and $$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$5 diffusion steps per sample.

4. Dataset and Preprocessing

Data is sourced from ABCS: 42 hours, 100 speakers, with aligned air- and bone-conducted speech. Noise is synthetically added to air signals using CHiME3, training SNR sampled uniformly in $$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$6 dB and testing on SNRs $$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$7 dB. STFT parameters are: window size 510, hop 128, yielding 256 frequency bins and 256 time frames post-processing, resulting in $$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$8 complex-valued inputs. Each STFT channel is per-utterance zero-mean, unit-variance normalized.

5. Inference and Sampling Procedure

At inference, a Predictor–Corrector sampler is used, cycling through:

• Predictor: Euler–Maruyama step using $$dx_t = f(x_t, y)\,dt + g(t)\,dW_t, \quad t \in [0, T] \tag{1}$$9.
• Corrector: Langevin step; $$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$0.

Typically, $$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$1 steps are used, but step counts $$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$2–$$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$3 are supported. Step size $$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$4 is adapted to maintain sampler SNR. The model achieves real-time feasibility (e.g., 256×256 input, $$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$5 in 150 ms with a single GPU); real-time/causal deployment is projected possible with 8–16 steps ($$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$630 ms/frame). The small model variant achieves competitive results (PESQ) with as few as 20 steps, corresponding to roughly 40 score evaluations.

6. Empirical Performance and Model Analysis

BCDM-DC-L demonstrates consistent superiority over prior single- and multimodal baselines (BiNet, DCCRN) in multiple metrics:

SNR (dB)	POLQA	PESQ	ESTOI
–10	2.44	2.02	0.76
+15	3.70	3.25	0.94

Relative improvements over the strongest baseline are $$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$7 to $$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$8 in POLQA, $$f(x_t, y) = \gamma (y - x_t), \quad \gamma > 0 \tag{2}$$9 to $$g(t) = \sigma_{\min} \left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)^{t} \sqrt{2\ln\left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)} \tag{3}$$0 in PESQ, and $$g(t) = \sigma_{\min} \left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)^{t} \sqrt{2\ln\left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)} \tag{3}$$1 to $$g(t) = \sigma_{\min} \left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)^{t} \sqrt{2\ln\left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)} \tag{3}$$2 in ESTOI across all SNRs evaluated.

Ablation studies contrast "DC" (bone-conduction conditioning) versus "IC" (independent conditioning): DC (67.4 M parameters) outperforms IC (65.6 M) by $$g(t) = \sigma_{\min} \left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)^{t} \sqrt{2\ln\left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)} \tag{3}$$3 to $$g(t) = \sigma_{\min} \left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)^{t} \sqrt{2\ln\left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)} \tag{3}$$4 PESQ but exhibits slower performance scaling as the number of reverse steps increases. Smaller BCDM-DC-S variants surpass DCCRN at $$g(t) = \sigma_{\min} \left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)^{t} \sqrt{2\ln\left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)} \tag{3}$$5 steps and BiNet at $$g(t) = \sigma_{\min} \left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)^{t} \sqrt{2\ln\left(\frac{\sigma_{\max}}{\sigma_{\min}}\right)} \tag{3}$$6.

7. Significance and Context

BCDM-DC-L demonstrates the effectiveness of integrating bone-conducted signals through conditional diffusion modeling, leveraging the intrinsic noise-resilience of bone transmission and the generative flexibility of score-based SDEs. These results substantiate the potential of multimodal SDE-guided diffusion models to robustly enhance speech in adverse acoustic regimes, setting a new reference for quantitative performance in the multimodal speech enhancement domain (Khanagha et al., 18 Jan 2026). The explicit SDE-based fusion of modalities, and efficient inference via predictor–corrector samplers, positions this approach for further adoption both in batch and real-time applications where robust speech quality under noise is requisite.