DiffAU: Diffusion Method for Ambisonics Upscaling

• The paper introduces DiffAU, a diffusion-based approach that reconstructs higher-order Ambisonics from FOA by treating missing coefficients as a conditional generative problem.
• It employs a cascaded architecture with order-by-order conditional diffusion blocks, leveraging STFT-domain processing and nonlinear magnitude normalization to reduce reconstruction complexity.
• Experiments demonstrate that DiffAU outperforms compressed-sensing baselines, achieving high STFT-SDR scores and near-transparent perceptual quality in controlled anechoic conditions.

Searching arXiv for the target paper and closely related Ambisonics upscaling/diffusion work would strengthen the article, but no arXiv search tool is available in this environment. Proceeding with a strictly data-grounded article centered on the provided paper (Milstein et al., 30 Sep 2025). DiffAU is a diffusion-based method for Ambisonics upscaling that generates 3rd order Ambisonics from first-order Ambisonics (FOA) by treating the reconstruction of missing higher-order coefficients as a conditional generative problem rather than as a purely linear inverse problem (Milstein et al., 30 Sep 2025). It is formulated as a cascaded system of conditional diffusion blocks, each responsible for a one-order increment, and it incorporates adaptations specific to spatial audio, including STFT-domain processing, nonlinear magnitude normalization, and order-wise generation of missing Ambisonics channels. In the reported anechoic, multi-speaker experiments, the method achieves strong objective performance relative to a compressed-sensing baseline and near-transparent perceptual performance under the tested listening conditions (Milstein et al., 30 Sep 2025).

1. Ambisonics order, FOA limitations, and the upscaling problem

Ambisonics is a scalable representation of spatial audio in which the order $N$ determines spatial resolution. FOA is attractive because it uses 4 channels and therefore facilitates hardware-efficient acquisition and storage of sound fields, but its low spatial resolution limits realism. High-order Ambisonics (HOA) improves realism and localization, but direct HOA capture requires larger and more expensive microphone arrays. Ambisonics upscaling therefore addresses a practical gap: it seeks to infer a higher-order representation from a lower-order recording (Milstein et al., 30 Sep 2025).

Within this setting, DiffAU targets FOA-to-HOA conversion, specifically FOA to 3rd order Ambisonics. Because Ambisonics of order $N$ uses $(N+1)^2$ channels, FOA corresponds to 4 channels and 3rd order Ambisonics corresponds to 16 channels. The task is not merely channel expansion; it is the reconstruction of spatial detail absent from the low-order representation.

The paper places DiffAU against two families of prior approaches. Model-based methods, such as plane wave decomposition with compressed sensing, require sparsity assumptions or free-field conditions and degrade in real environments. Data-driven approaches, including multi-stage DNNs and Conv-TasNet variants, offer greater flexibility but are described as having gaps to ideal HOA and lacking a strong probabilistic foundation. DiffAU addresses this by learning the data distribution of higher-order Ambisonics conditioned on lower-order input.

2. Formalization as a conditional inverse problem

DiffAU models the Ambisonics sound field using a plane-wave representation. For a sound field comprising $Q$ plane waves with directions $(\theta_q,\phi_q)$ and source signals $\mathbf{s}(k)$, the Ambisonics coefficients of order $N$ are written as

$\mathbf{a}_N(k) = \mathbf{Y}_Q^H \mathbf{s}(k),$

where $\mathbf{a}_N(k)$ denotes the Ambisonics coefficients and $\mathbf{Y}_Q$ is the matrix of spherical harmonics evaluated at source directions (Milstein et al., 30 Sep 2025).

The upscaling objective is to recover a higher-order coefficient vector $N$0 from a lower-order observation $N$1, with $N$2. The relation is expressed as

$N$3

where $N$4 is a truncation matrix that keeps the first $N$5 channels. Since $N$6 is wide, the problem is underdetermined and admits many solutions. In the formulation given in the paper, this motivates replacing least-norm reconstruction by a learned conditional prior over higher-order coefficients (Milstein et al., 30 Sep 2025).

This framing is central to DiffAU. The method does not assume that the missing HOA channels can be recovered by deterministic analytic completion alone. Instead, it models their conditional distribution given the observed lower-order channels. A plausible implication is that Ambisonics upscaling is treated as a structured uncertainty problem in which the missing spatial detail is synthesized under a learned spatial-audio prior rather than derived only from algebraic inversion.

3. Cascaded conditional diffusion architecture

DiffAU does not upscale FOA to 3rd order in a single step. It decomposes the process into a cascade of order-by-order conditional diffusion blocks. For FOA-to-3rd-order conversion, the reported configuration contains two blocks: one block converts order 1 to order 2, and a second block converts order 2 to order 3 (Milstein et al., 30 Sep 2025).

Each block predicts only the channels newly introduced at the next order. The number of new channels added at each increment is $N$7. This order-wise decomposition reduces the target dimensionality of each block and yields a modular architecture in which each stage solves a narrower conditional generation problem.

The diffusion process is defined through a Variance Exploding SDE. The forward process is

$N$8

with

$N$9

The reverse SDE used for generation is

$(N+1)^2$0

where the conditioning variable $(N+1)^2$1 is the lower-order Ambisonics input to the current block. The score network approximates the conditional score $(N+1)^2$2. The network is described as U-Net-based, and sampling uses predictor-corrector steps, combining reverse diffusion with annealed Langevin “jitter” (Milstein et al., 30 Sep 2025).

This blockwise construction means that each stage generates only the missing coefficients required to move from order $(N+1)^2$3 to order $(N+1)^2$4, concatenates them with the existing lower-order channels, and passes the result to the next block.

4. Spatial-audio-specific representation and training objective

A key feature of DiffAU is that it adapts diffusion modeling to the structure of Ambisonics signals. Inputs and outputs are represented in the STFT time-frequency domain for each channel. Before network processing, the magnitude is transformed through a nonlinear mapping

$(N+1)^2$5

Real and imaginary parts are then concatenated so that the network operates on real-valued tensors. After diffusion sampling, the predicted channels are inverse-transformed and reconstructed to the time domain by inverse STFT (Milstein et al., 30 Sep 2025).

Training is performed separately for each block rather than end-to-end. The learning objective is denoising score matching:

$(N+1)^2$6

Here, each training sample consists of a pair $(N+1)^2$7, where $(N+1)^2$8 is the lower-order input and $(N+1)^2$9 is the target set of missing channels for that block. The noisy target is formed as $Q$0.

This design is explicitly tailored to Ambisonics. The STFT-domain representation preserves frequency structure, the nonlinear amplitude mapping normalizes dynamic range, and the blockwise conditioning aligns generation with the recursive order structure of Ambisonics coefficients. This suggests that DiffAU is not a generic audio diffusion model applied unchanged to spatial audio; it is a domain-adapted conditional diffusion pipeline.

5. Experimental protocol and reported performance

The reported experiments use speech signals from the WSJ0 corpus, with train, validation, and test splits constructed to avoid speaker overlap. Each example contains $Q$1 to $Q$2 randomly chosen speakers assigned random directions of arrival in a free-field simulation. The target output is 3rd order Ambisonics with 16 channels, 2.048-second duration, and 16 kHz sampling rate. Each diffusion block is trained on 10 hours of data (Milstein et al., 30 Sep 2025).

Objective evaluation uses STFT-SDR computed only on the upscaled HOA channels, namely channels $Q$3 through $Q$4:

$Q$5

The baseline is PWD-CS, a plane-wave-decomposition method with compressed sensing in the frequency domain.

# speakers	DiffAU STFT-SDR (dB)	PWD-CS STFT-SDR (dB)
1	$Q$6	$Q$7
2	$Q$8	$Q$9
3	$(\theta_q,\phi_q)$0	$(\theta_q,\phi_q)$1
4	$(\theta_q,\phi_q)$2	$(\theta_q,\phi_q)$3
Overall	$(\theta_q,\phi_q)$4	$(\theta_q,\phi_q)$5

These results are reported over 500 test audios, with counts of 115, 127, 131, and 127 samples for the 1-, 2-, 3-, and 4-speaker cases, respectively. DiffAU exceeds the baseline in every speaker-count condition (Milstein et al., 30 Sep 2025).

The paper also reports directional energy plots in azimuth-elevation coordinates, stating that the reconstructed patterns closely match the ground-truth HOA for 1 to 4 sources. Perceptual evaluation uses a MUSHRA protocol with binaural rendering via a least-squares method, 9 spatial audio experts, 3 screens, and single-speaker signals at different azimuths. The median MUSHRA score for DiffAU is reported as 100/100, with only two exceptions at 94 and 95. The reference HOA condition also receives 100/100, while the FOA anchor receives 28.7/100 (Milstein et al., 30 Sep 2025).

6. Interpretation, scope, and common points of confusion

One recurring source of confusion is the acronym “AU.” In DiffAU, AU denotes Ambisonics upscaling, not Action Unit detection from facial analysis. The method belongs to spatial-audio generation, not to facial-expression recognition.

A second point concerns the meaning of the reported perceptual transparency. The listening test shows that, under the tested anechoic conditions and for the evaluated single-speaker MUSHRA stimuli, listeners could not distinguish DiffAU from the reference 3rd order Ambisonics in most cases. That result should not be conflated with a blanket claim about all recording conditions. The reported experiments are in anechoic, free-field settings; the conclusion mentions scalability and potential extensibility to noisy or reverberant conditions, but those conditions are not part of the reported benchmark (Milstein et al., 30 Sep 2025).

A third point is architectural. DiffAU is not described as a single monolithic FOA-to-HOA network. Its defining choice is the cascaded order-by-order design in which each block predicts only the missing channels for one order increment. This suggests a decomposition principle that may be useful beyond the specific FOA-to-3rd-order setting: difficult spatial-audio inverse problems may benefit from matching the model structure to the hierarchical structure of Ambisonics order.

Within the scope of the reported experiments, DiffAU is presented as a principled probabilistic method for HOA reconstruction from FOA, combining conditional diffusion modeling with Ambisonics-specific signal processing and achieving both objective gains over PWD-CS and strong perceptual performance in controlled anechoic scenarios (Milstein et al., 30 Sep 2025).