Dual-Resolution Attentive Stats Pooling

• The paper introduces DRASP, a dual-resolution pooling strategy that fuses global statistics with segment-level attentive pooling to enhance MOS prediction.
• It partitions variable-length audio features into non-overlapping segments, applying learned attention to emphasize locally salient events while retaining overall structure.
• Experiments on MusicEval and AES-Natural demonstrate that DRASP outperforms traditional pooling methods, improving metrics such as MSE, LCC, and SRCC.

Searching arXiv for the specified paper and closely related work on attentive/statistics pooling for MOS prediction. Dual-Resolution Attentive Statistics Pooling (DRASP) is a pooling framework for automatic mean opinion score (MOS) prediction in generated audio. It addresses the transformation of variable-length frame-level audio features into a fixed-size utterance-level representation by combining two temporal views: a coarse global summary over the full utterance and a fine local summary over short segments with attention applied at the segment level rather than the frame level. In the formulation reported for automatic MOS prediction, DRASP is inserted as a replacement for the original pooling layer inside existing backbones, specifically a CLAP-based model for MusicEval and AudioBox-Aesthetics for AES-Natural, and is presented as a mechanism for capturing both overall structural context and perceptually salient local events concurrently (Yang et al., 29 Aug 2025).

1. Problem setting and perceptual motivation

The method is introduced in the setting of automatic MOS prediction for generated audio, where the objective is to predict human perceptual ratings directly from an audio clip without requiring listening tests. The reported application scope includes text-to-music evaluation on MusicEval and broader audio aesthetics or quality prediction on AES-Natural, spanning speech, music, and general audio (Yang et al., 29 Aug 2025).

A central technical difficulty is that audio encoders produce variable-length sequences of frame-level embeddings, whereas the MOS regressor requires a fixed-dimensional utterance-level representation. The input sequence is written as

$$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t\in\mathbb{R}^d,$$

with $T$ varying with audio duration. Pooling is therefore required to convert this sequence into a fixed-size vector encoding perceived quality (Yang et al., 29 Aug 2025).

The motivating critique of prior pooling approaches is that they usually operate at a single granularity. Average pooling assigns equal weight to all frames, which can obscure rare but perceptually important quality events. Statistics pooling improves on this by using global mean and standard deviation, but remains static and indiscriminate because all frames still contribute equally. Attentive pooling and attentive statistics pooling introduce learned weighting, yet the reported argument is that strictly frame-level attention can be unstable for MOS prediction: it may overreact to transient or noisy frames, may neglect broader acoustic context, and in the reported experiments did not consistently outperform plain global statistics pooling (Yang et al., 29 Aug 2025).

The perceptual premise underlying DRASP is that human audio quality judgment depends on both coarse global structure and fine local salient regions. In the paper’s framing, these correspond respectively to overall consistency, temporal dynamics, and broad production or musical characteristics on the one hand, and short segments containing artifacts, expressive events, alignment failures, distortions, or other quality cues on the other. This motivates a pooling design that combines a holistic summary of the full utterance with a selective local view over perceptually important short segments (Yang et al., 29 Aug 2025).

2. Architectural organization

DRASP is a dual-branch pooling framework. It receives frame-level features extracted by an upstream audio encoder and outputs a fixed-size utterance-level embedding $\mathbf{p}$. The framework is described as operating at two resolutions: a coarse or global resolution over the entire utterance and a fine or local resolution over short segments, with the two views fused into a single representation (Yang et al., 29 Aug 2025).

The first branch, termed the Global Overview Branch, operates on the full frame sequence and computes unweighted global mean and standard deviation. Its stated role is to capture the overall temporal structure and holistic characteristics of the utterance. The second branch, termed the Segmental Attention Branch, partitions the frame sequence into non-overlapping segments, computes one embedding per segment by averaging frames within that segment, applies attention over segments rather than frames, and then computes weighted mean and weighted standard deviation over the segment embeddings (Yang et al., 29 Aug 2025).

The conceptual distinction between the branches is explicit. The global branch is characterized as broad, stable, robust, context-rich, and less sensitive to local noise. The fine branch is characterized as selective, salience-aware, and sensitive to localized quality events, but stabilized by segmenting before attention. The framework’s novelty relative to standard attentive statistics pooling lies in this segment-level attention and in the explicit fusion of coarse global and fine local summaries rather than reliance on a single attentional view (Yang et al., 29 Aug 2025).

A plausible implication is that DRASP occupies an intermediate point between purely global pooling and purely frame-selective pooling. The paper’s own comparison language supports this reading by presenting the global and segmental branches as complementary rather than competing descriptions of the same temporal evidence.

3. Mathematical formulation

The input audio is represented as

$$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$

For the coarse branch, a standard statistics pooling layer is applied to the full sequence to obtain a global mean vector $\boldsymbol{\mu}$ and a global standard deviation vector $\boldsymbol{\sigma}$. The paper states these quantities but does not explicitly print their formulas in the DRASP section; any standard closed-form expression for them is therefore implied rather than explicitly stated (Yang et al., 29 Aug 2025).

For the fine branch, the sequence is partitioned into

$S = T / n$

non-overlapping segments, where $n$ is the segment size. Each segment embedding is formed by segment-wise average pooling:

$$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$

The segment embedding $\mathbf{a}_s$ is then scored by a learnable attention mechanism:

$T$0

where $T$1, $T$2, and $T$3 are trainable parameters. Softmax normalization yields the segment attention weight

$T$4

Using these weights, the fine branch computes a weighted mean and weighted standard deviation:

$T$5

$T$6

Here $T$7 denotes element-wise multiplication. The paper does not define variance separately; it gives only the weighted mean and weighted standard deviation (Yang et al., 29 Aug 2025).

The final utterance-level representation is obtained through trainable scalar fusion:

$T$8

where $T$9 denotes concatenation. The fusion weights $\mathbf{p}$0 and $\mathbf{p}$1 are trainable scalar parameters initialized as

$\mathbf{p}$2

This initialization biases optimization toward the stable global branch at the start of training and allows the model to learn how much segmental information to incorporate later (Yang et al., 29 Aug 2025).

The dimensionality of the fused representation is not stated explicitly in the paper text, but it follows from the formulation that if $\mathbf{p}$3, then each mean or standard-deviation vector is in $\mathbf{p}$4, each concatenated branch output is in $\mathbf{p}$5, and therefore $\mathbf{p}$6. This is an inference directly implied by the equations rather than a separately reported design statement.

4. Forward computation and downstream prediction

The reported forward pass begins with frame-level encoding by the upstream audio encoder, producing $\mathbf{p}$7. The coarse branch applies global statistics pooling over all $\mathbf{p}$8 frames to produce $\mathbf{p}$9 and $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$0, which are then concatenated. The fine branch partitions the $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$1 frames into $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$2 non-overlapping segments, each containing $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$3 frames, computes segment-level averages $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$4, scores them with the attention mechanism, normalizes the scores into attention weights $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$5, and computes the attentive segment-level mean and standard deviation before concatenation (Yang et al., 29 Aug 2025).

The paper explicitly notes that handling of remainder frames is not specified when $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$6 is not divisible by $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$7; it simply states $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$8. This omission is relevant to exact reproduction because no padding, truncation, or partial-segment rule is given (Yang et al., 29 Aug 2025).

After branch-specific pooling, the two resolutions are fused through the trainable scalar combination above to form $$\{\mathbf{a}_t\}_{t=1}^T,\qquad \mathbf{a}_t \in \mathbb{R}^d.$$9, which is then sent to task-specific MLP heads. In the CLAP-based MusicEval backbone, one MLP predicts musical impression, denoted $\boldsymbol{\mu}$0, from $\boldsymbol{\mu}$1, and another predicts textual alignment, denoted $\boldsymbol{\mu}$2, from the concatenation of $\boldsymbol{\mu}$3 and the text embedding. In AudioBox-Aesthetics augmented with DRASP, four parallel MLP heads predict

$\boldsymbol{\mu}$4

The exact regression equations, layer sizes, and head architectures are not provided in the paper text (Yang et al., 29 Aug 2025).

This division between a pooling module and downstream prediction heads is important for interpretation. DRASP is not introduced as a standalone end-to-end predictor; it is a replacement pooling layer designed to be inserted into existing MOS prediction backbones.

5. Relation to prior pooling methods

DRASP is positioned relative to several established pooling strategies. Average pooling computes only the temporal mean and therefore captures a single global centroid while ignoring variability. Statistics pooling extends this by computing global mean and standard deviation over all frames. Attentive pooling and self-attentive pooling learn weights over frames. Attentive statistics pooling computes weighted mean and weighted standard deviation over frames (Yang et al., 29 Aug 2025).

Against this background, DRASP retains the stability of global statistics pooling while adding a second branch for local salience. Its distinction from frame-level attentive statistics pooling is threefold: segment-level attention instead of frame-level attention in the fine branch, a dual-resolution architecture that combines non-attentive global statistics with attentive local statistics, and trainable fusion between the two views (Yang et al., 29 Aug 2025).

The comparison to multi-head attentive pooling and multi-resolution multi-head attentive pooling is also structurally specific. Those methods are described as computing attention multiple times over the full frame sequence. By contrast, DRASP defines resolution through whole-utterance versus segmental structure, not through repeated heads or alternative temperature settings, and applies attention to a shorter segment sequence. The paper argues that this makes the mechanism computationally cheaper, although no FLOP or runtime quantification is reported (Yang et al., 29 Aug 2025).

A common misconception would be to treat DRASP as ordinary attentive statistics pooling with a modified attention function. The reported formulation does not support that simplification. The method alters the unit of attention from frames to segments and combines the resulting local attentive statistics with a separate global statistical branch. In that sense, the architecture is dual-view rather than merely reweighted.

6. Experimental evidence, practical interpretation, and limitations

The reported evaluation covers MusicEval and AES-Natural. MusicEval contains 2,748 music clips generated by 31 text-to-music systems from 384 prompts and rated by expert musicians on two perceptual dimensions: musical impression and textual alignment. The AES-Natural subset used in the study contains 2,776 clips spanning speech, music, and general audio, rated on production quality (PQ), production complexity (PC), content enjoyment (CE), and content usefulness (CU). The backbones evaluated are a CLAP-based MOS prediction model and AudioBox-Aesthetics. System-level metrics are mean squared error (MSE), linear correlation coefficient (LCC), Spearman’s rank correlation coefficient (SRCC), and Kendall’s tau (KTAU), with system-level SRCC emphasized as the primary metric because the practical goal is ranking generative systems according to human judgments (Yang et al., 29 Aug 2025).

On MusicEval, averaged over musical impression and textual alignment, the reported results are as follows. Average Pooling yields MSE $\boldsymbol{\mu}$5, LCC $\boldsymbol{\mu}$6, SRCC $\boldsymbol{\mu}$7, and KTAU $\boldsymbol{\mu}$8. Statistics Pooling yields MSE $\boldsymbol{\mu}$9, LCC $\boldsymbol{\sigma}$0, SRCC $\boldsymbol{\sigma}$1, and KTAU $\boldsymbol{\sigma}$2. Attentive Statistics Pooling yields MSE $\boldsymbol{\sigma}$3, LCC $\boldsymbol{\sigma}$4, SRCC $\boldsymbol{\sigma}$5, and KTAU $\boldsymbol{\sigma}$6. Segmental Attentive Statistics Pooling yields MSE $\boldsymbol{\sigma}$7, LCC $\boldsymbol{\sigma}$8, SRCC $\boldsymbol{\sigma}$9, and KTAU $S = T / n$0. DRASP yields MSE $S = T / n$1, LCC $S = T / n$2, SRCC $S = T / n$3, and KTAU $S = T / n$4. The abstract reports a 10.39% relative improvement in system-level SRCC over average pooling, consistent with the comparison between $S = T / n$5 and $S = T / n$6 (Yang et al., 29 Aug 2025).

The comparison is informative even without a dedicated branch-removal ablation. Frame-level attentive statistics pooling underperforms plain global statistics pooling. Segmental attentive statistics pooling improves ranking metrics relative to frame-level attention. Statistics pooling is particularly strong in MSE and LCC, whereas segmental attentive statistics pooling is particularly strong in SRCC and KTAU. DRASP surpasses both, which the paper interprets as evidence that global and segmental views are complementary (Yang et al., 29 Aug 2025).

On AES-Natural, Table 2 reports system-level SRCC. The AudioBox-Aesthetics baseline achieves PQ $S = T / n$7, PC $S = T / n$8, CE $S = T / n$9, CU $n$0, and average $n$1. CLAP-based + DRASP achieves PQ $n$2, PC $n$3, CE $n$4, CU $n$5, and average $n$6. AudioBox-Aesthetics + DRASP achieves PQ $n$7, PC $n$8, CE $n$9, CU $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$0, and average $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$1. The paper uses these results to argue that DRASP generalizes across datasets, backbones, and content types including speech, music, and general audio (Yang et al., 29 Aug 2025).

A segment size study in Figure 1 shows that DRASP outperforms average pooling across all tested segment sizes, while also showing sensitivity to the segment size $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$2. The qualitative trade-off reported is that smaller $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$3 preserves more detail but increases noise sensitivity, whereas larger $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$4 smooths more aggressively and may blur perceptual events. The exact best numerical $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$5 values are not recoverable from the text alone (Yang et al., 29 Aug 2025).

The reported training configurations are backbone-specific. For the CLAP-based setup, the batch size is 64, the optimizer is SGD, the learning rate is 0.0005, and the loss is MAE. For AudioBox-Aesthetics + DRASP, the batch size is 32, the optimizer is AdamW, the learning rate is 0.0001, and the loss is an MAE + MSE composite. Both use early stopping on validation loss with patience of 20 epochs (Yang et al., 29 Aug 2025).

The main limitations and caveats are also explicit. Performance depends on the segment size $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$6, making it a meaningful hyperparameter. The segmental branch uses fixed non-overlapping segmentation. The conclusion suggests adaptive segmentation aligned with perceptual units such as phonemes, musical notes, or acoustic events, and also suggests extension to multi-resolution hierarchical pooling. Computational savings relative to multi-head attention methods are argued but not quantified. In addition, several implementation details are omitted, including the hidden dimension of the attention MLP, exact tensor sizes inside $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$7, $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$8, and $$\mathbf{a}_s = \frac{1}{n} \sum_{t=(s-1)n+1}^{sn} \mathbf{a}_t.$$9, exact MLP head architectures, normalization layers inside DRASP, the padding or truncation strategy when $\mathbf{a}_s$0 is not divisible by $\mathbf{a}_s$1, added computational cost in FLOPs or runtime, and the exact best segment size used in each experiment (Yang et al., 29 Aug 2025).

Taken together, the reported evidence situates DRASP as a pooling module for MOS prediction that replaces single-granularity temporal aggregation with a two-branch representation: global statistics over all frame embeddings and segmental attentive statistics over short non-overlapping segments, fused through learned scalar weights. The paper’s interpretation is that automatic MOS prediction benefits from jointly modeling holistic global structure and locally salient perceptual events (Yang et al., 29 Aug 2025).