Generative Machine Listener (GML)

• Generative Machine Listener (GML) is a deep-learning approach that models conditional distributions of listener scores using paired reference and degraded signals.
• It employs auditory front ends like Gammatone spectrograms and CNN architectures with Inception and Squeeze-and-Excitation blocks to capture spectral and spatial deviations.
• Extended variants such as RF-GML and GMLv2 enhance codec coverage and performance metrics, offering improved confidence interval predictions and multimodal integration.

Generative Machine Listener (GML), in audio quality assessment, denotes a family of deep-learning, full-reference quality predictors trained on individual intrusive listening-test scores to predict a distribution of listener ratings for each pair of reference and coded signals. Rather than producing only a single MOS-like estimate, GML parameterizes a score distribution from which one can derive mean quality, confidence intervals, and an arbitrary number of simulated listener scores. Subsequent variants extend the framework to reference-free assessment, Beta-distribution modeling, broader codec coverage including neural audio codecs, and audio-visual quality prediction pipelines that reuse GML as an audio backbone (Jiang et al., 2023, Raj et al., 25 Sep 2025).

1. Definition and scope

In the audio-quality literature, GML is defined around intrusive evaluation: a reference signal $x$ and a degraded or coded signal $y$ are jointly processed, and the model predicts how human listeners would rate $y$ relative to $x$ in a listening test. The central departure from conventional objective metrics is that GML is trained on individual listener scores, not only on per-item MOS, and therefore models a conditional distribution of ratings rather than a deterministic point estimate. The formulation is explicitly motivated by listening-test regimes such as MUSHRA, where both mean scores and confidence intervals matter operationally for codec comparison and tuning (Jiang et al., 2023).

A recurrent misconception concerns the word “generative.” In this lineage, “generative” does not primarily mean waveform synthesis or text-to-audio generation. It refers to the model’s ability to represent and sample from a distribution of possible listener scores for a given input condition. In the reference-free extension RF-GML, this is stated directly: the term “generative” refers to the model’s ability to generate an arbitrary number of simulated listening scores (Biswas et al., 2024).

The scope of the term has broadened in later work. Within audio-visual quality prediction, GML is treated as a state-of-the-art deep audio quality predictor whose internal representation can be reused as an audio branch in a multimodal fusion network. In that setting, GML is both a scalar quality metric $Q_a$ and a feature extractor exposing a 512-dimensional hidden representation $X_a$ (Salaj et al., 21 Sep 2025).

2. Probabilistic formulation of listener ratings

The canonical GML formulation models a conditional score distribution

$p_\theta(s \mid x,y),$

where $s$ is an individual listener score and $\theta$ are network parameters. Training proceeds by maximum likelihood over individual ratings: $$\mathcal{L}(\theta) = - \sum_i \log p_\theta(s_i \mid x_i,y_i).$$ This choice preserves information about inter-listener variability that is lost when training only on MOS (Biswas et al., 2024).

The original GML family considered Gaussian and Logistic output distributions, with the Logistic model reported as superior in the cited work. For the Logistic parameterization, the network outputs a mean $y$0 and a scale parameter $y$1, and the density is

$y$2

The corresponding per-example negative log-likelihood is

$y$3

and the standard deviation is

$y$4

From these parameters, GML directly provides a predicted mean score and, for a listening panel of size $y$5, a $y$6 CI via a standard $y$7-distribution construction (Jiang et al., 2023).

GMLv2 replaces the Gaussian/Logistic formulation with a Beta-distribution model over normalized MUSHRA scores $y$8. Its density is

$y$9

with

$y$0

so that $y$1 and the predicted distribution is unimodal. The mean and variance are

$y$2

The predicted MUSHRA score is

$y$3

Because the Beta support matches normalized MUSHRA exactly, GMLv2 avoids the boundary mismatch of unbounded distributions and treats uncertainty as a first-class output rather than a post hoc statistic (Raj et al., 25 Sep 2025).

3. Architecture and signal representation

The dominant architectural instantiation of GML uses an auditory front end based on Gammatone spectrograms. For stereo or binaural content, the representation includes left and right channels together with derived mid/side channels,

$y$4

In the full-reference setting, the network sees corresponding reference and degraded channels, so spatial and spectral deviations are jointly encoded (Biswas et al., 2024).

The backbone described for GML and RF-GML is Stereo InSE-NET: four modified Inception blocks arranged as In-A, In-A, In-B, In-C, with three Squeeze-and-Excitation blocks interspersed between them, followed by three fully connected layers. Inception branches capture multi-scale temporal and spectral patterns, while SE blocks implement channel-wise attention. The design targets 48 kHz mono, stereo, and binaural material (Biswas et al., 2024).

GMLv2 preserves the same general auditory-CNN paradigm but specifies front-end hyperparameters explicitly: 32 Gammatone filterbank channels, an 80 ms window, a 20 ms hop, and a lowest frequency of 50 Hz. The network ingests concatenated Gammatone power spectrograms of reference and degraded signals and produces two unconstrained outputs $y$5, which are transformed into Beta parameters (Raj et al., 25 Sep 2025).

Reference-free RF-GML is derived from the full-reference model by removing reference channels from the input layer and transferring weights from a pre-trained FR-GML with minimal architectural changes. Four variants are described: RF-GML (def), RF-GML (deg), RF-GML (degF), and RF-GML (all). RF-GML (deg), which initializes the degraded input channels in the first In-A block from FR-GML and then fine-tunes them, is reported as the best-performing configuration (Biswas et al., 2024).

4. Variants and system integration

The GML family now includes three distinct but closely related operating points: the original full-reference distributional predictor, a reference-free transfer-learned variant, and a Beta-based enhanced reference model with broader codec coverage.

Variant	Regime	Distinguishing property
GML	Full-reference	Predicts a listener-score distribution from $y$6
RF-GML	Reference-free	Uses only $y$7; transfer-learned from FR-GML
GMLv2	Reference-based	Uses a Beta-distribution loss on normalized MUSHRA

RF-GML is intended for deployment settings where no clean reference is available, such as streaming and broadcasting monitoring or archive-scale quality screening. GMLv2 remains explicitly reference-based, but expands applicability to neural audio codecs and neural codec enhancement conditions while retaining distributional outputs (Biswas et al., 2024, Raj et al., 25 Sep 2025).

A separate line of reuse appears in audio-visual quality prediction. In Attentive AV-FusionNet, GML is not rederived but is treated as a frozen, pre-trained, full-reference deep audio quality predictor. Its scalar output $y$8 is used in classical fusion baselines, while deeper-layer features just before the final fully connected layer provide a clip-level embedding

$y$9

with $x$0 in the implementation. These 512-dimensional GML features form the audio branch of the AV model and are fused with six pooled VMAF video features by bidirectional cross-attention followed by self-attention. The same paper also defines a modality relevance estimator whose ablation and feature-change terms quantify how much the final AV prediction relies on the GML audio representation for a given content item (Salaj et al., 21 Sep 2025).

5. Empirical performance, datasets, and applications

The original GML study trains on 67,505 individual subjective scores from internal MUSHRA tests covering stereo and binaural coded audio. It reports that, compared with a baseline system using regression over mean scores, GML yields lower outlier ratios for mean score predictions and straightforward prediction of confidence intervals. It also reports that data augmentation techniques from the image domain, especially CutMix on spectrograms, produce a significant increase in CI prediction accuracy as well as Pearson and Spearman rank correlation of mean scores (Jiang et al., 2023).

RF-GML extends the framework to coded mono, stereo, and binaural audio at 48 kHz without a reference signal. Training uses 67,505 subjective scores from stereo codecs, binaural Dolby Atmos and IVAS-derived material, and multiple content types. Across five test sets totaling 953 signals, RF-GML (deg) achieves overall $x$1, $x$2, and MU $x$3. The paper further reports that RF-GML accurately rates unencoded audio near the top of the scale, distinguishes different levels of coding artifacts, and performs far better than SESQA on general audio, while being comparable to ViSQOL on mono and stereo correlation (Biswas et al., 2024).

GMLv2 broadens the training substrate to 82,191 sample pairs, including 68,503 traditional stereo and binaural codec samples and 14,688 neural-audio-codec samples. Evaluation spans eight test sets: three USAC sets, two binaural sets, mono and stereo NAC sets, and ODAQ. Averaged across these eight sets, GMLv2 reports $x$4, $x$5, and $x$6, corresponding to approximately $x$7 improvement in Pearson correlation, $x$8 improvement in Spearman correlation, and $x$9 reduction in outlier ratio relative to GMLv1 (Raj et al., 25 Sep 2025).

These models are now used in several application regimes. RF-GML is positioned for streaming and broadcasting monitoring, codec development and verification, large-scale music distribution and archiving, and general audio quality assurance (Biswas et al., 2024). Full-reference GML and GMLv2 target codec R&D, especially when rapid approximation of MUSHRA-like outcomes is needed (Raj et al., 25 Sep 2025). In audio-visual QoE modeling, GML features support multimodal prediction and a modality relevance estimator that could potentially enable adaptive bitrate allocation (Salaj et al., 21 Sep 2025).

The principal limitation remains regime dependence. GMLv2 is explicitly reference-based, so deployment requires a time-aligned reference signal (Raj et al., 25 Sep 2025). RF-GML removes that requirement, but its CI prediction capability is not quantitatively evaluated, and the paper notes that subjective tests without a given reference may yield different CI patterns; it also describes the model as relatively heavy at about 15.25M parameters (Biswas et al., 2024).

6. Terminology, broader extensions, and alternate usages

The acronym GML is not semantically stable across fields. In knowledge-graph systems such as KGNet, GML means graph machine learning, not Generative Machine Listener, and refers to GNNs, KG embeddings, and SPARQL$Q_a$0-style model invocation over RDF graphs (Abdallah et al., 2023). In forecasting research, GML can also mean Grey Machine Learning, a semi-parametric grey-model framework for time-series prediction (He et al., 2021). These usages are terminologically unrelated to the audio-quality lineage.

A broader reinterpretation has also appeared in multimodal interaction research, where “machine listener” denotes a system that generates listener behavior rather than predicts perceptual quality. VividListener, trained on ListenerX, is presented as a controllable conditional diffusion model for listener head dynamics conditioned on speaker audio, speaker motion, text descriptions, and continuous valence–arousal Emotional Intensity Tags $Q_a$1. It generates 8-second, 240-frame sequences of 50-dimensional facial expression parameters and 6-dimensional head-pose features, and is explicitly framed as a concrete version of a Generative Machine Listener in the sense of responsive interaction (Li et al., 30 Apr 2025).

ReactMotion extends that embodied interpretation to full-body nonverbal behavior. It defines the task of reactive listener motion generation from speaker utterance as learning

$Q_a$2

where $Q_a$3 can contain text, audio, and emotion cues from the speaker, and $Q_a$4 is a listener motion sequence. The framework uses multimodal tokenization, a T5-based Seq2Seq backbone, and preference-based objectives over Gold, Silver, and Negative listener motions to model the one-to-many nature of human reactions (Luo et al., 16 Mar 2026).

This broader usage suggests a terminological split. In the audio-quality tradition, GML remains a probabilistic evaluator of subjective listening scores. In newer multimodal interaction work, the same phrase is used more loosely for embodied generative listener systems. The common thread is the attempt to emulate some aspect of human listening behavior, but the underlying task, data, and output space differ fundamentally.