Emo-TTA: Test-Time Adaptation for SER

• The paper presents Emo-TTA, a training-free, online EM framework that incrementally updates class-conditional statistics for improved SER under domain shifts.
• Emo-TTA operates sample-wise without modifying model weights, making it practical for streaming and out-of-domain scenarios in speech emotion recognition.
• Experimental results across six cross-corpus benchmarks demonstrate consistent gains over prior test-time adaptation methods in zero-shot and TTA settings.

Searching arXiv for the specified paper and closely related SER test-time adaptation work. arXiv search query: "(Shi et al., 29 Sep 2025) OR EMO-TTA Improving Test-Time Adaptation of Audio-LLMs for Speech Emotion Recognition" Emo-TTA denotes a training-free, sample-wise test-time adaptation framework for speech emotion recognition (SER) with audio-LLMs (ALMs) such as CLAP under distribution shift. It incrementally updates class-conditional statistics in the ALM embedding space through an online Expectation-Maximization (EM) procedure, uses ALM predictions as priors, and re-scores each unlabeled test sample without modifying model weights or prompts. The method is designed for out-of-domain SER settings in which zero-shot ALM performance degrades because of new speakers, recording devices, noise conditions, or taxonomies, and it is evaluated on six cross-corpus SER benchmarks with consistent gains over prior test-time adaptation baselines (Shi et al., 29 Sep 2025).

1. Problem Setting and Scope

Emo-TTA addresses a specific failure mode of modern SER systems: robustness loss under test-time distribution shift. In the formulation used by the method, ALMs such as CLAP perform zero-shot SER by measuring cosine similarity between an utterance’s audio embedding $f(a)$ and a set of text-prompt embeddings $g(t_i)$. In out-of-domain scenarios, this zero-shot accuracy can drop severely, with the provided summary reporting a decline from approximately $49\%$ in-domain to approximately $31\%$ average across six cross-corpus benchmarks (Shi et al., 29 Sep 2025).

The framework is positioned against two common test-time adaptation paradigms. Prior TTA methods either require gradient-based prompt or adapter tuning, which introduces computational overhead and potential overfitting, or require batch-mode access to multiple augmented views, which limits sample-wise and streaming applicability. Emo-TTA is defined in contrast to both: it never updates model weights, it operates on one test sample at a time, and it explicitly estimates test-time class distributions with lightweight EM (Shi et al., 29 Sep 2025).

Within the broader SER adaptation literature, this design aligns with a more general observation that backpropagation-free TTA methods are especially promising for SER, whereas entropy minimization and pseudo-labeling often fail because emotional expression is inherently ambiguous and does not reliably satisfy a single-label confidence assumption (Dong et al., 21 Jan 2026). This suggests that Emo-TTA’s statistical, non-backpropagated adaptation strategy is not an incidental implementation choice but a response to a structural property of the SER problem.

2. Probabilistic Formulation in ALM Embedding Space

Emo-TTA models the incoming unlabeled test stream as a latent mixture of emotion classes in the ALM’s $d$-dimensional audio embedding space. If $F = f(a)$ denotes the embedding of an utterance, each emotion class $y=i$ is assumed to generate embeddings according to a Gaussian

$p(F \mid y=i) = \mathcal{N}(F; \mu_i, \Sigma)$

with class prior $\pi_i$. Under Bayes’ rule, prediction is written as

$$P(y=i \mid F) \propto \pi_i \cdot \exp\!\left[-\frac{1}{2}(F-\mu_i)^\top \Sigma^{-1}(F-\mu_i)\right].$$

The initialization is deliberately lightweight. Class means are set to the ALM text-prompt embeddings, $g(t_i)$0; the shared covariance is initialized as $g(t_i)$1; priors are uniform, $g(t_i)$2; effective class counts are $g(t_i)$3; and the running total is $g(t_i)$4 (Shi et al., 29 Sep 2025).

This choice anchors the initial generative model in the zero-shot semantics of the frozen ALM. A plausible implication is that the method begins from the model’s language-aligned class geometry rather than from arbitrary statistics, then adjusts that geometry as the target stream unfolds.

3. Online Expectation-Maximization and Entropy-Weighted Updates

For the $g(t_i)$5-th test sample with embedding $g(t_i)$6, the E-step computes class responsibilities

$g(t_i)$7

or equivalently

$g(t_i)$8

After observing $g(t_i)$9, the M-step performs incremental updates:

$49\%$0

$49\%$1

$49\%$2

$49\%$3

$49\%$4

The updated state is then assigned as $49\%$5 (Shi et al., 29 Sep 2025).

To mitigate noisy early predictions, Emo-TTA introduces entropy-weighted updates. Let $49\%$6 be the self-entropy of CLAP’s zero-shot distribution on $49\%$7, and define

$49\%$8

The E-step is unchanged, but in the M-step each $49\%$9 is replaced by $31\%$0, and $31\%$1 is incremented by $31\%$2. This down-weights low-confidence samples (Shi et al., 29 Sep 2025).

The resulting procedure is not framed as converging to a fixed optimum under a stopping criterion. Instead, there is no explicit convergence test; the statistics simply evolve as more data arrive. In practice, performance stabilizes after a few dozen samples per class (Shi et al., 29 Sep 2025).

4. Sample-Wise Inference Without Weight Updates

Emo-TTA is explicitly weight-free. No model weights or prompts are ever updated; only the Gaussian parameters $31\%$3 are adapted. Each test utterance is processed independently through four steps: compute the CLAP embedding $31\%$4, zero-shot logits, and entropy $31\%$5; perform the E-step to obtain $31\%$6; perform the M-step to update $31\%$7; and fuse zero-shot and generative Gaussian evidence to produce the final prediction (Shi et al., 29 Sep 2025).

The fusion rule is

$31\%$8

followed by

$31\%$9

The reported hyperparameters are $d$0 for fusion and $d$1 for entropy weighting (Shi et al., 29 Sep 2025).

Because statistics accumulate over the entire test stream, the method adapts continuously to new acoustic domains without storing past samples. This property distinguishes it from batch-dependent TTA schemes and makes it naturally compatible with streaming or sequential inference (Shi et al., 29 Sep 2025).

5. Experimental Evaluation and Reported Results

The evaluation uses six out-of-domain SER benchmarks: IEMOCAP, MELD, RAVDESS, TESS, SAVEE, and CREMA-D. The frozen ALM backbones are CLAP-PANN-14 and CLAP-HTS-AT. Emo-TTA is compared against zero-shot CLAP and prior TTA baselines including CoOp, CoCoOp, Treff-Adapter, TPT, MTA, and ZERO (Shi et al., 29 Sep 2025).

Setting	Baseline summary	Emo-TTA result
CLAP-PANN-14	36.11% avg (Treff-Adapter), 31.37% (zero-shot)	38.02% avg; best on 5/6 datasets
CLAP-HTS-AT	37.10% avg (Treff-Adapter), 35.69% (ZERO)	40.47% avg; best on 5/6 datasets
Foundation audio model comparison	35.77% avg (SELM)	40.05% avg

Under CLAP-PANN-14, Emo-TTA achieves $d$2 average accuracy versus $d$3 for Treff-Adapter and $d$4 for zero-shot CLAP, corresponding to a $d$5 and $d$6 gain, and it is best on $d$7 datasets. Under CLAP-HTS-AT, it reaches $d$8 average accuracy versus $d$9 for Treff-Adapter and $F = f(a)$0 for ZERO, a $F = f(a)$1 gain, again best on $F = f(a)$2 datasets (Shi et al., 29 Sep 2025).

In a strict out-of-domain comparison against foundation audio models, including Whisper, Pengi, and SELM, Emo-TTA scores $F = f(a)$3 average versus $F = f(a)$4 for SELM, with reported gains of $F = f(a)$5 on IEMOCAP, $F = f(a)$6 on CREMA-D, and $F = f(a)$7 on RAVDESS (Shi et al., 29 Sep 2025).

These results situate Emo-TTA as an explicit statistical adaptation mechanism layered on top of frozen ALMs rather than a replacement for the backbone itself. This suggests that a substantial share of the cross-domain error arises from test-distribution mismatch in embedding space rather than from irreparable representational failure.

6. Ablations, Interpretation, and Relation to Adjacent Uses of “Emo-TTA”

The ablation results isolate three components. Removing mean updates, so that $F = f(a)$8 is fixed, reduces average accuracy from $F = f(a)$9 to $y=i$0. Freezing the covariance at $y=i$1 reduces it to $y=i$2. Disabling ALM priors and entropy-weighting reduces it to $y=i$3. The interpretation given in the summary is that adapting class centroids captures domain-shifted cluster centers, updating $y=i$4 models intra-class scatter, and ALM priors stabilize early updates (Shi et al., 29 Sep 2025).

In the context of the broader SER TTA literature, these findings are consistent with the comparative study of eleven TTA methods across three SER tasks. That study reports that backpropagation-free methods are generally the most promising, while entropy minimization and pseudo-labeling often fail because their core assumption of a single, confident ground-truth label is incompatible with the inherent ambiguity of emotional expression (Dong et al., 21 Jan 2026). Emo-TTA extends that general direction by replacing heuristic recalibration with an explicit online density-estimation procedure in ALM embedding space.

The term “Emo-TTA” can also invite a nomenclature ambiguity because nearby literature concerns emotional text-to-speech rather than SER. For example, MsEmoTTS is a multi-scale emotional speech synthesis framework with global-, utterance-, and local-level emotion modules (Lei et al., 2022), and CoCoEmo studies activation steering for controllable emotional TTS in hybrid architectures (Wang et al., 3 Feb 2026). In the usage anchored by the paper titled “EMO-TTA: Improving Test-Time Adaptation of Audio-LLMs for Speech Emotion Recognition,” however, the term refers specifically to test-time adaptation for recognition, not to emotional speech generation (Shi et al., 29 Sep 2025).

Taken together, Emo-TTA represents a particular synthesis of ALM zero-shot semantics, online EM, entropy-aware reliability weighting, and weight-free continual recalibration. Its central claim is not that domain shift in SER disappears, but that a lightweight generative model over frozen embeddings can align predictions with evolving target distributions more effectively than prompt tuning or gradient-based adaptation in the evaluated out-of-domain settings (Shi et al., 29 Sep 2025).