CA Score: Metrics for Conversational Alignment

• Conversational Alignment (CA) Score is a family of metrics converting diverse observables like acoustic-prosodic changes, embedding dynamics, and semantic checks into scalar summaries.
• It encompasses distinct formulations such as turn-level entrainment, trajectory-based rewards, context-coherence verification, and explicit human-likeness scoring for targeted applications.
• The score’s adaptable design aids in diagnostic classification, satisfaction prediction, and reinforcement for LLM alignment while challenging the notion of a universal conversational metric.

Conversational Alignment (CA) Score denotes a class of quantitative constructs used to measure whether a conversation exhibits alignment, coherence, or human-likeness, but the term is not standardized across domains. In the literature represented here, it refers to at least four distinct formulations: a turn-level acoustic-prosodic entrainment score in spoken dialogue (Willi et al., 2018), a reward derived from the geometry of an embedding trajectory in multi-turn LLM interaction (Gooding et al., 11 Nov 2025), a semantic-consistency score between a current prompt and a time-weighted dialogue history in a pre-processing framework for human-LLM dialog (Wei, 17 Mar 2026), and a scalar human-likeness reward built from explicit conversational traits for preference optimization of LLMs (Hasan et al., 6 Jan 2026). The shared pattern is the reduction of structured conversational evidence into a scalar or compact summary that can be used for diagnosis, classification, thresholding, or alignment.

1. Scope of the term

Across these works, “CA Score” is a family resemblance term rather than a canonical metric. The underlying observables range from acoustic turn transitions to sentence-embedding trajectories, weighted context embeddings, and LLM-judged trait vectors. Accordingly, the score’s semantics differ: in one setting it measures local entrainment, in another it predicts satisfaction, in another it verifies contextual continuity, and in another it operationalizes conversational human-likeness (Willi et al., 2018, Gooding et al., 11 Nov 2025, Wei, 17 Mar 2026, Hasan et al., 6 Jan 2026).

Formulation	Primary signal	Output or use
Local entrainment	Acoustic-prosodic turn differences	Turn-level $CA_t$ and session features
TRACE	Embedding-trajectory geometry	Scalar reward $r(\tau)$
C.A.P.	Prompt-to-history semantic similarity	Thresholded alignment check
HAL	HL16Q trait ratings	Scalar reward for DPO

A common misconception is to treat CA Score as a single universal formula. The cited work instead uses the label for task-specific scalar constructions whose meaning depends on the representation of conversation and the downstream objective.

2. Local acoustic-prosodic entrainment

In "A Discriminative Acoustic-Prosodic Approach for Measuring Local Entrainment" (Willi et al., 2018), conversational alignment is defined as local acoustic-prosodic entrainment: the tendency of two interlocutors to adapt speech acoustics to each other on a turn-by-turn basis. “Local” is explicit: the unit of analysis is the speaking-turn change rather than the entire conversation. The operational definition is discriminative. Acoustic-prosodic feature differences that capture entrainment should be maximally different between real conversations and sham conversations produced by randomly mixing the speaking turns of the original partners.

Each utterance, defined as an inter-pausal unit up to $0.5\,\mathrm{s}$ of pause, down-sampled to $16\,\mathrm{kHz}$ and loudness normalized, is represented by a $418$-dimensional vector $x_t$. The representation comprises MFCC statistics ($234$ dimensions), Envelope Modulation Spectrum or EMS ($60$), Long-Term Average Spectrum or LTAS ($99$), phonation features ($24$), and intensity ($r(\tau)$0). MFCCs use $r(\tau)$1 coefficients plus $r(\tau)$2 and $r(\tau)$3, summarized by six functionals over $r(\tau)$4 windows with $r(\tau)$5 hops; EMS uses nine octave bands plus the full signal; LTAS uses the same ten bands; phonation features are Praat-derived; and intensity is OpenSMILE mean intensity (Willi et al., 2018).

For each speaker change at time index $r(\tau)$6, if $r(\tau)$7 is the feature vector of the speaker who just finished and $r(\tau)$8 is that of the speaker who just began, the turn-difference is defined as

$r(\tau)$9

with the absolute value taken element-wise. Real dialogues produce a class $0.5\,\mathrm{s}$0, while sham dialogues produce $0.5\,\mathrm{s}$1. Sham conversations are generated by block-randomizing each speaker’s turns in thirds and re-pairing them, which preserves within-speaker statistics while destroying natural turn exchange.

The discriminative subspace is learned separately for the main feature groups using Fisher LDA. With labels $0.5\,\mathrm{s}$2 for real and $0.5\,\mathrm{s}$3 for sham, the within-class and between-class scatter matrices are

$0.5\,\mathrm{s}$4

$0.5\,\mathrm{s}$5

and the projection solves

$0.5\,\mathrm{s}$6

equivalently the generalized eigenproblem

$0.5\,\mathrm{s}$7

Once $0.5\,\mathrm{s}$8 is learned, each turn-difference is projected as $0.5\,\mathrm{s}$9. In the one-dimensional case, the local Conversational Alignment score is

$16\,\mathrm{kHz}$0

The source explicitly notes that smaller $16\,\mathrm{kHz}$1 often indicates closer matching for certain feature sets, or larger values if alignment appears as separation in discriminant space, depending on sign conventions. For a held-out conversation, one scalar $16\,\mathrm{kHz}$2 is computed per feature group for each turn change, and the session-level representation is formed by the functionals minimum, maximum, mean, and standard deviation over each sequence, yielding a $16\,\mathrm{kHz}$3-dimensional vector. Those session features were then used with Logistic Regression, SVM, and Naïve Bayes to predict binary conversational success, defined as low versus high efficiency in a picture-difference task; the best result was $16\,\mathrm{kHz}$4 classification accuracy with Naïve Bayes, outperforming three prior local entrainment measures (Willi et al., 2018).

This formulation is distinctive because it does not assume that alignment is a directly observed similarity in raw feature space. Instead, it learns a discriminative subspace in which real turn transitions differ systematically from temporally disrupted ones.

3. Trajectory-based CA scores from interaction dynamics

"Interaction Dynamics as a Reward Signal for LLMs" (Gooding et al., 11 Nov 2025) defines a CA Score through TRACE, a Trajectory-based Reward for Agent Collaboration Estimation. Here the conversation is represented as an embedding trajectory rather than as text content. If a dialogue has $16\,\mathrm{kHz}$5 alternating turns, each turn is embedded into a $16\,\mathrm{kHz}$6-dimensional semantic space, producing

$16\,\mathrm{kHz}$7

The ordered set $16\,\mathrm{kHz}$8 is the conversation’s embedding trajectory. TRACE extracts summary statistics over this trajectory, referred to as “conversational geometry.” Core quantities include turn-to-turn distance, directional change, curvature, volatility, drift, and self-similarity. Using cosine distance

$16\,\mathrm{kHz}$9

the turn-to-turn step length is

$418$0

with aggregates such as AvgStep and MaxStep. Directional change is based on successive difference vectors $418$1 and $418$2:

$418$3

A curvature proxy is

$418$4

Additional features include LateVolatility over the last $418$5 turns, goal drift if a goal embedding $418$6 is available, and repetition statistics such as SelfSim and MaxSelfSim among model turns. The paper states that in practice it computes around $418$7 such features (Gooding et al., 11 Nov 2025).

These geometric features are assembled into a feature vector $418$8, typically after z-score normalization. A non-linear regressor $418$9, specified as a random forest, is trained to predict human-rated satisfaction. The reward is

$x_t$0

and, if needed, can be affinely rescaled to $x_t$1:

$x_t$2

The reported offline pairwise preference evaluation used $x_t$3 pairs. TRACE-Only achieved $x_t$4 mean accuracy with standard deviation $x_t$5, compared with $x_t$6 and $x_t$7 for an LLM-Only transcript-based baseline. A hybrid TRACE + LLM model reached $x_t$8 with standard deviation $x_t$9. No statistically significant difference was found between TRACE-Only and LLM-Only ($234$0), while the hybrid was significantly better ($234$1) (Gooding et al., 11 Nov 2025).

The paper further characterizes the method as privacy-preserving, because once each turn is mapped to an embedding, no raw text is stored or processed, and as scalable and efficient because embedding-space operations are orders of magnitude faster than running a $234$2B-parameter LLM. It also emphasizes diagnostic transparency: because the score is built from interpretable geometric features, low-scoring trajectories can be analyzed in terms of volatility, drift, or other structural signatures. This directly challenges the assumption that conversational alignment must be assessed from transcript content alone.

4. Context-alignment verification in human-LLM dialog

"A Context Alignment Pre-processor for Enhancing the Coherence of Human-LLM Dialog" (Wei, 17 Mar 2026) uses CA Score in a different sense: as a semantic fit check between a user’s current instruction and a time-weighted representation of recent dialogue history. The framework, called C.A.P., is positioned as a pre-processing module between user input and response generation. Its alignment-verification component defines the score over a semantic-expansion set and a weighted history.

Let $234$3 be the current instruction at time $234$4, let $234$5 be its semantic-expansion set, let $234$6 denote the $234$7 most recent user-model turns with timestamps $234$8, let $234$9 be a fixed sentence-embedding map, and let

$60$0

be cosine similarity. A strictly decreasing temporal-decay weight is assigned to each historical turn: $60$1. The weighted context embedding is

$60$2

optionally normalized as

$60$3

The final CA Score is the maximum similarity between the weighted context and one of the three semantic variants of the current prompt:

$60$4

Equivalently, weighting can be folded into the dot product directly:

$60$5

The semantic-expansion set has three roles: $60$6 is the literal center, $60$7 contains implicit prerequisites or definitions needed to ground $60$8, and $60$9 contains logical extensions, applications, or exploratory next steps. The source specifies that these are generated by a light LLM pass using prompts such as “What must be true before you can do A?” and “How might we extend A?” (Wei, 17 Mar 2026).

Two temporal-decay functions are given. The inverse-proportional form is

$99$0

and the exponential form is

$99$1

where $99$2 is a tunable time-scale hyperparameter. The decision rule compares the CA Score to a threshold $99$3. If $99$4, the prompt is deemed aligned and is passed for normal generation; if $99$5, C.A.P. triggers a structured Clarification Protocol. The threshold is to be tuned on held-out dialogues to balance recall of genuine misalignments against precision, and the source notes that in practice $99$6 often lies in $99$7 for high-dimensional embeddings, though it can be domain-specific or adaptive (Wei, 17 Mar 2026).

The paper explicitly links this design to cognitive science and Common Ground theory. Theoretical motivations include metacognitive reflection, the salience decay of shared commitments, the robustness of cosine similarity in modern sentence embeddings, and the claim that semantic expansion reduces false negatives by covering implied premises and plausible extensions. In this formulation, CA Score is neither a reward nor an entrainment statistic; it is a context-coherence criterion for deciding whether generation should proceed or a repair sequence should be initiated.

5. Human-likeness as a conversational alignment reward

"HAL: Inducing Human-likeness in LLMs with Alignment" (Hasan et al., 6 Jan 2026) uses CA Score as a scalar human-likeness reward, also called the HL16Q or HAL score. The construction begins from contrastive Turing-test dialogues released by Jones & Bergen (2025), contrasting human-human with human-AI interactions. For each dialogue pair, an LLM judge, specified as GPT-5, predicts which side is human and outputs $99$8-$99$9 short natural-language reason statements such as “uses casual slang,” “builds on context,” and “admits not knowing.” Roughly $24$0 such statements are clustered with HDBSCAN over MiniLM embeddings into $24$1 cluster centroids, then distilled by an LLM into $24$2 distinct Likert-style statements, denoted HL32Q (Hasan et al., 6 Jan 2026).

For a dialogue $24$3, the judge rates agreement with each statement $24$4 on a $24$5-$24$6 Likert scale, giving

$24$7

Human dialogues are labeled $24$8 and AI dialogues $24$9, and a logistic regression is fit:

$r(\tau)$00

with loss

$r(\tau)$01

In $r(\tau)$02-fold $r(\tau)$03-split cross-validation, this classifier reaches $r(\tau)$04 accuracy. The $r(\tau)$05 statements with largest $r(\tau)$06 are retained, and the logistic model is retrained on this reduced representation, producing HL16Q with $r(\tau)$07 accuracy (Hasan et al., 6 Jan 2026).

With $r(\tau)$08, the scalar HAL score is

$r(\tau)$09

The source states that the weighting coefficients are exactly the logistic-regression coefficients and that no further normalization is applied beyond the original $r(\tau)$10 scaling of the trait ratings. The retained statements include, among others, “Keeps replies brief and casual without over-explaining,” “Builds on the other person’s message and context,” “Admits not knowing and asks to learn instead of inventing,” and “Gives direct answers about self with concrete personal details.” The reported bias is $r(\tau)$11 (Hasan et al., 6 Jan 2026).

HAL is then used as a reward for preference optimization. For each synthetic persona prompt, $r(\tau)$12 candidate dialogues are generated from a diverse set of models and scored with $r(\tau)$13. All pairs whose score difference is at least $r(\tau)$14 are kept, labeling the higher-scoring dialogue as chosen and the lower-scoring one as rejected, yielding approximately $r(\tau)$15 ranked pairs. The policy $r(\tau)$16 is fine-tuned with Direct Preference Optimization:

$r(\tau)$17

which the source presents as equivalent to

$r(\tau)$18

with $r(\tau)$19. The specified settings are $r(\tau)$20, $r(\tau)$21 epochs, AdamW with learning rate $r(\tau)$22, LoRA rank $r(\tau)$23, and no SFT (Hasan et al., 6 Jan 2026).

Evaluation is reported in two forms. On a separate medical role-play dataset of $r(\tau)$24 human-human and $r(\tau)$25 human-AI dialogues, HL16Q still separates the two with one-sided Mann-Whitney $r(\tau)$26, $r(\tau)$27, and mean $r(\tau)$28. In Chatbot Arena-style A/B testing, $r(\tau)$29 participants played doctor with two chatbots side by side on $r(\tau)$30 held-out personas. Qwen2.5-14B (HAL) beat Qwen2.5-14B Base at $r(\tau)$31 under the paper’s point-based win-rate, and the reported Elo values were $r(\tau)$32 for HAL, $r(\tau)$33 for Base, and $r(\tau)$34 for GPT-4o-mini (Hasan et al., 6 Jan 2026).

This usage of CA Score is noteworthy because the quantity being optimized is not task reward in the narrow sense and not contextual consistency in the C.A.P. sense. It is an interpretable estimate of conversational human-likeness derived from explicit, judge-rated traits.

6. Comparative interpretation

The four formulations differ most clearly in what they regard as the substrate of alignment. The acoustic-prosodic formulation measures adaptation in speech acoustics at each turn boundary (Willi et al., 2018). TRACE measures the geometry of a semantic trajectory through embedding space (Gooding et al., 11 Nov 2025). C.A.P. measures the similarity between a new request and a recency-weighted common ground built from prior turns (Wei, 17 Mar 2026). HAL measures how strongly a dialogue exhibits a learned set of human-like conversational traits (Hasan et al., 6 Jan 2026). These are not interchangeable observables, even though each is summarized as a CA score.

They also differ in supervision and decision role. The local entrainment score is learned discriminatively from real versus sham dialogues and then validated by success prediction (Willi et al., 2018). TRACE uses a random-forest regressor trained on human-rated satisfaction and can serve as a reward model (Gooding et al., 11 Nov 2025). C.A.P. uses no reward-learning stage in the described component; instead, it computes a cosine-based score and compares it against a threshold $r(\tau)$35 to decide whether clarification is required (Wei, 17 Mar 2026). HAL learns a logistic human-versus-AI separator over interpretable traits and then uses the resulting linear score as a transparent reward for DPO (Hasan et al., 6 Jan 2026).

A second distinction is granularity. The acoustic-prosodic construction is explicitly local, producing one $r(\tau)$36 per turn change and only later aggregating to session-level statistics (Willi et al., 2018). TRACE and HAL are dialogue-level scores by design, although TRACE is built from sequential turn geometry (Gooding et al., 11 Nov 2025, Hasan et al., 6 Jan 2026). C.A.P. evaluates the current instruction relative to a sliding history window, so its granularity is interactional but decision-centric rather than fully global (Wei, 17 Mar 2026).

Taken together, these works show that conversational alignment can be formalized through very different representations: low-level acoustics, embedding-space dynamics, context-history similarity, or interpretable trait ratings. This suggests that “alignment” in conversation is not a single latent property with one accepted measurement protocol, but a family of operational targets chosen according to whether the objective is entrainment analysis, success prediction, clarification control, or reward-based model alignment.