Curved Contrastive Learning for Dialogue Planning

• The paper introduces CCL, a method that captures dialogue trajectories by training a bi-encoder with continuous cosine similarity targets based on turn distance and temporal order.
• It employs a combined training strategy with NLI pre-training and specialized tokens for encoding directional and speaker cues to shape the conversational embedding space.
• Empirical results demonstrate that CCL achieves competitive Hits@ metrics and reduces encoding overhead, enabling effective zero-shot response ranking and multi-turn planning.

Searching arXiv for the target paper and related works cited in the provided data. Curved Contrastive Learning (CCL) is a self-supervised representation learning method for multi-turn dialogue that trains a sentence-level bi-encoder to encode not only semantic relatedness between utterances, but also their relative turn distance and temporal direction within a conversation (Erker et al., 2022). In the formulation introduced in "Imagination is All You Need! Curved Contrastive Learning for Abstract Sequence Modeling Utilized on Long Short-Term Dialogue Planning" (Erker et al., 2022), the latent space is described as a conversational “curved” space-time: cosine similarity is trained to reflect how close one utterance is to another along the dialogue trajectory, and whether one utterance lies before or after another. The resulting representations are used for zero-shot response ranking toward a future goal, next-utterance selection, and ordering of multiple future goal utterances, while preserving an efficient inference regime in which only the last utterance needs to be encoded (Erker et al., 2022).

1. Conceptual basis and problem formulation

CCL is motivated by an analogy to curvature in space-time. The central intuition is that conversations exhibit trajectories in which earlier utterances “curve toward” later ones, especially toward latent goals. The paper reports an empirical observation on DailyDialog using off-the-shelf sentence transformers: when a turn is selected as a goal utterance and cosine similarity is computed between that goal and earlier turns, similarity tends to increase as the dialogue approaches the goal turn (Erker et al., 2022). This behavior is treated as evidence that dialogue embeddings can be organized as paths in a manifold-like space whose global structure reflects conversational progression.

The learning problem is formulated over a multi-turn dialogue $d = (u_0, u_1, \dots, u_T)$. Rather than learning only whether two utterances are semantically similar, CCL aims to encode three properties simultaneously: latent distance should reflect relative turn distance; direction should reflect temporal order; and these properties should hold transitively enough to support multi-turn planning (Erker et al., 2022). In effect, the model is trained so that $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$ decreases as $|i-0|$ grows, while distinguishing “$u_0$ before $u_i$” from “$u_i$ before $u_0$.”

This differentiates CCL from standard contrastive learning schemes such as binary positive-negative objectives. In the description provided for CCL, the supervision is continuous rather than categorical: target similarity values are proportional to turn distance within a sliding window, and directionality is injected through dedicated tokens (Erker et al., 2022). The paper therefore frames the learned representations as forward-entailing: earlier utterances are trained to be closer, in a graded sense, to plausible later utterances.

A common misconception is that CCL introduces an explicit non-Euclidean manifold. The paper explicitly rejects that reading. The embedding space remains mathematically Euclidean, with standard transformer encoders and cosine similarity; “curvature” is an emergent property of the learned similarity structure rather than a prescribed hyperbolic or Riemannian geometry (Erker et al., 2022).

2. Architecture and learning objective

CCL uses a Sentence-BERT–style Siamese bi-encoder with shared transformer weights, instantiated with BERT and RoBERTa variants (Erker et al., 2022). Each utterance is encoded separately, using standard sentence-transformer pooling, and mapped to a sentence embedding suitable for cosine-similarity comparisons. The architectural contribution is not a new encoder topology, but a training objective and tokenization scheme that shape the embedding space along a dialogue-time axis.

Direction is encoded through prepended special tokens: $[B]$ for $[BEFORE]$ and $[A]$ for $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$0 (Erker et al., 2022). Speaker-aware variants add $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$1 and $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$2 to $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$3, corresponding to even and odd distances, in order to tag who will speak relative to the goal utterance. At inference time, context or history utterances are encoded with $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$4, while candidate future utterances are encoded with $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$5. Cosine similarity in this “imaginary space” is then interpreted as a measure of conversational closeness and reachability (Erker et al., 2022).

Training pairs are constructed from a sliding window of length $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$6. For a window $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$7, the anchor $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$8 is paired with later utterances $\cos(\mathbf{z}_{u_0}, \mathbf{z}_{u_i})$9, and the target similarity is a linear function of positional distance (Erker et al., 2022):

$|i-0|$0

For $|i-0|$1, this yields target similarities $|i-0|$2 and $|i-0|$3 for distances $|i-0|$4 and $|i-0|$5, respectively (Erker et al., 2022). Negative training pairs are assigned $|i-0|$6 for direction-swapped pairs and for random utterances sampled from elsewhere in the corpus. This makes wrong temporal direction and unrelated content collapse toward zero similarity.

The similarity function is standard cosine similarity:

$|i-0|$7

The paper states that training uses a Cosine Similarity loss for the DailyDialog conversational objective, described in the structured explanation as the sentence-transformer style mean squared error between predicted cosine similarity and the target score $|i-0|$8 (Erker et al., 2022). There is no InfoNCE temperature term or margin-based ranking term in the formulation as presented.

CCL is also combined with NLI pre-training. The training pipeline first uses AllNLI, consisting of SNLI and MultiNLI, with a standard sentence-transformer Softmax Loss for entailment, contradiction, and neutral labels, and then applies Curved Contrastive Learning on DailyDialog (Erker et al., 2022). This mixed objective is described as anchoring the representation space in semantic entailment while shaping it along conversational time.

3. Emergent curved space and entailment strength

The paper’s notion of curvature is tied to the emergence of global structure from local constraints. CCL is trained only on local windows, but the resulting embedding space is intended to exhibit non-local properties: sequences of utterances form paths that bend toward future goals, and local directional similarities induce global reachability relations (Erker et al., 2022). Appendix visualizations are reported to show average similarity decaying with distance and low similarity for false-direction pairs, alongside a t-SNE view in which earlier and later turns occupy distinguishable regions when marked with $|i-0|$9 and $u_0$0 (Erker et al., 2022).

The mechanism used to operationalize this non-local structure is entailment strength. For a candidate future utterance $u_0$1 and a dialogue history $u_0$2, the paper defines

$u_0$3

Here every history utterance is encoded independently with $u_0$4, the candidate is encoded with $u_0$5, and the sum aggregates support from the history (Erker et al., 2022). The description in the source material likens this to an attention-like measure, although the operational object is simply a sum of cosine similarities.

This quantity underlies both sequence scoring and planning. Because utterances are encoded independently, a future utterance that is many turns away from the current history can still be projected into the same space and assigned an entailment strength. The paper treats this as a form of “imagination”: the model can assess how close a candidate lies to a future goal without simulating full conversation trees (Erker et al., 2022). A plausible implication is that CCL substitutes geometric scoring for explicit rollout.

The forward-entailing behavior is described as transitive. If $u_0$6 is close to $u_0$7 and $u_0$8 is close to $u_0$9, then $u_i$0 tends also to be relatively close to $u_i$1, even when no direct pair was explicitly supervised (Erker et al., 2022). This is presented as the basis for multi-turn planning and sequence likelihood estimation. Another common misconception follows from this point: CCL is not trained with an autoregressive sequence-likelihood objective, yet it yields usable sequence scores by aggregating pairwise directional similarities.

4. Planning and sequence-modeling procedures

CCL is applied to short-term planning as a response-ranking mechanism. In the reported evaluation protocol, a dialogue $u_i$2 is split into a context $u_i$3, a true next utterance $u_i$4, and a goal utterance $u_i$5, where $u_i$6 is several turns ahead (Erker et al., 2022). A generative dialogue model is then prompted with the context: DialoGPT is used for DailyDialog and GODEL for MDC. Each generator produces 100 candidate utterances using nucleus sampling with $u_i$7 and $u_i$8 (Erker et al., 2022). Candidates are embedded, compared against the goal in the learned space, and ranked by cosine similarity, after which the evaluation measures where the true next utterance appears in that ranking.

The paper characterizes this use as zero-shot planning because the bi-encoder is trained only with NLI and CCL, not with supervised planning labels (Erker et al., 2022). In deployment terms, the described workflow is to generate multiple candidate replies, rank them by closeness to a specified future goal, and greedily select the top candidate, thereby “following the curve” toward that goal.

For long-term planning, the model is given a history $u_i$9 and a set of future goals $u_i$0, all of which occur later in the same dialogue (Erker et al., 2022). The task is to predict the order in which these goals will appear. Three strategies are described.

The first is Imaginary Embedding Chains (IEC), which scores a candidate order $u_i$1 by summing pairwise cosine similarities between successive goals:

$u_i$2

The best order is then

$u_i$3

where $u_i$4 is the set of all permutations (Erker et al., 2022).

The second is Imaginary Embedding Chains + Curving (IEC + CU), which augments the chain score with history-dependent entailment terms:

$u_i$5

As described in the source material, this rewards the first goal for being closer to the history and penalizes later goals for being too close (Erker et al., 2022).

The third is Greedy Curving (GC), which chooses the next goal directly:

$u_i$6

In evaluation, GC only needs to pick the immediate next goal rather than the entire permutation (Erker et al., 2022).

The same entailment-strength formalism is used for next-utterance selection. Given a history and a set of candidate next utterances $u_i$7, candidates are ranked by $u_i$8, and the normalized average rank of the true next utterance is used as the evaluation metric (Erker et al., 2022). The paper therefore treats $u_i$9 as a sequence-likelihood surrogate, while emphasizing that the model was never trained with a conventional language-modeling objective.

5. Empirical findings

The empirical study is conducted on DailyDialog and the Microsoft Dialogue Challenge (MDC) (Erker et al., 2022). DailyDialog is described as a chit-chat corpus with average 7.84 turns, 11,118 training dialogues, and 1000 test dialogues. MDC is described as a task-oriented corpus covering movie, restaurant, and taxi domains, with average 6.51 turns, 9088 training dialogues, and test sets formed by holding out 333 dialogues per domain (Erker et al., 2022).

For short-term planning, the reported metrics are Hits@5, Hits@10, Hits@25, Hits@50, and average rank for the true next utterance relative to 100 sampled candidates (Erker et al., 2022). On DailyDialog without speaker tokens and for even goal distances, the aggregated results are given as approximately Hits@5 $u_0$0, Hits@25 $u_0$1, and average rank $u_0$2. On DailyDialog with speaker tokens and odd distances, the reported values are approximately Hits@5 $u_0$3, Hits@10 $u_0$4, Hits@25 $u_0$5, Hits@50 $u_0$6, and average rank $u_0$7 (Erker et al., 2022). The study also reports that, in many settings, planning three turns ahead is easier than planning two turns ahead, and gives DailyDialog results of $u_0$8 Hits@5 and $u_0$9 Hits@10 for some 3-turn-ahead setups with history length 5 and goal distance 3 (Erker et al., 2022).

For next-utterance selection, the baseline set includes DialogRPT, ConvRT, BM25, and non-curved ablations (Erker et al., 2022). On DailyDialog, CCL with speaker tokens and entire history is reported to achieve a normalized average rank in the top 10% across history lengths. For sequences longer than 2 turns, it is reported to outperform DialogRPT by at least 2.8 percentage points and ConvRT by about 0.5 points (Erker et al., 2022). On MDC, the paper reports weaker overall results and stronger dependence on intent type, while noting that using only the last utterance often works better in task-oriented settings.

For long-term planning over three goals, the paper reports an average rank of approximately 1.87 out of 6 for the correct order across all models and datasets (Erker et al., 2022). On DailyDialog, IEC without speaker tokens yields approximately Hits@1 $[B]$0, Hits@4 $[B]$1 for partially ordered evaluation, reverse-order identification around $[B]$2, and average rank around $[B]$3. IEC + Curving is described as similar or slightly better when goals are close in distance, and Greedy Curving achieves Hits@1 around $[B]$4 on DailyDialog for classic embeddings, with better results when speaker tokens are used (Erker et al., 2022). On MDC, IEC + Curving and GC are reported to do even better because of the more rigid task structure, with GC reaching Hits@1 approximately $[B]$5 and average rank $[B]$6 without speaker tokens (Erker et al., 2022).

These findings are summarized in the paper as evidence that the learned space encodes enough temporal structure to support multi-turn planning, response re-ranking toward goals, and sequence scoring (Erker et al., 2022). The paper also presents this as evidence for planning capability in chit-chat dialogue, not only in task-oriented settings.

6. Efficiency, interpretive limits, and relation to adjacent work

A major claimed advantage of CCL is inference efficiency. In contrast to sequence encoders that repeatedly encode conversational context, CCL encodes utterances independently and can reuse stored embeddings (Erker et al., 2022). For online scoring, only the newest utterance needs to be encoded, after which pairwise similarities can be computed against precomputed history embeddings. The appendix comparison cited in the source material states that, for DailyDialog next-utterance selection with history length at most 10, ConvRT must generate 6219 context representations involving 26,733 utterance encodings, whereas imaginary embeddings encode each utterance once and reuse the result, reducing encoding computations by a factor of about $[B]$7 (Erker et al., 2022). The paper further notes that cosine similarities can be batched as a single matrix multiplication over L2-normalized embeddings, which suits large-scale retrieval and ranking.

The analysis section distinguishes plannable from unplannable conversations. Two research questions are posed: whether chit-chat conversations have planning capability, and which characteristics make dialogue planning possible (Erker et al., 2022). The reported findings state that request intents in MDC are much easier to plan than inform intents, with more peaked ranking distributions indicating clearer pathways toward goals. Inform turns are described as often supplying details that do not strongly constrain future conversation, making the curved space “flatter” and planning harder. In DailyDialog, speaker tokens reveal that utterances from the goal speaker tend to be more similar to the goal earlier in the dialogue, which the paper interprets as speaker-specific planning capacity (Erker et al., 2022).

The limitations are explicit. The evaluation splits short-term and long-term planning tasks at fixed positions regardless of whether planning is actually possible there, conflating model performance with the intrinsic plannability or unplannability of the example (Erker et al., 2022). Candidate utterances for short-term planning are generated by DialoGPT or GODEL using sampling and may include odd but goal-reaching replies that humans would not regard as likely. In MDC, history-based curved next-utterance selection is often worse than using only the last utterance. The paper also emphasizes that there is no explicit geometric guarantee: despite the rhetoric of curvature and relativity, the underlying space is still Euclidean with cosine similarity (Erker et al., 2022).

Within the broader literature, CCL is positioned against standard contrastive learning, sentence-embedding methods anchored by NLI, dialogue-specific contrastive representation learning, retrieval models such as ConvRT, feedback-trained ranking models such as DialogRPT, and word-level long-term control approaches such as Ramakrishnan et al. (Erker et al., 2022). Its novelty, as stated in the source material, lies in treating conversational representation learning as a curved contrastive problem in which continuous similarity targets encode turn distance, special tokens encode direction and speaker structure, and forward-entailing, transitive properties emerge that can be exploited for both planning and sequence modeling (Erker et al., 2022). A plausible implication is that CCL occupies an intermediate position between geometric representation learning and practical dialogue control: it does not impose an explicit non-Euclidean manifold, yet it uses learned geometry as an operational planning interface.