MoCHA: Canonicalization for Motion Retrieval

• The paper introduces MoCHA, a text canonicalization framework that treats captions as samples from a distribution to reduce within-motion text variance.
• It employs both rule-based and learned canonicalizers (LLM-based and FlanT5) to extract motion-recoverable semantics, leading to significant gains in retrieval metrics on HumanML3D and KIT-ML.
• The method enhances cross-dataset transfer by stripping annotator nuisances, thereby sharpening congruence between motion embeddings and text descriptions.

MoCHA is a text canonicalization framework for text–motion retrieval that denoises caption supervision before contrastive training by projecting each caption onto its motion-recoverable content, thereby reducing within-motion text-embedding variance and improving alignment in a shared motion–language space (Warner et al., 24 Mar 2026). In this formulation, captions for the same motion are treated not as deterministic labels but as samples from a distribution of valid descriptions, mixing recoverable motion semantics with annotator-specific nuisance such as style, hedging, and inferred context. MoCHA operationalizes canonicalization as a preprocessing step compatible with retrieval backbones such as MotionPatches (MoPa), and the paper reports gains in both in-distribution retrieval and cross-dataset transfer on HumanML3D and KIT-ML (Warner et al., 24 Mar 2026).

1. Problem setting and motivation

Text–motion retrieval learns a joint embedding space in which natural-language captions and 3D human motions are aligned. Standard practice uses a contrastive InfoNCE or CLIP-style objective and treats each caption paired with a motion as the single positive target. MoCHA is motivated by the mismatch between that objective and the annotation structure of multi-caption motion datasets: the same motion can receive different captions from different annotators, and those captions vary not only in wording but in how much they contain motion-recoverable semantics versus annotator-specific additions (Warner et al., 24 Mar 2026).

The paper distinguishes between semantics recoverable from 3D joint trajectories—such as action type, body parts, directionality, repetitions, and temporal ordering—and nuisance factors that are not encoded in the kinematic signal, including hedging, verbosity, emotion attributions, and inferred intent. HumanML3D captions are described as verbose and intent-laden, whereas KIT-ML captions are terse and templated. This mismatch magnifies the problem in cross-dataset settings, because training on one captioning style miscalibrates the model for the other (Warner et al., 24 Mar 2026).

A central claim of MoCHA is therefore that the supervision signal in text–motion retrieval is distributional rather than deterministic. Treating each caption as the sole positive target pulls the motion embedding toward a moving text target, injects variance into the positive set, and weakens alignment. Canonicalization is introduced as the mechanism for stripping captions down to the motion-recoverable semantics they actually encode while suppressing annotator nuisance.

2. Formalization of caption variance

Let $m$ denote a motion and $\{c_i\}$ its multiple captions. The paper models each caption as a draw from a distribution over valid descriptions formed by mixing motion-recoverable semantics $s$ with annotator nuisance $a$:

$$t \sim p(t \mid s, a), \qquad a \sim p(a), \qquad p(t \mid s) = \int p(t \mid s, a)\, p(a)\, da.$$

With a motion encoder $f_m(\cdot)$ and a text encoder $f_t(\cdot)$ mapping inputs into a shared space with cosine similarity $\mathrm{sim}(\cdot,\cdot)$, the standard InfoNCE loss over a batch of size $B$ is written as

$$L = \sum_{i=1}^{B} -\log \frac{\exp(\mathrm{sim}(f_m(m_i), f_t(c_i))/\tau)}{\sum_{j=1}^{B} \exp(\mathrm{sim}(f_m(m_i), f_t(c_j))/\tau)},$$

where $\{c_i\}$0 is the temperature (Warner et al., 24 Mar 2026).

For a motion with $\{c_i\}$1 captions, MoCHA characterizes the spread of positives through the mean and variance of caption embeddings:

$\{c_i\}$2

An empirical alternative used in the paper is the mean pairwise cosine dissimilarity

$\{c_i\}$3

with the decomposition

$\{c_i\}$4

To evaluate cluster tightness and separation, the paper reports four embedding-geometry statistics: Intra Sim, the average cosine among captions for the same motion; Align, the average cosine between each caption and its paired motion embedding; Inter NN Sim, the average cosine to the nearest negative caption; and Sep Ratio, defined as $\{c_i\}$5 (Warner et al., 24 Mar 2026).

This formalization grounds MoCHA’s main intervention. If the caption distribution contains nuisance variation unrelated to motion, then reducing that variation before encoding should tighten the positive cluster around $\{c_i\}$6, lower gradient variance, and improve the contrastive objective’s ability to separate positives from negatives.

3. Canonicalization as the core method

MoCHA defines canonicalization as a mapping from a raw caption $\{c_i\}$7 to a canonical form $\{c_i\}$8 that preserves motion semantics while suppressing annotator nuisance:

$\{c_i\}$9

The intended canonical form follows a normalized “verb [object] [limb] [direction] $s$0 next action” style that retains temporal structure while removing subjects, filler, hedges, and inferred context. The paper’s illustrative example transforms “a person nervously walks forward, stops, turns around, and walks back” into “walk forward $s$1 stop $s$2 turn around $s$3 walk back,” and “a person walks forward to greet someone” into “walk forward” (Warner et al., 24 Mar 2026).

MoCHA considers both deterministic and learned canonicalizers. Rule-based methods such as stopword stripping remove obvious filler and already improve cross-dataset transfer, which the paper presents as evidence that canonicalization is a general principle rather than an artifact of a particular LLM. Learned canonicalizers, however, provide larger gains because they preserve sequential structure and fine-grained motion semantics while suppressing nuisance more effectively.

Two learned variants are introduced. The first is an LLM-based canonicalizer using GPT-5.2, prompted to keep action verbs, directions, body parts or limbs, objects, poses, repetitions, and motion-relevant manner, while removing subjects, hedges, and discourse filler. The second is a distilled FlanT5-base canonicalizer of approximately 250M parameters, trained on pairs $s$4, where the targets are GPT-5.2 outputs. The FlanT5 model is trained on about 168K pairs combining canonicalized paraphrases from TMR++ and original training captions. Its main practical role is to preserve most of the gains of the LLM variant while requiring no LLM at inference time (Warner et al., 24 Mar 2026).

A frequent misconception addressed by the ablations is that any surface transformation of the text could help. The paper reports that backtranslation does not help and often hurts, which it interprets as confirmation that the benefit comes from removing annotator nuisance rather than from applying arbitrary text transformations.

4. Training strategy and integration with retrieval backbones

MoCHA is explicitly architecture-agnostic. It modifies only the text channel and operates as a preprocessing stage before text encoding. In the paper’s main experiments it is applied to MoPa, which uses a ViT-B/16 motion encoder over 22-joint 3D inputs, a DistilBERT text encoder with CLS pooling, a 256-dimensional shared embedding space, and symmetric InfoNCE with frozen temperature $s$5 (Warner et al., 24 Mar 2026).

The simplest use of MoCHA is to canonicalize all training captions, and optionally test captions, before text encoding. Canonical captions use an arrow-separated sequence format that emphasizes temporal ordering, for example “walk forward $s$6 stop $s$7 turn around $s$8 walk back.” Because canonicalization can compress useful nuance, the paper introduces a dual-pass contrastive objective called Blend training:

$s$9

The canonicalized pass anchors the embedding space on motion-recoverable semantics with lower-variance gradients, while the original-caption pass regularizes for natural queries and preserves useful detail that the canonicalizer may compress. In the training-strategy ablations, Cardinal—pure canonical training—delivers the largest gains on templated KIT-ML but risks over-canonicalization on verbose HumanML3D, whereas Blend and Blend-Rev provide the best balance across datasets (Warner et al., 24 Mar 2026).

From a deployment perspective, the distinction between the two learned canonicalizers is significant. The GPT-5.2 variant adds inference-time overhead and an external dependency, while the FlanT5 variant runs locally, avoids LLM calls, reduces latency, and is reported to yield more consistent normalization. A practical implication is that MoCHA can be treated as a lightweight text-normalization layer for motion–language systems rather than as a redesign of the retrieval model itself.

5. Empirical performance and embedding geometry

The evaluation uses HumanML3D, with approximately 15k motions and 45k captions, and KIT-ML, with approximately 3.9k motions and 6.3k captions. Retrieval is measured under the DsPair protocol using text-to-motion and motion-to-text Recall@k and Median Rank (Warner et al., 24 Mar 2026).

On the MoPa backbone, the paper reports the following text-to-motion R@1 results:

Model	HumanML3D	KIT-ML
MoPa	10.8%	14.0%
MoCHA (LLM)	13.9%	24.3%
MoCHA (T5)	13.3%	22.1%

These correspond to gains of +3.1 percentage points and +10.3 percentage points for the LLM variant, and +2.5 percentage points and +8.1 percentage points for the T5 variant. The paper states that gains are consistent across R@5 and R@10 and in both retrieval directions (Warner et al., 24 Mar 2026).

The paper attributes these gains to measurable changes in embedding geometry. Canonicalization reduces within-motion text-embedding variance by 11–19%, with reported reductions of 11% on HumanML3D and 19% on KIT-ML. Gradient variance and InfoNCE softmax entropy also decrease. At the cluster level, Intra Sim and Align increase, Inter NN Sim slightly decreases, and Sep Ratio improves by 8% on HumanML3D and 25% on KIT-ML, which is presented as evidence for tighter positive clusters and better separation from negatives (Warner et al., 24 Mar 2026).

Cross-dataset transfer is where the method shows its largest relative effects. Training on HumanML3D and testing on KIT-ML improves text-to-motion R@1 by up to 94% relative over MoPa, while the reverse direction improves by 52%. The proposed mechanism is that standardizing language reduces sensitivity to dataset-specific nuisance $a$0 and aligns the representation to motion semantics shared across datasets (Warner et al., 24 Mar 2026).

Qualitative examples reinforce this interpretation. The caption “a figure leans forward as though impersonating an airplane” is canonicalized to “lean forward (imitate airplane),” which separates it from head-rotation motions. Similarly, “figure appears to be fighting or dancing” becomes “dance,” removing annotator uncertainty and retrieving dancing rather than boxing.

6. Limitations, misconceptions, and broader implications

The principal limitation identified by the paper is over-canonicalization. Removing too much linguistic content can discard subtle motion-relevant cues, especially manner information that genuinely affects kinematics. Blend training mitigates this but does not eliminate it. Dependence on LLM quality and prompt design is another limitation: poor prompts can under- or over-denoise, and the distilled model inherits the prompt biases of the LLM-generated targets (Warner et al., 24 Mar 2026).

The paper also notes ambiguity in what counts as motion-recoverable semantics. Age-related gait characteristics and affect cues may weakly correlate with kinematics but are not always unambiguously recoverable from 3D joint coordinates alone. This creates a boundary problem for canonicalization. In addition, canonicalizers may encode stylistic or cultural preferences from the data used to build them, so curation of canonicalization pairs matters.

Several misconceptions are addressed directly by the empirical analysis. One is that MoCHA is tied to a specific retrieval architecture; the paper instead presents it as architecture-agnostic preprocessing and reports compatibility with MoPa and TMR-style encoders. Another is that the gains come only from external LLM access; the distilled FlanT5 model preserves most of the improvements while eliminating LLM calls at inference. A third is that the benefit arises from generic paraphrasing; the backtranslation ablation argues against that interpretation (Warner et al., 24 Mar 2026).

The broader significance advanced by the paper is that canonicalization may generalize beyond motion to other multimodal retrieval settings in which captions are sampled from a distribution rather than acting as deterministic labels. The authors explicitly mention vision–language, audio–text, and robotics instructions as candidate domains. This suggests a more general view of supervision in contrastive multimodal learning: when annotations mix recoverable signal with annotator nuisance, standardizing language to the recoverable content can reduce supervision noise, sharpen alignment, and improve transfer (Warner et al., 24 Mar 2026).