Cross-Space In-Betweening

• Cross-Space In-Betweening is a representational strategy that performs interpolation across diverse spaces—such as pixel, graph, latent, and coordinate domains—to preserve structural integrity and precise control.
• It reformulates conventional dense interpolation by leveraging structured representations and conditioning mechanisms (e.g., masked diffusion, FiLM modulation) to reduce ambiguity and enhance quality.
• Key applications include two-character interaction synthesis, cartoon line inbetweening, and motion models that integrate arbitrary temporal and spatial constraints for robust long-duration animation.

Searching arXiv for papers explicitly using or closely related to “Cross-Space In-Betweening” and adjacent inbetweening formulations. Cross-Space In-Betweening denotes a family of inbetweening formulations in which interpolation is not performed solely within a single signal domain, such as pixels or joint trajectories, but instead across multiple representation spaces, conditioning spaces, or coordinate frames. In recent work, the term has been used most explicitly for two-character interaction synthesis, where each character is modeled in both its own keypose space and the counterpart’s relative space (Zhang et al., 30 Sep 2025). Closely related formulations appear in cartoon line inbetweening that converts raster drawings into graphs and rasterizes back after fusion (Siyao et al., 2023), in motion models that accept arbitrary spatial and temporal constraints or partial keyframes (Cohan et al., 2024), and in systems that bridge real-video priors and rendered arbitrary-character animation (Yun et al., 11 Mar 2025). Taken together, these works define Cross-Space In-Betweening as a shift from direct interpolation in a single observation space toward interpolation mediated by structured latent, geometric, relational, or conditional spaces.

1. Definition and scope

In the most explicit usage, Cross-Space In-Betweening is introduced for “long-horizon interaction in-betweening” of two characters, where interactions are modeled “across different conditioning representation spaces” (Zhang et al., 30 Sep 2025). The motivation is that densely interacting characters impose constraints that are not well captured by single-character formulations: the system must preserve “precise spatial-temporal correspondence between the characters,” maintain interaction quality, and simultaneously steer both motions toward predefined keyposes (Zhang et al., 30 Sep 2025).

A broader reading of the literature shows that the same underlying idea arises whenever the interpolation problem is reformulated through a representation better aligned with structure than the raw input domain. In “Deep Geometrized Cartoon Line Inbetweening,” raster line drawings are converted into graphs of endpoints and connectivity, the inbetween is synthesized in graph space, and the result is rasterized afterward (Siyao et al., 2023). In “Flexible Motion In-betweening with Diffusion Models,” constraints can be placed arbitrarily across frames and joints, so the conditioning space is no longer a fixed set of full poses but a masked spatio-temporal subset of the motion tensor (Cohan et al., 2024). In “AnyMoLe,” the problem spans rendered animations, arbitrary joint structures, and video diffusion priors learned from real videos, requiring explicit mechanisms to bridge those spaces (Yun et al., 11 Mar 2025).

This suggests that Cross-Space In-Betweening is best understood as a representational strategy rather than a single architecture. The common premise is that interpolation becomes more tractable when the model operates in spaces where correspondence, control, visibility, or interaction can be stated explicitly.

2. Representation-space reformulation

A central technical pattern is the replacement of dense observation-space interpolation with sparse or structured representations. In cartoon line inbetweening, an image $I$ is mapped to a graph $G=\{V,T\}$, where $V$ contains endpoint coordinates and $T$ is a binary adjacency matrix encoding topology (Siyao et al., 2023). The task is then reframed from image warping to graph fusion with vertex repositioning: $$G_0=(V_0,T_0),\quad G_1=(V_1,T_1)\ \rightarrow\ G_t=(V_t,T_t).$$ This avoids the ambiguity of white-space-dominated raster images and reduces blurring artifacts that damage line structure (Siyao et al., 2023).

In human motion, a different kind of space shift appears. CondMDI replaces the common relative-root formulation with a global-root representation so that arbitrary keyframe constraints can be imposed directly in absolute coordinates at any time index (Cohan et al., 2024). The same paper further uses random masking over frames and joints, so the conditioning space itself becomes variable and sparse rather than fixed (Cohan et al., 2024). In diffusion over motion implicit neural representations, a motion sequence is represented as a neural function $\mathbb{D}(z,t)\mapsto (X_t,R_t)$, which maps a latent code and continuous time to pose variables (Fan et al., 12 May 2026). Sparse keyframes are not interpolated directly; instead, they condition diffusion in the INR latent space, and the final sequence is decoded from that continuous representation (Fan et al., 12 May 2026).

For arbitrary-character animation, AnyMoLe similarly decomposes the problem into a video-generation space and a motion-parameter space. It first synthesizes context-aware inbetween frames with a video diffusion model, then performs “motion-video mimicking” to recover root positions and per-joint rotations consistent with the generated video (Yun et al., 11 Mar 2025). This is cross-space in a literal sense: motion is inferred by aligning between generated image sequences and character-specific kinematic parameters.

These reformulations are heterogeneous, but their functional role is consistent. They move the interpolation problem into spaces where structural invariants—graph connectivity, absolute pose, latent motion manifolds, or articulated kinematics—are easier to preserve.

3. Cross-space correspondence and conditioning mechanisms

Once multiple spaces are introduced, the core problem becomes how to transfer information between them. Different works instantiate this transfer with different operators.

AnimeInbet uses vertex geometric embedding and a vertex correspondence Transformer with alternating self-attention and cross-attention to match graph vertices across frames (Siyao et al., 2023). The self-attention and cross-attention layers are given as

$$SA(F) = \text{softmax}\left(\frac{\mathcal{Q}(F)\mathcal{K}(F)^T}{\sqrt{C}}\right)\mathcal{V}(F),$$

$$CA(F_0,F_1) = \text{softmax}\left(\frac{\mathcal{Q}(F_0)\mathcal{K}(F_1)^T}{\sqrt{C}}\right)\mathcal{V}(F_1),$$

followed by a correlation matrix and optimal transport for mutual matching (Siyao et al., 2023). Here, cross-space transfer is implemented as cross-graph correspondence.

CondMDI uses a masked conditioning mechanism. During training and inference, observed entries from arbitrary frames and joints are inserted into the noisy motion sample, and a binary mask is concatenated to indicate which values are constraints (Cohan et al., 2024). The masked sample is

$$\tilde{\mathbf{x}}_t = m \odot \mathbf{x}_0 + (1-m)\odot \mathbf{x}_t,$$

and the model is trained with

$$\mathcal{L} = \mathbb{E}_{(\mathbf{x}_0,c),t}\left[\left\|\mathbf{x}_0 - G_\theta(\tilde{\mathbf{x}}_t,t,c,m)\right\|^2\right]$$

(Cohan et al., 2024). This makes the conditioning operator independent of a fixed pose layout and supports arbitrary dense-or-sparse keyframe placement, partial keyframe constraints, and text conditioning (Cohan et al., 2024).

The interaction-focused Cross-Space In-Betweening model uses an explicit transformation into the counterpart’s coordinate frame: $G=\{V,T\}$0 then projects the relative-space motion into a latent space and applies FiLM modulation: $G=\{V,T\}$1

$G=\{V,T\}$2

$G=\{V,T\}$3

(Zhang et al., 30 Sep 2025). This is one of the clearest examples of a cross-space bridge: motion predicted in one representation is transformed into another representation and used to modulate the original prediction.

Diffusion over INRs uses cross-attention to inject sparse keyframe information into latent diffusion and then applies Implicit Manifold Guidance: $G=\{V,T\}$4 (Fan et al., 12 May 2026). The guidance couples geometric fidelity to keyframes with manifold adherence in latent space.

Across these formulations, the operational meaning of “cross-space” is the same: the model must learn or enforce a map between heterogeneous spaces rather than assume a single homogeneous interpolation domain.

4. Major methodological families

The current literature can be organized into several methodological families, each emphasizing a different notion of space.

Family	Representative formulation	Representative paper
Geometric graph reformulation	Raster $G=\{V,T\}$5 graph $G=\{V,T\}$6 raster	(Siyao et al., 2023)
Masked spatio-temporal conditioning	Arbitrary frames/joints with partial constraints and text	(Cohan et al., 2024)
Latent continuous motion manifolds	Diffusion over INR latent space with guided sampling	(Fan et al., 12 May 2026)
Explicit structural control in image generation	Skeletons plus mixed trajectory maps	(Pan et al., 15 Jul 2025)
Arbitrary-character bridging	Video diffusion plus motion-video mimicking	(Yun et al., 11 Mar 2025)
Interaction-relative representation learning	Own-keypose space plus counterpart-relative space	(Zhang et al., 30 Sep 2025)

The graph-based family is exemplified by AnimeInbet, which uses vertex geometric embedding, a vertex correspondence Transformer, repositioning propagation, and a visibility predictor to synthesize intermediate line drawings while preserving topology and line sharpness (Siyao et al., 2023).

The masked-diffusion family is represented by CondMDI, where a single unified model handles arbitrary dense-or-sparse keyframes, partial keyframe constraints, and text conditioning without retraining or separate modules for different constraint placements (Cohan et al., 2024).

The continuous-latent family is represented by motion in-betweening with latent diffusion over INRs. There, the INR decoder defines a continuous-time motion function, while the latent diffusion model samples plausible motions from sparse keyframes and IMG refines the sampling trajectory (Fan et al., 12 May 2026).

The explicit structural guidance family appears in StructInbet, which conditions a Stable Diffusion UNet using skeletons and rasterized mixed trajectory maps via a ControlNet encoder, and uses bidirectional reference attention to preserve appearance consistency from both endpoint keyframes (Pan et al., 15 Jul 2025). The attention is

$G=\{V,T\}$7

(Pan et al., 15 Jul 2025).

The arbitrary-character family, as in AnyMoLe, uses two-stage frame generation, ICAdapt for bridging real-world and rendered domains, and scene-specific joint estimation with DINOv2 and FiT3D features to recover motion for characters with arbitrary joint structures (Yun et al., 11 Mar 2025).

The interaction-relative family is the one that introduces the phrase “Cross-Space In-Betweening” directly, combining DCT-based individual prediction, relative-space transformation, FiLM modulation, periodic interaction modeling, and a motion refiner (Zhang et al., 30 Sep 2025).

5. Long-horizon interaction as the canonical explicit formulation

Among the cited works, the most canonical explicit definition is given in “Motion In-Betweening for Densely Interacting Characters” (Zhang et al., 30 Sep 2025). The problem setting is two characters engaged in actions such as boxing or dancing across multiple keyposes. The authors identify two main difficulties: the increased constraints sharply reduce the solution space, and interaction quality degrades over time unless the synthesis remains in a “stable region of the solution space” (Zhang et al., 30 Sep 2025).

The model decomposes synthesis into two stages. First, each character predicts inbetween motion relative to its own keyposes. Second, the predicted motion is transformed into the coordinate space of the other character, creating a relative representation used for conditioning (Zhang et al., 30 Sep 2025). Because direct conditioning across these spaces is difficult, the method applies FiLM based on a latent code from the transformed relative pose (Zhang et al., 30 Sep 2025).

To sustain long-term interaction quality, the model extracts Pairwise Joint Distance trajectories across the two characters and encodes them with a Periodic Autoencoder: $G=\{V,T\}$8 (Zhang et al., 30 Sep 2025). A discriminator then imposes adversarial loss on these periodic interaction patterns: $G=\{V,T\}$9 (Zhang et al., 30 Sep 2025). To control error accumulation, a Motion Refiner reconstructs and corrects predicted motion: $V$0 (Zhang et al., 30 Sep 2025).

The reported evaluation emphasizes reconstruction quality, interaction quality, long-term robustness, and user preference. The paper states lower L2 error than baselines on long 50-frame segments, improved FID and NPSS when the interaction GAN and motion refiner are present, and a user study with 50 participants in which outputs were rated closer to ground truth than baselines (Zhang et al., 30 Sep 2025).

This formulation is significant because it makes “space” concrete in three distinct senses: individual keypose space, cross-character relative space, and the latent modulation space used to bridge them.

6. Control, ambiguity reduction, and quality preservation

A recurring justification for cross-space methods is that the native input space is too ambiguous for reliable interpolation. The details differ by domain.

In cartoon line inbetweening, image-based interpolation damages sparse line structures through blur and broken lines, especially under large motion (Siyao et al., 2023). Geometrization reduces this ambiguity by restricting matching to meaningful endpoints and connectivity, and visibility prediction handles occlusion explicitly (Siyao et al., 2023). The graph fusion rule is

$V$1

(Siyao et al., 2023). The same paper reports significant improvement in Chamfer Distance over raster-based video frame interpolation baselines, including a margin of $V$2 on the test set with the largest motion, as well as a 36-person user study in which AnimeInbet is preferred in over 92% of cases and almost 100% for large motions (Siyao et al., 2023).

In controllable image inbetweening, StructInbet reduces ambiguity in “pixel trajectories” by introducing skeletons and mixed trajectory maps as explicit structural guidance (Pan et al., 15 Jul 2025). The reported metrics are FID, LPIPS, PSNR, and human ratings for image quality, character consistency, and motion alignment (Pan et al., 15 Jul 2025). On the reported table, StructInbet has the best Motion Alignment score of 3.86 and the lowest LPIPS at 0.54, while maintaining comparable IQ and CC to baselines (Pan et al., 15 Jul 2025). The paper attributes this to explicit structure and bidirectional reference attention (Pan et al., 15 Jul 2025).

In motion diffusion, ambiguity is often induced by sparse keyframes. CondMDI addresses this through direct masked training over arbitrary frame-joint subsets (Cohan et al., 2024). The INR-based diffusion model addresses it by mapping sparse and ambiguous keyframes into the INR latent manifold and then balancing geometric error against manifold error through IMG (Fan et al., 12 May 2026). The latter paper reports that its model significantly improves motion generation quality in scenarios with few keyframes while ensuring both keyframe accuracy and diversity of in-between motions (Fan et al., 12 May 2026).

A plausible implication is that cross-space formulations are especially useful when the observation space is underconstrained: sparse line art, sparse keyframes, multi-character interactions, or arbitrary-character renders all produce failure modes that direct interpolation handles poorly.

7. Relation to adjacent ideas and current directions

Cross-Space In-Betweening overlaps with, but is not identical to, several neighboring research ideas. It is related to structural guidance, because explicit structure often defines the auxiliary space in which conditioning occurs (Pan et al., 15 Jul 2025). It is related to diffusion-based keyframing, because diffusion models are particularly well suited to conditioning on sparse or heterogeneous constraints (Cohan et al., 2024, Fan et al., 12 May 2026). It is also related to domain bridging, as in AnyMoLe’s use of ICAdapt to fine-tune a video diffusion model at inference using a few seconds of rendered context, while freezing the temporal module and frame-rate embedding (Yun et al., 11 Mar 2025).

Another nearby direction is real-time space-time control. A data-driven framework based on a Dynamic Conditional Mixture-of-Experts network conditions next-frame prediction on phase features, style embedding, and time-to-arrive embedding, while a trajectory gallery provides explicit path and duration control (Chu et al., 2024). The next-pose prediction is

$V$3

with $V$4 experts (Chu et al., 2024). The paper explicitly describes its setting as supporting “cross-space in-betweening” in the sense of bridging poses that are far apart in space, action or style, and duration constraints (Chu et al., 2024). This is a broader and more control-oriented use of the term than the two-character relative-space formulation of (Zhang et al., 30 Sep 2025).

The literature also includes earlier cross-domain correspondence formulations. “Deep Sketch-guided Cartoon Video Inbetweening” estimates dense correspondence between sketches and cartoon keyframes using a transformer-augmented sketch representation, adapted PWC-Net flow estimation, occlusion via consistency checking, a blending module, and a temporal processing network (Li et al., 2020). Although it does not use the same label, it is structurally cross-space: sketch and cartoon frames are distinct domains, and interpolation depends on estimating correspondence between them (Li et al., 2020).

Across these works, no single consensus definition yet governs the term. The strict definition is the one given for interacting-character motion across different conditioning representation spaces (Zhang et al., 30 Sep 2025). The broader literature uses the phrase more loosely for interpolation across geometric, structural, spatio-temporal, semantic, or domain-shifted spaces (Siyao et al., 2023, Cohan et al., 2024, Chu et al., 2024, Yun et al., 11 Mar 2025). This suggests that Cross-Space In-Betweening is evolving into an umbrella concept for interpolation under heterogeneous constraints, provided that the method explicitly models the mappings between those spaces rather than collapsing all inputs into a single undifferentiated representation.