Group-based Displacement Network (GDN)

• Group-based Displacement Network (GDN) is a neural module that refines coarse vector sketch animations by predicting displacement fields for semantically grouped strokes.
• It employs Context-conditioned Feature Enhancement to integrate frame-aware positional encoding and cross-attentional image features, ensuring temporal consistency.
• GDN utilizes a dual-path design to decouple local deformations from global affine transformations, optimized zero-shot via text-guided diffusion without ground-truth animations.

Group-based Displacement Network (GDN) is the motion-refinement component introduced in GroupSketch, a method for text-guided animation of vector sketches containing multiple interacting objects. Within the GroupSketch framework, GDN operates after an initial coarse animation has been created from semantic grouping and user-defined keyframes, and refines that motion by predicting group-specific displacement fields guided by trajectory priors and text-to-video motion priors. Its purpose is to preserve stroke identity and object shape while producing temporally coherent animation that follows the text prompt and supports plausible multi-object interactions (Liang et al., 21 Aug 2025).

1. Task definition and problem setting

GroupSketch addresses text-guided animation of vector sketches represented with cubic Bézier strokes. A single frame is written as a set of control points

$$P = \{p_1, p_2, \dots, p_N\} \in \mathbb{R}^{N \times 2}, \quad p_n = (x, y),$$

and a video with $K$ frames is

$$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$

The initial input sketch is static, and users interactively specify semantic grouping and keyframes or motion trajectories for each group. The expected output is an animated sketch that preserves stroke identity and object shape, follows user-provided group trajectories and text-prompt semantics, and exhibits plausible multi-object interactions and smooth motion over time (Liang et al., 21 Aug 2025).

The animation is encoded as 2D displacements of control points,

$$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$

yielding animated control points

$p_n^{k}{}' = p_n^k + \Delta p_n^k.$

In this setting, GDN is not a generic animation backend but a task-specific group-based neural displacement field for multi-object vector sketches. It is motivated by four challenges identified in the paper: multi-object interactions, complex non-trivial motions, temporal consistency, and generalization and control. Existing approaches such as LiveSketch and FlipSketch are described as largely treating the sketch as a single entity; global transformations can therefore make objects “stick together,” as in a ball becoming glued to a player’s hand, while raster-based approaches can exhibit flickering, shape changes, and inconsistent motion (Liang et al., 21 Aug 2025).

2. Placement within the two-stage GroupSketch pipeline

GDN appears in the second stage of a two-stage pipeline consisting of Motion Initialization followed by Motion Refinement. In Motion Initialization, the user segments strokes into semantic groups, each roughly corresponding to an object or body part, and the total number of groups is $M$. For each group $i$, the strokes and motion trajectory across $K$ frames form a group representation

$G_i \in \mathbb{R}^{N_i \times K \times 2},$

and the collection of groups is $\{G_i\}_{i=1}^M$. This grouping is described as the core structural prior enabling motion disentanglement (Liang et al., 21 Aug 2025).

Keyframes are specified by dragging a group on an interactive Canvas. The initial sketch serves as the first keyframe, and the last user-defined keyframe is the final frame. Between adjacent keyframes $K$0 and $K$1, intermediate positions are obtained by linear interpolation:

$K$2

For each group, the basic per-frame offset is $K$3, the group offsets across frames are $K$4, and the collection across all groups is $K$5. These offsets transform the initial sketch sequence into a coarse animated sequence consistent with user-specified group trajectories. The paper characterizes this stage as giving GDN a strong prior: rather than starting from a static sketch, it starts from a rough but semantically sensible animation (Liang et al., 21 Aug 2025).

Motion Refinement then applies GDN to the coarse group-wise sequences. The network uses $K$6 parallel subnetworks, one per group, and outputs refined displacements that are merged and rendered into a final $K$7-frame vector video using DiffVG. A frozen text-to-video diffusion model, ModelScope, evaluates the rendered video against the text prompt and provides SDS gradients for refinement (Liang et al., 21 Aug 2025).

3. Group-wise displacement formulation

GDN is described as the key innovation of GroupSketch: a group-based neural displacement field tailored for multi-object vector sketches. For each group $K$8, GDN takes the group geometry

$K$9

after Motion Initialization together with context features extracted by Context-conditioned Feature Enhancement. It predicts a local displacement $$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$0 and a global displacement $$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$1, and combines them as

$$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$2

The refined animated sketch is then

$$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$3

This formulation makes the group the fundamental unit of motion prediction rather than the entire sketch (Liang et al., 21 Aug 2025).

Each subnetwork begins with a Group Feature Extraction module that maps raw coordinates to a high-dimensional embedding:

$$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$4

with hidden dimension $$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$5 set to 128. The paper notes that this is analogous to the shared backbone in LiveSketch’s Neural Displacement Field, but applied per-group, which enables object-specific representation learning. The group-wise formulation is emphasized repeatedly: features are extracted per group, displacements are predicted per group, and affine transformations are group-specific. The immediate significance is avoidance of motion entanglement, such as a ball and player moving as a single rigid body (Liang et al., 21 Aug 2025).

4. Context-conditioned Feature Enhancement

Raw point coordinates are stated to be insufficient for coherent motion learning, particularly because each group is processed independently. GDN therefore incorporates Context-conditioned Feature Enhancement (CCFE), which injects temporal position information through Frame-aware Positional Encoding (FPE) and visual context through Motion Context Learning (MCL). The stated purpose of CCFE is to improve temporal consistency and restore inter-group and global scene context that would otherwise be weakened by fully separate group-wise modeling (Liang et al., 21 Aug 2025).

FPE attaches two kinds of positional encoding to every point. The first is point-level spatial encoding within a frame, using standard sinusoidal encoding over the 2D coordinates. The second is frame-level temporal encoding across frames, assigning the same temporal code to all points in the same frame index $$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$6 using multi-frequency sine and cosine functions over time. The augmented feature is

$$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$7

The paper states that high-frequency temporal components distinguish adjacent frames and low-frequency components capture long-term trends, helping GDN learn physically plausible, temporally coherent motion and avoid erratic frame-to-frame jumps (Liang et al., 21 Aug 2025).

MCL adds image-level context even though GDN itself operates in vector space. The current sketch sequence, with all groups combined, is rendered with DiffVG,

$$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$8

and dense visual features are extracted using DINOv2:

$$Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.$$9

These features are reshaped and processed with a temporal attention module to model long-range temporal dependencies. Fusion between vector and image domains then occurs through spatial cross-attention, with vector features $$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$0 as queries and temporally aware image features as keys and values. The implementation is described as six stacked “Vector-Image Spatio-Temporal Attention” layers. The resulting context-enriched features encode where a point is in the object, how the object relates to other objects and background, and how motion evolves across frames (Liang et al., 21 Aug 2025).

5. Local and global motion paths

The Group Displacement Field Prediction stage adopts a dual-path architecture inspired by the Neural Displacement Field in LiveSketch. The Grouping Local Path models fine-grained deformations at the control-point level, while the Grouping Global Path models object-level affine transformations per group per frame. The paper explicitly frames this as a decoupling of micro-deformation, such as bending arms, from macro motion, such as translating or rotating the entire object (Liang et al., 21 Aug 2025).

The local path is parameterized by an MLP $$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$1 and outputs per-point offsets for each frame:

$$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$2

This path captures local motion dynamics, including articulation and subtle jitter, and provides precise control-point adjustments. The global path, parameterized by $$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$3, predicts an affine transformation for each group at each frame,

$$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$4

composed of scaling, shearing, rotation, and translation. Applied in homogeneous coordinates, the corresponding global displacement is written as

$$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$5

Subtracting the original positions yields the global displacement field (Liang et al., 21 Aug 2025).

The affine path also includes user-controllable scaling factors for the transformation components: translation $$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$6, rotation $$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$7, scaling $$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$8, and shear $$\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,$$9. The default values reported in the implementation details are

$p_n^{k}{}' = p_n^k + \Delta p_n^k.$0

Because coarse translation is already provided in Motion Initialization, the translation scale is intentionally small, making global translation a refinement rather than a primary driver. The total displacement for group $p_n^{k}{}' = p_n^k + \Delta p_n^k.$1 remains

$p_n^{k}{}' = p_n^k + \Delta p_n^k.$2

applied to the coarse coordinates as $p_n^{k}{}' = p_n^k + \Delta p_n^k.$3 (Liang et al., 21 Aug 2025).

6. Optimization without ground-truth animation

A notable property of GDN is that it is not trained with ground-truth animations. Instead, it is optimized zero-shot using Score Distillation Sampling (SDS) with a frozen text-to-video diffusion model, ModelScope. The optimization loop renders the entire animated sketch, with all groups merged, through DiffVG; samples a time step $p_n^{k}{}' = p_n^k + \Delta p_n^k.$4 and noise $p_n^{k}{}' = p_n^k + \Delta p_n^k.$5; constructs noisy video frames

$p_n^{k}{}' = p_n^k + \Delta p_n^k.$6

and queries the diffusion model for the predicted noise $p_n^{k}{}' = p_n^k + \Delta p_n^k.$7 conditioned on text prompt $p_n^{k}{}' = p_n^k + \Delta p_n^k.$8. The resulting SDS gradients are used to refine GDN so that the animation becomes semantically aligned with the prompt and physically plausible (Liang et al., 21 Aug 2025).

This optimization regime places GDN between user control and generative prior. User-defined grouping and trajectories specify explicit object structure and coarse motion, while the frozen text-to-video prior contributes motion realism and semantic alignment. A plausible implication is that the network’s refinement capacity depends on both sources of supervision: the explicit structural prior from semantic grouping and the implicit motion prior from ModelScope. The paper reports that extensive experiments demonstrate the overall approach significantly outperforms existing methods in generating high-quality, temporally consistent animations for complex, multi-object sketches, thereby expanding the practical applications of sketch animation (Liang et al., 21 Aug 2025).

7. Nomenclature and acronym ambiguity

The acronym “GDN” is not unique across arXiv literature. In GroupSketch, GDN denotes Group-based Displacement Network, a component for group-wise motion refinement in vector sketch animation (Liang et al., 21 Aug 2025). In contrast, the later paper "FG$p_n^{k}{}' = p_n^k + \Delta p_n^k.$9-GDN: Enhancing Long-Context Gated Delta Networks with Doubly Fine-Grained Control" states explicitly that in that work GDN stands for Gated Delta Network, a delta-rule-based linear attention mechanism for long-context modeling, and that the model is not described there as “Group-based Displacement Network” (Sun et al., 21 Apr 2026).

The distinction is substantive rather than terminological. Group-based Displacement Network is defined around vector control points, semantic grouping, displacement fields, CCFE, and zero-shot optimization against text-to-video priors. Gated Delta Network is defined around an associative-memory state update in linear attention. This suggests that “GDN” should be treated as a domain-dependent acronym whose meaning must be inferred from the surrounding research area rather than as a single architecture family spanning sketch animation and long-context sequence modeling (Liang et al., 21 Aug 2025, Sun et al., 21 Apr 2026).