---
title: Group-based Displacement Network (GDN)
url: https://www.emergentmind.com/topics/group-based-displacement-network-gdn
type: topic
---

# Group-based Displacement Network (GDN)

Group-based Displacement Network (GDN) is the motion-refinement component introduced in GroupSketch, a method for text-guided animation of vector sketches containing multiple interacting objects. Within the GroupSketch framework, GDN operates after an initial coarse animation has been created from semantic grouping and user-defined keyframes, and refines that motion by predicting group-specific displacement fields guided by trajectory priors and text-to-video motion priors. Its purpose is to preserve stroke identity and object shape while producing temporally coherent animation that follows the text prompt and supports plausible multi-object interactions [2508.15535].

## 1. Task definition and problem setting

GroupSketch addresses text-guided animation of vector sketches represented with cubic Bézier strokes. A single frame is written as a set of control points
$$
P = \{p_1, p_2, \dots, p_N\} \in \mathbb{R}^{N \times 2}, \quad p_n = (x, y),
$$
and a video with $K$ frames is
$$
Z = \{P^k\}_{k=1}^K \in \mathbb{R}^{N \times K \times 2}.
$$
The initial input sketch is static, and users interactively specify semantic grouping and keyframes or motion trajectories for each group. The expected output is an animated sketch that preserves stroke identity and object shape, follows user-provided group trajectories and text-prompt semantics, and exhibits plausible multi-object interactions and smooth motion over time [2508.15535].

The animation is encoded as 2D displacements of control points,
$$
\Delta Z = \left\{ \Delta p_n^k \right\}_{n=1..N}^{k=1..K}, \quad \Delta p_n^k \in \mathbb{R}^2,
$$
yielding animated control points
$$
p_n^{k}{}' = p_n^k + \Delta p_n^k.
$$
In this setting, GDN is not a generic animation backend but a task-specific group-based neural displacement field for multi-object vector sketches. It is motivated by four challenges identified in the paper: multi-object interactions, complex non-trivial motions, temporal consistency, and generalization and control. Existing approaches such as LiveSketch and FlipSketch are described as largely treating the sketch as a single entity; global transformations can therefore make objects “stick together,” as in a ball becoming glued to a player’s hand, while raster-based approaches can exhibit flickering, shape changes, and inconsistent motion [2508.15535].

## 2. Placement within the two-stage GroupSketch pipeline

GDN appears in the second stage of a two-stage pipeline consisting of Motion Initialization followed by Motion Refinement. In Motion Initialization, the user segments strokes into semantic groups, each roughly corresponding to an object or body part, and the total number of groups is $M$. For each group $i$, the strokes and motion trajectory across $K$ frames form a group representation
$$
G_i \in \mathbb{R}^{N_i \times K \times 2},
$$
and the collection of groups is $\{G_i\}_{i=1}^M$. This grouping is described as the core structural prior enabling motion disentanglement [2508.15535].

Keyframes are specified by dragging a group on an interactive Canvas. The initial sketch serves as the first keyframe, and the last user-defined keyframe is the final frame. Between adjacent keyframes $t_1$ and $t_2$, intermediate positions are obtained by linear interpolation:
$$
e_{t_i} = (1 - t) \cdot e_{t_1} + t \cdot e_{t_2}, \quad
t = \frac{t_i - t_1}{t_2 - t_1} \in [0,1].
$$
For each group, the basic per-frame offset is $o_k = \{\Delta x, \Delta y\} \in \mathbb{R}^2$, the group offsets across frames are $O_i = \{o_k\}_{k=1}^K$, and the collection across all groups is $L = \{O_i\}_{i=1}^M$. These offsets transform the initial sketch sequence into a coarse animated sequence consistent with user-specified group trajectories. The paper characterizes this stage as giving GDN a strong prior: rather than starting from a static sketch, it starts from a rough but semantically sensible animation [2508.15535].

Motion Refinement then applies GDN to the coarse group-wise sequences. The network uses $M$ parallel subnetworks, one per group, and outputs refined displacements that are merged and rendered into a final $K$-frame vector video using DiffVG. A frozen text-to-video diffusion model, ModelScope, evaluates the rendered video against the text prompt and provides SDS gradients for refinement [2508.15535].

## 3. Group-wise displacement formulation

GDN is described as the key innovation of GroupSketch: a group-based neural displacement field tailored for multi-object vector sketches. For each group $i$, GDN takes the group geometry
$$
G_i \in \mathbb{R}^{N_i \times K \times 2}
$$
after Motion Initialization together with context features extracted by Context-conditioned Feature Enhancement. It predicts a local displacement $\Delta \hat{Z}_l^i$ and a global displacement $\Delta \hat{Z}_g^i$, and combines them as
$$
\Delta G_i = \Delta \hat{Z}_l^i + \Delta \hat{Z}_g^i.
$$
The refined animated sketch is then
$$
G_i' = G_i + \Delta G_i \quad \forall i.
$$
This formulation makes the group the fundamental unit of motion prediction rather than the entire sketch [2508.15535].

Each subnetwork begins with a Group Feature Extraction module that maps raw coordinates to a high-dimensional embedding:
$$
G_i \xrightarrow{\text{MLP}} F^i_{\text{pos}} \in \mathbb{R}^{N_i \times K \times d},
$$
with hidden dimension $d$ set to 128. The paper notes that this is analogous to the shared backbone in LiveSketch’s Neural Displacement Field, but applied per-group, which enables object-specific representation learning. The group-wise formulation is emphasized repeatedly: features are extracted per group, displacements are predicted per group, and affine transformations are group-specific. The immediate significance is avoidance of motion entanglement, such as a ball and player moving as a single rigid body [2508.15535].

## 4. Context-conditioned Feature Enhancement

Raw point coordinates are stated to be insufficient for coherent motion learning, particularly because each group is processed independently. GDN therefore incorporates Context-conditioned Feature Enhancement (CCFE), which injects temporal position information through Frame-aware Positional Encoding (FPE) and visual context through Motion Context Learning (MCL). The stated purpose of CCFE is to improve temporal consistency and restore inter-group and global scene context that would otherwise be weakened by fully separate group-wise modeling [2508.15535].

FPE attaches two kinds of positional encoding to every point. The first is point-level spatial encoding within a frame, using standard sinusoidal encoding over the 2D coordinates. The second is frame-level temporal encoding across frames, assigning the same temporal code to all points in the same frame index $k$ using multi-frequency sine and cosine functions over time. The augmented feature is
$$
F^i_{\text{FPE}} = F^i_{\text{pos}} + \mathrm{PE}_{\text{space}} + \mathrm{PE}_{\text{time}}.
$$
The paper states that high-frequency temporal components distinguish adjacent frames and low-frequency components capture long-term trends, helping GDN learn physically plausible, temporally coherent motion and avoid erratic frame-to-frame jumps [2508.15535].

MCL adds image-level context even though GDN itself operates in vector space. The current sketch sequence, with all groups combined, is rendered with DiffVG,
$$
\mathcal{R}(\{G_i\}_{i=1}^M) \Rightarrow \text{raster frames},
$$
and dense visual features are extracted using DINOv2:
$$
F_{\text{bg}} = \text{DINOv2}(\mathcal{R}(\{G_i\}_{i=1}^M)).
$$
These features are reshaped and processed with a temporal attention module to model long-range temporal dependencies. Fusion between vector and image domains then occurs through spatial cross-attention, with vector features $F^i_{\text{FPE}}$ as queries and temporally aware image features as keys and values. The implementation is described as six stacked “Vector-Image Spatio-Temporal Attention” layers. The resulting context-enriched features encode where a point is in the object, how the object relates to other objects and background, and how motion evolves across frames [2508.15535].

## 5. Local and global motion paths

The Group Displacement Field Prediction stage adopts a dual-path architecture inspired by the Neural Displacement Field in LiveSketch. The Grouping Local Path models fine-grained deformations at the control-point level, while the Grouping Global Path models object-level affine transformations per group per frame. The paper explicitly frames this as a decoupling of micro-deformation, such as bending arms, from macro motion, such as translating or rotating the entire object [2508.15535].

The local path is parameterized by an MLP $\hat{M}_l$ and outputs per-point offsets for each frame:
$$
\Delta \hat{Z}_l^i = \hat{M}_l(F^i_{\text{context}}) \in \mathbb{R}^{N_i \times K \times 2}.
$$
This path captures local motion dynamics, including articulation and subtle jitter, and provides precise control-point adjustments. The global path, parameterized by $\hat{M}_g$, predicts an affine transformation for each group at each frame,
$$
\mathbf{T}_i^k \in \mathbb{R}^{3 \times 3},
$$
composed of scaling, shearing, rotation, and translation. Applied in homogeneous coordinates, the corresponding global displacement is written as
$$
\Delta \hat{Z}_{g}^k = \mathbf{T}^k \cdot G_i^k - G_i^k.
$$
Subtracting the original positions yields the global displacement field [2508.15535].

The affine path also includes user-controllable scaling factors for the transformation components: translation $\lambda_t$, rotation $\lambda_r$, scaling $\lambda_s$, and shear $\lambda_{sh}$. The default values reported in the implementation details are
$$
\lambda_t = 10^{-2}, \quad
\lambda_r = 10^{-2}, \quad
\lambda_s = 5 \times 10^{-2}, \quad
\lambda_{sh} = 10^{-1}.
$$
Because coarse translation is already provided in Motion Initialization, the translation scale is intentionally small, making global translation a refinement rather than a primary driver. The total displacement for group $i$ remains
$$
\Delta G_i = \Delta \hat{Z}_l^i + \Delta \hat{Z}_g^i,
$$
applied to the coarse coordinates as $G_i' = G_i + \Delta G_i$ [2508.15535].

## 6. Optimization without ground-truth animation

A notable property of GDN is that it is not trained with ground-truth animations. Instead, it is optimized zero-shot using Score Distillation Sampling (SDS) with a frozen text-to-video diffusion model, ModelScope. The optimization loop renders the entire animated sketch, with all groups merged, through DiffVG; samples a time step $t$ and noise $\epsilon$; constructs noisy video frames
$$
\boldsymbol{x}_t = \alpha_t \boldsymbol{x} + \sigma_t \epsilon;
$$
and queries the diffusion model for the predicted noise $\epsilon_\theta(\boldsymbol{x}_t; y, t)$ conditioned on text prompt $y$. The resulting SDS gradients are used to refine GDN so that the animation becomes semantically aligned with the prompt and physically plausible [2508.15535].

This optimization regime places GDN between user control and generative prior. User-defined grouping and trajectories specify explicit object structure and coarse motion, while the frozen text-to-video prior contributes motion realism and semantic alignment. A plausible implication is that the network’s refinement capacity depends on both sources of supervision: the explicit structural prior from semantic grouping and the implicit motion prior from ModelScope. The paper reports that extensive experiments demonstrate the overall approach significantly outperforms existing methods in generating high-quality, temporally consistent animations for complex, multi-object sketches, thereby expanding the practical applications of sketch animation [2508.15535].

## 7. Nomenclature and acronym ambiguity

The acronym “GDN” is not unique across arXiv literature. In GroupSketch, GDN denotes **Group-based Displacement Network**, a component for group-wise motion refinement in vector sketch animation [2508.15535]. In contrast, the later paper "FG$^2$-GDN: Enhancing Long-Context Gated Delta Networks with Doubly Fine-Grained Control" states explicitly that in that work **GDN** stands for **Gated Delta Network**, a delta-rule-based linear attention mechanism for long-context modeling, and that the model is not described there as “Group-based Displacement Network” [2604.19021].

The distinction is substantive rather than terminological. Group-based Displacement Network is defined around vector control points, semantic grouping, displacement fields, CCFE, and zero-shot optimization against text-to-video priors. Gated Delta Network is defined around an associative-memory state update in linear attention. This suggests that “GDN” should be treated as a domain-dependent acronym whose meaning must be inferred from the surrounding research area rather than as a single architecture family spanning sketch animation and long-context sequence modeling [2508.15535; 2604.19021].

Source: https://www.emergentmind.com/topics/group-based-displacement-network-gdn