Feature-aligned Motion Transformation (FMT)

• FMT is a dynamic point cloud compression framework that uses implicit spatiotemporal feature alignment instead of explicit motion vectors.
• It employs fixed KNN graphs and lightweight MLPs to synthesize motion-aware context features for conditional latent coding.
• The approach achieves notable BD-Rate reductions and faster encoding/decoding compared to methods like D-DPCC and AdaDPCC.

Searching arXiv for the cited paper and related baselines to ground the article in current literature. Searching (Deng et al., 18 Sep 2025) "Feature-aligned Motion Transformation for Efficient Dynamic Point Cloud Compression" Feature-aligned Motion Transformation (FMT) is a framework for dynamic point cloud compression that replaces explicit motion vectors with spatiotemporal feature alignment in latent space, using aligned features as temporal context within a latent-space conditional encoding framework (Deng et al., 18 Sep 2025). In the formulation used for dynamic point cloud sequences, a frame $t$ contains geometry $P_t \in \mathbb{R}^{N_t\times3}$ and per-point attributes $A_t \in \mathbb{R}^{N_t\times d_a}$, and compression seeks to encode $\{P_t, A_t\}_t$ efficiently by removing spatial redundancy within a frame and temporal redundancy across frames under a rate–distortion trade-off. FMT is motivated by the fact that point clouds are unordered, sparse, and non-uniform, so explicit motion estimation is difficult because local variations in density and scale, inconsistent orientation, and the lack of regular neighborhoods lead to unstable motion fields and frequent neighborhood reassignments. The framework therefore uses a fixed KNN correspondence graph and lightweight MLPs to synthesize motion-aware context features for conditional latent coding, and combines this with a random access reference strategy that enables bidirectional motion referencing, layered encoding, and frame-level parallel compression (Deng et al., 18 Sep 2025).

1. Problem setting and conceptual basis

Dynamic point cloud compression depends heavily on motion estimation and compensation, but the geometric irregularity of point sets makes conventional scene-flow-style estimation unstable (Deng et al., 18 Sep 2025). In explicit motion vector (MV)-based dynamic point cloud compression, such as D-DPCC and AdaDPCC, the pipeline estimates scene flow between frames, compensates the reference, and encodes residuals. The reported limitations are threefold: irregular structure produces noisy point-to-point correspondence, subtle localized deformations are difficult to capture with sparse scene flow and often require aggregation across scales, and motion vectors themselves must be coded, increasing bitstream overhead and enabling error propagation in sequential pipelines (Deng et al., 18 Sep 2025).

FMT addresses these limitations by shifting motion handling from explicit vector transmission to implicit alignment in feature space. Instead of estimating flow $f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$, the framework models

$c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$

where $y_r$ denotes a latent feature map from a reference frame, $C_t$ and $C_r$ are current and reference coordinates, and $\theta$ denotes the parameters of the alignment MLPs (Deng et al., 18 Sep 2025). The central claim is not that motion disappears, but that it is represented implicitly through aligned context features rather than through an explicitly transmitted MV field. A common misconception is therefore to treat FMT as a variant of scene-flow coding; more precisely, it is a conditional latent modeling approach in which temporal variation is embedded into aligned latent features.

2. Codec architecture and data flow

The codec is a multi-stage pipeline built around sparse convolutional feature extraction, bidirectional feature alignment, contextual latent coding, mixed geometry coding, and progressive reconstruction (Deng et al., 18 Sep 2025). The input is represented as $P_t \in \mathbb{R}^{N_t\times3}$0, with coordinates $P_t \in \mathbb{R}^{N_t\times3}$1 and features $P_t \in \mathbb{R}^{N_t\times3}$2. A feature encoder progressively downsamples the frame using sparse convolutions implemented with MinkowskiEngine, producing

$P_t \in \mathbb{R}^{N_t\times3}$3

Architectural details for the downsampling and upsampling blocks, together with an Inception-Residual Network (IRN) used for feature aggregation, are specified in the framework diagrams summarized in the source description (Deng et al., 18 Sep 2025).

At the alignment scale, a bidirectional FMT module aligns reference features from frames $P_t \in \mathbb{R}^{N_t\times3}$4 and $P_t \in \mathbb{R}^{N_t\times3}$5 to the current downsampled coordinates $P_t \in \mathbb{R}^{N_t\times3}$6, yielding motion-aware context features $P_t \in \mathbb{R}^{N_t\times3}$7. These aligned features are then injected into a contextual encoder that fuses current latent information with temporal context: $P_t \in \mathbb{R}^{N_t\times3}$8 This produces a higher-level latent $P_t \in \mathbb{R}^{N_t\times3}$9 (Deng et al., 18 Sep 2025).

Compression is split across coordinate and feature streams. The high-level feature tensor $A_t \in \mathbb{R}^{N_t\times d_a}$0 is compressed lossily with a latent-space conditional entropy model. The coordinate tensor $A_t \in \mathbb{R}^{N_t\times d_a}$1 is compressed losslessly via a G-PCC octree, while $A_t \in \mathbb{R}^{N_t\times d_a}$2 is losslessly coded by a learned geometry codec in order to preserve alignment-critical structure at the decoder (Deng et al., 18 Sep 2025). On the decoder side, the system recovers $A_t \in \mathbb{R}^{N_t\times d_a}$3, $A_t \in \mathbb{R}^{N_t\times d_a}$4, and $A_t \in \mathbb{R}^{N_t\times d_a}$5, re-runs bidirectional FMT to reconstruct $A_t \in \mathbb{R}^{N_t\times d_a}$6, and uses a contextual decoder

$A_t \in \mathbb{R}^{N_t\times d_a}$7

to synthesize $A_t \in \mathbb{R}^{N_t\times d_a}$8. Progressive upsampling then yields $A_t \in \mathbb{R}^{N_t\times d_a}$9, and decoded frames are inserted into forward and backward reference buffers according to the random access schedule (Deng et al., 18 Sep 2025).

This architecture makes aligned features serve a dual role. They are used both as synthesis context for latent reconstruction and as temporal prior information for entropy modeling. This suggests that the gains attributed to FMT are not confined to motion compensation in the narrow sense, but extend to uncertainty reduction in latent coding.

3. Alignment mechanism in latent space

The core FMT mechanism constructs a fixed KNN graph from current coordinates to reference coordinates, gathers local reference features, applies soft neighbor selection, and synthesizes aligned features with MLPs (Deng et al., 18 Sep 2025). Let current coordinates be $\{P_t, A_t\}_t$0, reference coordinates $\{P_t, A_t\}_t$1, and reference features $\{P_t, A_t\}_t$2, where $\{P_t, A_t\}_t$3. For each current point $\{P_t, A_t\}_t$4, the method finds $\{P_t, A_t\}_t$5 nearest neighbors in the reference set: $\{P_t, A_t\}_t$6

Neighbor features and relative offsets are then gathered: $\{P_t, A_t\}_t$7 After concatenation,

$\{P_t, A_t\}_t$8

a soft selection mask is computed: $\{P_t, A_t\}_t$9 The mask modulates neighbor features,

$f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$0

which are then concatenated with the anchor coordinate,

$f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$1

and mapped to the aligned feature

$f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$2

Collecting over all anchors yields

$f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$3

In bidirectional mode, contexts from $f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$4 and $f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$5 are fused, for example by concatenation followed by $f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$6, to form $f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$7 (Deng et al., 18 Sep 2025). The source description emphasizes three properties of this design: the KNN graph is fixed per anchor point during training and inference, soft neighbor selection $f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$8 modulates correspondence quality, and the decoder recomputes alignment from decoded coordinates and the same MLPs, so no motion vectors are transmitted.

The distinction from iterative adjacency refinement, such as the KNN-based 3DAWI in D-DPCC, is operationally important. FMT avoids iterative adjacency updates and instead anchors the graph at current coordinates. The reported interpretation is that this improves structural consistency during training and better tolerates local non-rigid motion and scale changes (Deng et al., 18 Sep 2025).

4. Conditional latent modeling, quantization, and distortion criteria

The high-level latent $f: c_j^{t-1} \rightarrow c_j^{t-1} + v_j$9 is modeled with a Laplace distribution whose parameters are predicted from fused hyper, autoregressive spatial, and temporal priors (Deng et al., 18 Sep 2025). The conditional likelihood is

$c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$0

with

$c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$1

Here, $c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$2 is the quantized hyper-latent, $c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$3 is the hyperprior decoder, $c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$4 is the autoregressive network over previously decoded symbols, $c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$5 is the temporal prior encoder using temporal context $c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$6, and $c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$7 is the prior fusion network (Deng et al., 18 Sep 2025).

Quantization is modeled by rounding at inference,

$c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$8

while training may use a uniform-noise approximation. The expected rate is the cross-entropy

$c_t = \mathrm{Align}(y_r, C_t, C_r; \theta),$9

and the end-to-end objective is summarized as

$y_r$0

Because hyper-latents $y_r$1 are coded, the total rate includes both main-latent and hyper-latent terms: $y_r$2

The distortion term follows PCGCv2 and combines reconstruction losses across the multi-stage decoder for geometry occupancy: $y_r$3 where $y_r$4 is ground-truth voxel occupancy and $y_r$5 is predicted occupancy at stage $y_r$6 (Deng et al., 18 Sep 2025). For evaluation, the framework reports MPEG CTC geometry metrics: $y_r$7 with $y_r$8 for 8iVFB, $y_r$9 defined by mean squared distance to nearest neighbors, and $C_t$0 defined by squared distances projected onto local tangent planes (Deng et al., 18 Sep 2025).

Within this model, aligned features are not merely auxiliary reconstruction signals. They explicitly condition both the contextual encoder/decoder and the entropy model’s temporal prior. The stated effect is a reduction in uncertainty in $C_t$1, and hence a reduction in bitrate.

5. Random access reference strategy and parallel scheduling

The random access (RA) reference strategy organizes frames into open Groups of Frames (GOFs) of length 16, uses five hierarchical temporal layers, and adopts a non-sequential encoding order to enable bidirectional referencing and frame-level parallelism (Deng et al., 18 Sep 2025). Only the first frame of the entire sequence is intra-coded; the first frame of each GOF references the last frame of the previous GOF to maintain continuity. The coding order within a GOF is

$C_t$2

Lower layers are coded first, higher-layer frames reference only lower layers, and frames within the same layer can be processed in parallel (Deng et al., 18 Sep 2025).

Reference management is buffer-based. Each frame maintains forward and backward buffers, and unidirectional or bidirectional prediction is chosen according to proximity and spatiotemporal relevance. The reported examples are that Frame 14 uses Frame 12 as a single forward reference, while Frame 10 uses Frames 8 and 12 bidirectionally (Deng et al., 18 Sep 2025). The dependency graph is acyclic across layers: if $C_t$3 denotes the layer index, an edge $C_t$4 exists only when $C_t$5. Non-sequential order is therefore a topological sort from lower to higher layers.

The encoder pseudo-procedure partitions the sequence into GOFs of 16, processes frames in the RA order, selects references $C_t$6 satisfying $C_t$7, computes $C_t$8, runs bidirectional FMT, forms $C_t$9, encodes $C_r$0 by octree, $C_r$1 by learned lossless coding, and $C_r$2 by the conditional entropy model, and stores decoded $C_r$3 into buffers for future frames (Deng et al., 18 Sep 2025). The decoder mirrors this schedule, with layerwise processing and optional parallelization within layers.

RA is also evaluated against low-delay coding. The reported result is that RA generally yields higher quality due to bidirectional references, while a few frames show minor dips when temporal gaps introduce less relevant features (Deng et al., 18 Sep 2025). A plausible implication is that FMT and RA are mutually reinforcing: the alignment module benefits from richer temporal anchors, while the hierarchical schedule depends on alignment that can tolerate non-sequential reference distances.

6. Empirical results, ablation, limitations, and integration

The training setup uses the Owlii dataset, consisting of four sequences and 2400 frames, quantized from 11-bit to 10-bit for efficiency, with Adam $C_r$4, a learning-rate scheduler with $C_r$5 decay every 15 epochs, and a two-stage schedule per rate: 5 epochs with $C_r$6 for reconstruction warm-up, followed by 45 epochs with $C_r$7; the batch size is 1 and the hardware is NVIDIA A100 (Deng et al., 18 Sep 2025). Evaluation is conducted on MPEG 8i Voxelized Full Bodies (8iVFB), comprising four sequences and 1200 frames at 30 fps, with bitrate measured in bpp and geometry quality measured by D1 and D2 PSNR (Deng et al., 18 Sep 2025).

The baselines are V-PCC (TM v13, QP $C_r$8), D-DPCC, PatchDPCC, AdaDPCC, and PCGCv2. The reported BD-Rate gains are as follows (Deng et al., 18 Sep 2025):

Baseline	D1 BD-Rate	D2 BD-Rate
V-PCC	−75.00%	−89.11%
PCGCv2	−45.50%	−60.83%
D-DPCC	−20.15%	−20.74%
AdaDPCC	−9.40%	−12.55%

The abstract-level summary states BD-Rate reductions of 20% relative to D-DPCC and 9.4% relative to AdaDPCC, consistent with the averages reported in Table 1 (Deng et al., 18 Sep 2025). Coding-time comparisons are likewise reported in seconds per frame:

Method	Encode	Decode
Ours	0.59	0.57
AdaDPCC	0.68	0.67
D-DPCC	1.28	1.16
PatchDPCC	4.91	1.19

The stated efficiency interpretation is that the method is 1.15× faster than AdaDPCC in encoding and 1.18× faster in decoding, and 8.32× faster than PatchDPCC in encoding (Deng et al., 18 Sep 2025). Ablation results attribute approximately 10.86% BD-Rate reduction to FMT alone relative to explicit 3DAWI motion estimation in D-DPCC, an additional approximately 7.41% gain to the hyperprior-context entropy model, and a further approximately 5.07% reduction to RA (Deng et al., 18 Sep 2025). Alignment effect visualizations show that feature differences shrink after FMT, which is interpreted as improved temporal consistency.

The stated limitations are also specific. Very fast motion or severe occlusions can cause fixed KNN correspondences to miss true associations when temporal gaps are large. Extremely sparse or noisy regions can suffer from limited neighbor support when $C_r$9 is small, whereas larger $\theta$0 increases computation and may introduce noise. In the RA schedule, some higher-layer frames may reference temporally distant frames; bidirectional prediction helps, but context relevance can still drop, causing occasional quality dips (Deng et al., 18 Sep 2025). The described future directions include adaptive neighbor selection with learned $\theta$1, deformable sampling kernels, multi-scale FMT cascades, joint geometry–attribute alignment, and parallel acceleration exploiting the RA hierarchy as in HEVC/VVC (Deng et al., 18 Sep 2025).

The integration pathway is explicitly hybrid rather than exclusive. Geometry coding uses G-PCC octree coding for $\theta$2, allowing interoperability with MPEG PCC standards, while the learned lossless codec for $\theta$3 is retained to preserve alignment structure for decoder-side FMT (Deng et al., 18 Sep 2025). The framework focuses on geometry, but the source description states that the FMT concept extends to attribute streams $\theta$4 by aligning attribute features similarly and conditioning an attribute entropy model. It also states that, in V-PCC-like systems, FMT could provide better temporal priors for projected textures (Deng et al., 18 Sep 2025). This suggests that FMT is best understood not as a narrowly geometry-specific motion substitute, but as a latent alignment primitive that can be coupled to multiple point cloud coding back ends.