GVC1D: One-Dimensional Video Compression

• GVC1D is a generative video codec that replaces dense 2D latent grids with a compact 1D latent space to reduce both spatial and temporal redundancy.
• It employs a hybrid local-global transformer architecture with an autoregressive entropy model to optimize the rate–distortion trade-off.
• Experimental results show significant BD-Rate reductions and improved temporal consistency compared to traditional 2D latent video codecs.

Searching arXiv for GVC1D and cited related works to ground the article with current paper metadata and related-paper IDs. GVC1D, short for Generative Video Compression with One-Dimensional Latent Representation, is a generative neural video codec that replaces the dense 2D latent grids used by recent generative video codecs with a very small set of continuous 1D latent tokens and a 1D long-term memory. The method is designed around a representational claim: the main bottleneck in low-bitrate perceptual video compression is not only model capacity or loss type, but the rigid 2D latent structure itself. By removing fixed spatial correspondence, GVC1D encodes video in a compact semantic 1D space intended to reduce both intra-frame redundancy and long-term temporal redundancy while preserving perceptual quality (Zheng et al., 16 Mar 2026).

1. Conceptual basis and problem setting

Conventional codecs and PSNR-oriented neural video codecs optimize generic distortion measures such as MSE and PSNR. At low bitrates, the source paper states that they severely blur details because there are insufficient bits to preserve high-frequency content and no generative prior to hallucinate realistic details. Recent generative video codecs improve perceptual quality by encoding frames into dense 2D latent grids and using high-capacity generative decoders, including VQ-VAE- and diffusion-based designs. GVC1D departs from that paradigm by altering the bottleneck representation itself rather than only changing the decoder or loss formulation (Zheng et al., 16 Mar 2026).

The motivation is organized around two limitations of 2D latent grids. Spatially, a rectangular grid treats all patches equally: smooth sky and detailed text each receive the same token budget, leaving substantial intra-frame redundancy. Temporally, a 2D grid preserves local spatial structure but is less natural for compact, semantic long-term memory, because aggregating common content across frames requires operating over many spatial locations and large states. The paper therefore frames GVC1D as a representational alternative in which latent variables are no longer tied to fixed patches, and long-term context is stored directly in a compact 1D token space (Zheng et al., 16 Mar 2026).

The optimization target remains a standard rate–distortion trade-off,

$L = R + \lambda D,$

where $R$ is the bitrate, $D$ is a distortion or perceptual loss, and $\lambda$ controls the rate–distortion balance. In the video training stage, the paper uses

$$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$

with $T$ frames and $L_{\text{stage1},t}$ defined by the perceptual-adversarial reconstruction loss used in pretraining (Zheng et al., 16 Mar 2026).

2. One-dimensional latent representation

The defining operation in GVC1D is the conversion of each frame $x_t$ into a small sequence of continuous 1D latent tokens

$y_t \in \mathbb{R}^{D \times (N\cdot 32)},$

rather than a dense $H' \times W'$ latent grid. These tokens are described as global semantic “slots” rather than patch-aligned variables. They are conditioned on both short-term and long-term context, and they do not preserve rigid 2D spatial correspondence (Zheng et al., 16 Mar 2026).

The frame is first patch-embedded with effective patch size $R$0, yielding

$R$1

where each column of $R$2 is a $R$3-dimensional patch token and $R$4, $R$5. A fixed pool of learnable 1D latent tokens,

$R$6

acts as the semantic bottleneck. Long-term context $R$7 and short-term context $R$8 are concatenated into

$R$9

and the encoded 1D representation is produced by

$D$0

The paper argues that this representation reduces spatial redundancy because 1D tokens can adaptively attend to semantic regions such as objects and can naturally support token reduction. A fixed small number of tokens can therefore represent large frames without assigning separate latent variables to texture-less or otherwise simple regions. This suggests a semantic rather than geometric bottleneck: tokens are meant to aggregate information across multiple patches rather than merely summarize one patch each (Zheng et al., 16 Mar 2026).

The paper’s visualization analysis, based on attention outflow and FlowCut, supports that interpretation. Individual 1D tokens are reported to attend consistently to semantic regions such as “horse’s left foreleg” or “grassland” across multiple frames, even under large motion, and to reallocate attention when a new object appears. A plausible implication is that token identity in GVC1D is content-centric rather than coordinate-centric (Zheng et al., 16 Mar 2026).

3. Encoder, decoder, and entropy model

The per-frame pipeline has four components: an analysis transform or encoder, an entropy model, a synthesis transform or decoder, and a context model. The encoder is a ViT-style hierarchy composed of local and global transformers,

$D$1

Within each local window, image tokens $D$2, short-term context $D$3, learnable latent tokens $D$4, and long-term context $D$5 are concatenated and processed jointly. After every $D$6 local layers, a global transformer recombines all windows and applies full attention, with 2D RoPE used for the 2D tokens (Zheng et al., 16 Mar 2026).

This hybrid local-global design is meant to keep cost low while allowing the learnable 1D tokens to absorb semantic information from surrounding patch tokens and temporal context. The output remains conceptually a 1D token sequence, although the implementation groups tokens by windows during processing (Zheng et al., 16 Mar 2026).

After encoding, the latent tokens are quantized as $D$7 and modeled by an autoregressive transformer. For token sequence $D$8, the entropy model factorizes the distribution as

$D$9

and the estimated bitrate is

$\lambda$0

These probabilities are used during training through cross-entropy and during deployment through arithmetic coding. The paper emphasizes that $\lambda$1 is small and that tokens from different windows can be processed in parallel, which keeps computational costs manageable (Zheng et al., 16 Mar 2026).

The decoder mirrors the encoder. It introduces learnable mask tokens

$\lambda$2

one per patch position, and reconstructs the frame according to

$\lambda$3

The mask tokens are described as symmetric to the encoder’s learnable 1D tokens: they pull information from decoded 1D latents and context to reconstruct dense per-patch features, which are then mapped back to RGB by a convolutional output head (Zheng et al., 16 Mar 2026).

4. Temporal modeling and the 1D memory

Temporal modeling in GVC1D combines sequential processing, short-term decoder features, and a dedicated 1D long-term memory. Short-term context is formed from decoder features $\lambda$4 of the previous frame and carries fine-grain details and local structure. Long-term context is stored in a fixed-size memory state defined entirely in the 1D latent space (Zheng et al., 16 Mar 2026).

The memory operates in two stages. In the update stage, decoded latent tokens $\lambda$5 are used to update the previous state:

$\lambda$6

In the readout stage, learnable query tokens attend to memory to extract long-term context,

$\lambda$7

with $\lambda$8 matched to the latent size $\lambda$9. Because the memory operates on few semantically rich tokens instead of dense 2D feature maps, the paper argues that full-transformer memory becomes inexpensive and semantically cleaner than alternatives based on large 2D states (Zheng et al., 16 Mar 2026).

This design is contrasted with prior long-term context methods such as DCVC-LCG, which the paper characterizes as operating on dense 2D feature maps and requiring complex cluster search while remaining constrained by memory capacity. GVC1D instead uses a simple transformer memory over compact semantic tokens. The ablation evidence reported in the paper is strong: removing 1D memory increases BD-Rate by more than 40% on UVG, and replacing 1D memory with 2D-based memory degrades BD-Rate by 16–17% relative to the 1D memory setting (Zheng et al., 16 Mar 2026).

The paper also reports improved temporal consistency using FloLPIPS, a flow-guided LPIPS variant. GVC1D consistently achieves lower FloLPIPS than the baselines on all datasets considered, and qualitative examples show railing structures and textures remaining more stable across frames while baselines show blur or temporal artifacts. This suggests that semantic 1D memory is not only rate-efficient but also useful for suppressing flicker in generative reconstruction (Zheng et al., 16 Mar 2026).

5. Training procedure and bit allocation

Training is staged. Stage 1 trains only the encoder and decoder, without the entropy model, on two-frame samples from OpenVid-HD at resolution $$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$0. The loss is

$$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$1

with coefficients $$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$2 and $$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$3. The paper states that this stage is strongly initialized from TA-TiTok 1D tokenizer weights (Zheng et al., 16 Mar 2026).

Stage 2 introduces the autoregressive entropy model and trains on Vimeo with up to 32 frames. The bitrate term $$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$4 comes from the entropy model, and $$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$5 is sampled using 8 log-spaced values in $$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$6 to form a variable-bitrate model. Stage 3 sequentially introduces global transformer layers and 1D memory, then finetunes on OpenVid-HD videos with resolutions from $$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$7 to $$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$8, using the same rate-plus-perceptual-plus-adversarial loss, a smaller learning rate, and partial cascaded finetuning to manage GPU memory (Zheng et al., 16 Mar 2026).

Bit allocation in GVC1D is determined by quantization and entropy modeling rather than by changing token count. The design uses a fixed number of 1D tokens per window and fixed windows per frame at a given resolution, so the token length per frame is effectively fixed. The paper therefore describes variable bitrate as achieved by adjusting the rate–distortion balance through $$L_{\text{stage2}} = \frac{1}{T}\sum_{t=1}^{T} \left( R_t + \lambda\, L_{\text{stage1},t} \right),$$9, not through variable-length token sequences. A non-autoregressive entropy-model ablation, based on DCVC-RT, is reported to perform significantly worse in BD-Rate than the autoregressive model (Zheng et al., 16 Mar 2026).

6. Experimental performance, ablations, and scope

The experimental protocol uses Vimeo and OpenVid-HD for training, with HEVC Class B, UVG, and MCL-JCV for testing. Evaluation is conducted in a low-delay setting with intra-period $T$0, 96-frame sequences, and RGB colorspace. Metrics include LPIPS, DISTS, PSNR, MS-SSIM, FID, KID on $T$1 patches, FloLPIPS, bits per pixel, and BD-Rate versus GLC-Video as anchor. Baselines include HM-16.25, VTM-17.0, ECM-5.0, DCVC-FM, DCVC-RT, and GLC-Video (Zheng et al., 16 Mar 2026).

The main quantitative result is that on HEVC Class B, GVC1D achieves BD-Rate reductions of $T$2 under LPIPS and $T$3 under DISTS relative to GLC-Video. On the same dataset it also reports $T$4 for PSNR and $T$5 for MS-SSIM. On UVG and MCL-JCV, the reported BD-Rate numbers versus GLC-Video are likewise strongly negative across LPIPS, DISTS, PSNR, and MS-SSIM. The paper further states that FID and KID curves are consistently lower, indicating reconstructions closer to the real distribution, and that qualitative comparisons show fewer generative artifacts than GLC-Video and sharper details than DCVC-FM and ECM-5.0 (Zheng et al., 16 Mar 2026).

A key ablation concerns token size. The source reports that 32×16 tokens provide the best BD-Rate trade-off, while smaller or larger configurations worsen BD-Rate, either by reducing representation capacity or by increasing bitrate without commensurate gain. Another ablation shows that removing the autoregressive entropy model degrades BD-Rate substantially, and that replacing 1D memory with 2D memory is inferior to the full GVC1D design (Zheng et al., 16 Mar 2026).

The paper also reports coding time at 1080p on A100 in fp16: DCVC-FM uses 0.183 s for encoding and 0.190 s for decoding, GLC-Video uses 0.290 s and 0.183 s, and GVC1D uses 0.262 s and 0.207 s. The reported interpretation is that GVC1D is similar in speed to GLC-Video and slightly slower than DCVC-FM, while benefiting from operation in a compact 1D space, few tokens, windowed attention, and FlashAttention2 (Zheng et al., 16 Mar 2026).

The method is explicitly framed as a low-bitrate lossy codec. The paper states that with only 32 1D tokens per frame region, capacity is limited, and that extending the approach to lossless or near-lossless video compression would require handling higher information content, for example through more tokens or a hybrid 1D+2D representation. It also identifies possible future directions including adaptive token counts per frame or region, stronger generative decoders such as diffusion while maintaining efficiency, and cross-modal context using 1D tokens as a unified representation for generation and compression (Zheng et al., 16 Mar 2026).