Temporal-Aware Spatial Compression

• Temporal-aware spatial compression is a design principle that integrates temporal dynamics into spatial encoding to preserve motion coherence and data fidelity.
• It employs methods such as temporal selection before spatial encoding, spatial structure reuse, and explicit temporal consistency to enhance compression performance.
• This approach drives improvements across video synthesis, scientific simulations, LiDAR, and communication systems by achieving high compression ratios while maintaining essential temporal context.

Temporal-aware spatial compression denotes a family of compression strategies in which temporal structure is not treated as a secondary concern after spatial encoding, but instead governs what spatial data are encoded, how they are represented, and how fidelity is evaluated. Across video codecs, video-to-video synthesis, scientific simulation pipelines, LiDAR and event streams, dynamic 3D geometry, wireless CSI feedback, video tokenization, and streaming transformer caches, the recurring objective is to exploit spatial redundancy while preserving frame-to-frame coherence, motion-trajectory integrity, long-range temporal context, or physics-relevant evolution over time (Zsoldos, 18 May 2026, Zhuo et al., 2022, Cranganore et al., 10 Apr 2026, Feng et al., 2020, Tan et al., 2024, Park et al., 19 May 2026, Wang et al., 18 Mar 2026).

1. Conceptual scope and historical development

Early formulations were pragmatic. A practical “4D video” pipeline converted volumetric fields evolving over time into tiled 2D images and then compressed the resulting frame sequence with standard video codecs, thereby exploiting spatial redundancy within each field and temporal redundancy across successive fields (Robinson et al., 2016). In that work, each time step was quantized to 8-bit values, altitude slices were tiled into RGB channels, and the resulting Theora Ogg Vorbis q2 stream achieved about a 400:1 compression ratio relative to the original GRIB2 data for visualization-oriented use (Robinson et al., 2016).

Learned formulations made the coupling more explicit. “Learning Image and Video Compression through Spatial-Temporal Energy Compaction” generalized an image autoencoder to video by adding interpolation loops and by adapting the number of frames in one interpolation loop to motion characteristics, so that residual energy became temporally compact as well as spatially compact (Cheng et al., 2019). “Neural Video Compression using Spatio-Temporal Priors” then formalized a learned codec in which spatial priors, temporal priors, optical flow, and learned entropy models were jointly optimized for rate-distortion performance (Liu et al., 2019).

Subsequent work broadened the notion beyond conventional codec design. In contemporary usage, temporal-aware spatial compression encompasses temporal filtering before spatial neural compression in PDE simulations, reuse of spatial models across adjacent LiDAR scans, motion-driven prediction in asynchronous event streams, decoupled spatial and temporal tokenization for video models, and spatio-temporal cache compression for streaming transformers (Cranganore et al., 10 Apr 2026, Feng et al., 2020, Stumpp et al., 2024, Tan et al., 2024, Wang et al., 18 Mar 2026). This suggests that the topic is best understood as a cross-domain design principle rather than a single compression architecture.

2. Recurrent design principles

A first recurring principle is temporal selection before spatial encoding. ANTIC uses a Physics-aware Temporal Selector (PATS) to decide which PDE snapshots are salient enough to keep, and only those selected snapshots are then compressed spatially with neural fields (Cranganore et al., 10 Apr 2026). Fast-Vid2Vid retrains a generator on sparse temporal samples so that it synthesizes only keyframes, while intermediate frames are reconstructed by motion compensation (Zhuo et al., 2022). DynaTok allocates a token budget per frame according to temporal novelty, then subdivides each frame’s budget spatially (Park et al., 19 May 2026). In climate data compression, the same logic appears as a storage policy: keeping higher temporal frequency at reduced precision can preserve downstream spatiotemporal feature tracking better than exact but decimated outputs (Gong et al., 2024).

A second principle is reuse of spatial structure across time. Real-time LiDAR compression spatially encodes one key frame by iterative plane fitting, then reuses those planes to encode temporally adjacent frames after IMU-based alignment (Feng et al., 2020). Dynamic 3D mesh compression first expresses geometry in differential coordinates and then projects those differential trajectories onto a temporal covariance eigenspace, so temporal redundancy operates on a spatially meaningful representation (Arvanitis et al., 2021). STAC compresses evicted tokens from a streaming 3D reconstruction transformer into voxel-aligned spatial caches instead of discarding them, while preserving a temporally important working set (Wang et al., 18 Mar 2026).

A third principle is temporal consistency as an explicit objective rather than an incidental by-product. In compressed video super-resolution, SDATC adds a Spatial Degradation-Aware and Temporal Consistent diffusion design, with a Distortion Control Module, a Compression-Aware Prompt Module, a Spatio-Temporal Attention Module, and optical-flow-based alignment during denoising (An et al., 11 Feb 2025). In video quality evaluation, temporal stability is measured with ST-RRED, MOVIE Index, T-SSIM, and T-PSNR because frame-based metrics miss flicker, breathing, floating textures, and motion jitter (Zsoldos, 18 May 2026).

Several representative formulations make the coupling explicit. Fast-Vid2Vid spatially reduces the semantic input stream from $H \times W$ to $h \times w$, where

$h \times w = \frac{1}{(2^d)^2} H \times W,$

with $d=1$ in the reported setup (Zhuo et al., 2022). ANTIC updates neural-field parameters incrementally across selected snapshots,

$$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$

so spatial compression directly exploits temporal continuity (Cranganore et al., 10 Apr 2026). DynaTok assigns each frame a temporal budget

$B_t = w_t \cdot T \cdot N \cdot R,$

linking temporal novelty to downstream spatial token retention (Park et al., 19 May 2026).

3. Video compression, synthesis, and temporal quality assessment

In learned video compression, temporal-aware spatial compression usually appears as a fusion of motion modeling and conditional entropy modeling. Neural Video Compression using Spatio-Temporal Priors operates in an IPPP low-delay setting, uses spatial priors from downscaled low-resolution features, temporal priors from ConvLSTM states derived from previous reference frames and residuals, and optical flow for inter prediction (Liu et al., 2019). Multiscale Motion-Aware and Spatial-Temporal-Channel Contextual Coding Network extends this line by coupling a multiscale motion-aware module with a spatial-temporal-channel contextual entropy model, reporting average $10.15\%$ BD-rate savings against H.265/HEVC (HM-16.20) in PSNR and average $23.93\%$ BD-rate savings against H.266/VVC (VTM-13.2) in MS-SSIM (Wang et al., 2023).

In generative video systems, the temporal axis can be compressed before generation rather than after synthesis. Fast-Vid2Vid preserves the overall Vid2Vid generator design but reduces inference cost by feeding a low-resolution semantic stream and asking the network to synthesize only keyframes, followed by zero-parameter overlapped block motion compensation with enhanced predictive zonal search (Zhuo et al., 2022). The reported cost of motion compensation is only $0.0008146$ G MACs per video frame at $512 \times 512$, compared with $h \times w$0 G MACs for the generative part, and the method reaches around real-time performance as $h \times w$1 FPS while saving around $h \times w$2 computational cost on a single V100 GPU (Zhuo et al., 2022). This design illustrates that temporal-aware spatial compression is not limited to coding theory; it also governs inference scheduling in video generation.

The relation between compression strength and temporal quality has been examined directly. A systematic study across AV1, HEVC, VP9, and H.264 encoded 44 sequences at bitrate levels $h \times w$3, with

$h \times w$4

and found that temporal consistency degrades non-linearly with increasing compression (Zsoldos, 18 May 2026). Its central result is the “Predictability anomaly”: TI Group 3, containing medium-to-high irregular motion such as water or crowds, is harder to encode stably than TI Group 4, which has higher motion magnitude but more structured global motion (Zsoldos, 18 May 2026). The study reports average I-frame frequencies of $h \times w$5 for TI Group 2, $h \times w$6 for TI Group 3, $h \times w$7 for TI Group 4, and $h \times w$8 for TI Group 1, interpreting more frequent I-frame insertion as encoder fallback when inter-frame prediction becomes unreliable (Zsoldos, 18 May 2026). It also identifies a “VMAF paradox”: at and above $h \times w$9 kbps, TI Group 3 can obtain very high VMAF scores because frequent I-frames produce sharp spatial snapshots, while temporal metrics such as ST-RRED and MOVIE still indicate instability (Zsoldos, 18 May 2026).

A closely related restoration problem appears in compressed video super-resolution. SDATC addresses compressed low-quality videos by making diffusion conditioning degradation-aware and decoding temporally consistent through STAM and latent-space optical-flow alignment (An et al., 11 Feb 2025). The paper reports that SDATC obtains the best or second-best LPIPS, DISTS, FID, NIQE, MANIQA, and CLIP-IQA across tested settings while remaining competitive in PSNR and SSIM among generative methods (An et al., 11 Feb 2025). The underlying point is consistent with temporal-aware compression more broadly: improving per-frame realism is not sufficient if inter-frame smoothness is not explicitly constrained.

4. Scientific fields, geometric data, and asynchronous sensors

In large-scale scientific computing, temporal-aware spatial compression is often driven by storage and I/O limits rather than by perceptual quality. ANTIC is an in-situ, single-pass streaming pipeline in which PATS identifies informative snapshots from PDE simulations and a spatial neural compressor based on continual fine-tuning and neural fields encodes only the retained states (Cranganore et al., 10 Apr 2026). For 2D Kolmogorov flow, the paper reports temporal retention of about $h \times w = \frac{1}{(2^d)^2} H \times W,$0, spatial compression of $h \times w = \frac{1}{(2^d)^2} H \times W,$1 per snapshot with LoRA and $h \times w = \frac{1}{(2^d)^2} H \times W,$2 with full fine-tuning, and total compression up to $h \times w = \frac{1}{(2^d)^2} H \times W,$3 with ANTIC-LoRA and $h \times w = \frac{1}{(2^d)^2} H \times W,$4 with ANTIC-FT (Cranganore et al., 10 Apr 2026). For 3D binary black hole mergers, it reports temporal retention of about $h \times w = \frac{1}{(2^d)^2} H \times W,$5, per-snapshot spatial compression up to $h \times w = \frac{1}{(2^d)^2} H \times W,$6 with LoRA and $h \times w = \frac{1}{(2^d)^2} H \times W,$7 with full fine-tuning, and total trajectory compression up to $h \times w = \frac{1}{(2^d)^2} H \times W,$8 with ANTIC-LoRA and about $h \times w = \frac{1}{(2^d)^2} H \times W,$9 with ANTIC-FT (Cranganore et al., 10 Apr 2026).

A related but distinct scientific-data result concerns the trade-off between precision and temporal sampling. A spatiotemporally adaptive MGARD-based framework for extreme climate events argues that reducing data precision while preserving more timesteps can improve analysis outcomes relative to timestep decimation (Gong et al., 2024). In the E3SM cyclone-tracking case study, the adaptive framework improves perfectly matched cases in tropical cyclone tracking by $d=1$0 at medium compression ratios and by $d=1$1 at large compression ratios, with only $d=1$2 computational overhead (Gong et al., 2024). A common assumption is that exact values at fewer timesteps are preferable; this case study reports the opposite for a trajectory-dependent quantity of interest (Gong et al., 2024).

For LiDAR point clouds, temporal awareness takes the form of geometric reuse. Real-Time Spatio-Temporal LiDAR Point Cloud Compression chooses the middle point cloud in a sequence as a K-frame, transforms P-frames into the K-frame coordinate system using IMU-based 6-DoF motion estimation, converts them to range images, spatially encodes the K-frame by iterative plane fitting, and temporally encodes P-frames by reusing those planes for overlapped regions (Feng et al., 2020). Using KITTI, the paper reports that on average $d=1$3 of each point cloud is geometrically overlapped with the previous point cloud, enabling reuse of plane models across frames (Feng et al., 2020). The system achieves $d=1$4 to $d=1$5 compression, up to $d=1$6 FPS on a desktop CPU and up to $d=1$7 FPS on Nvidia Jetson TX2 (Feng et al., 2020).

Event-based sensing produces a different temporal-aware formulation because data are already sparse in space but dense in time. Flow-Based Visual Stream Compression predicts future events from transmitted events plus optical-flow vectors, yielding an average compression ratio of $d=1$8, a median temporal error of $d=1$9 ms, and an average spatiotemporal event-stream distance of $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$0; when combined with LZMA it reaches average compression ratios from $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$1 to $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$2 (Stumpp et al., 2024). A second event-stream method, PDS-LEC, aggregates events into polarity-separated temporal histograms, derives a quadtree spatial-priority map from adjacent intensity images, and applies Poisson Disk Sampling with rate-distortion-based allocation, reporting greater than $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$3 compression compared to the state of the art (Banerjee et al., 2020).

Dynamic 3D meshes and browser-streamed 4D fields show yet another variant. Fast Spatio-temporal Compression of Dynamic 3D Meshes uses differential coordinates, trajectory covariance eigenspaces, sparse coefficient coding, and anchor points to obtain very low bit-per-vertex per-frame rates (Arvanitis et al., 2021). A Practical Approach to Spatiotemporal Data Compression flattens 3D volumes into video-friendly image layouts and achieves 10–20 MB outputs from a 5.0 GB GRIB2 file for local browser visualization (Robinson et al., 2016). These systems are methodologically different, but both exploit the fact that temporal reuse is often more valuable when built on top of a suitable spatial representation.

5. Token, cache, and communication-oriented variants

Recent multimodal systems recast the problem in token space. SweetTok reduces video token count by decoupling spatial appearance and temporal motion into separate query autoencoders and separate language-informed codebooks (Tan et al., 2024). In the common setup, the method uses $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$4 spatial tokens and $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$5 temporal tokens, for a total of $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$6 tokens compared with $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$7 in OmniTokenizer (Tan et al., 2024). The paper reports a $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$8 improvement with respect to rFVD on UCF-101, a $$\mathbf{W}_{t+\Delta t} = \mathbf{W}_t + \Delta \mathbf{W}_{\Delta t},$$9 improvement with respect to gFVD in a compression-comparison setting, and compressed tokens that are semantically informative enough to support few-shot recognition with LLMs (Tan et al., 2024). Its central design claim is that compressing motion and appearance through a single flattened latent stream causes catastrophic temporal information loss (Tan et al., 2024).

DynaTok addresses a related problem for Video-LLMs, but in a training-free manner. Its Temporal Budget Allocation uses EMA memory to assign fewer tokens to redundant frames and more to novel frames, while Spatial Budget Allocation uses activation-based saliency plus spatial memory to reduce redundancy and positional bias (Park et al., 19 May 2026). Across MVBench, LongVideoBench, MLVU, and VideoMME, the method retains over $B_t = w_t \cdot T \cdot N \cdot R,$0 of baseline accuracy even with a $B_t = w_t \cdot T \cdot N \cdot R,$1 token reduction (Park et al., 19 May 2026). Supplementary latency analysis reports $B_t = w_t \cdot T \cdot N \cdot R,$2 ms savings at $B_t = w_t \cdot T \cdot N \cdot R,$3 retention and $B_t = w_t \cdot T \cdot N \cdot R,$4 ms savings at $B_t = w_t \cdot T \cdot N \cdot R,$5 retention relative to the full-token baseline (Park et al., 19 May 2026). The novelty here is not spatial saliency alone, but temporally adaptive budget assignment under long-horizon redundancy.

STAC applies the same logic to transformer memory rather than to input tokens. In causal VGGT-based streaming 3D reconstruction, the key-value cache grows linearly with stream length; STAC therefore keeps first-frame references, a recent sliding window, and high-value anchor tokens in a working temporal cache, while compressing evicted tokens into voxel-aligned spatial caches and jointly processing short chunks of consecutive frames (Wang et al., 18 Mar 2026). The reported outcome is nearly $B_t = w_t \cdot T \cdot N \cdot R,$6 lower memory consumption and about $B_t = w_t \cdot T \cdot N \cdot R,$7 faster inference while keeping reconstruction quality close to full Causal-VGGT (Wang et al., 18 Mar 2026). This shifts the topic from compression of raw data to compression of intermediate state in online models.

Communication systems provide another adaptation. SLATE, a SwinLSTM AutoEncoder for CSI compression in massive MIMO, jointly exploits temporal, spatial, and frequency correlations with a recurrent autoencoder rather than a separate recurrent module stacked on top of a large spatial-frequency autoencoder (Saini et al., 7 May 2025). At $B_t = w_t \cdot T \cdot N \cdot R,$8, $B_t = w_t \cdot T \cdot N \cdot R,$9, and $10.15\%$0, the paper reports $10.15\%$1 M parameters and $10.15\%$2 MFLOPs for SLATE, compared with $10.15\%$3 M parameters and $10.15\%$4 MFLOPs for ConvLSTM-TF TSF (Saini et al., 7 May 2025). It also reports CSI-overhead reductions of $10.15\%$5–$10.15\%$6 bits per CSI report relative to Rel-16 eTypeII and average user-throughput gains of $10.15\%$7–$10.15\%$8 in low-to-moderate mobility settings (Saini et al., 7 May 2025). Here temporal-aware spatial compression becomes temporal-spatial-frequency compression, but the structural idea remains unchanged: encode each spatial snapshot more efficiently by carrying state across time.

6. Evaluation regimes, misconceptions, and unresolved issues

One persistent misconception is that motion magnitude alone determines temporal compression difficulty. The 2026 video-quality study directly challenges this by showing that sequences with unpredictable or irregular dynamics can be less stable under compression than sequences with larger but structured motion (Zsoldos, 18 May 2026). Another misconception is that high spatial scores imply good video quality; the reported VMAF paradox shows that a codec can obtain very high VMAF through sharp I-frame snapshots while leaving temporal inconsistency unresolved (Zsoldos, 18 May 2026). A third misconception is that saving fewer exact timesteps is always preferable to storing more timesteps at lower precision; the climate case study reports superior cyclone tracking from temporally dense lossy outputs relative to timestep decimation at comparable storage cost (Gong et al., 2024).

The diversity of applications has produced correspondingly diverse evaluation regimes. Conventional video coding studies use PSNR, SSIM, VMAF, BD-Rate, ST-RRED, MOVIE, T-SSIM, T-PSNR, and FVD (Zsoldos, 18 May 2026). Generative video systems report FPS, G MACs, FID, FVD, and task-specific errors such as pose error (Zhuo et al., 2022). Scientific compressors emphasize relative $10.15\%$9 error, preservation of physics quantities such as enstrophy flux, or downstream analysis accuracy (Cranganore et al., 10 Apr 2026, Gong et al., 2024). Event-stream methods use temporal error and spatiotemporal event-stream distance (Stumpp et al., 2024). Token and cache compressors focus on downstream reasoning accuracy, latency, memory, and throughput (Park et al., 19 May 2026, Wang et al., 18 Mar 2026, Saini et al., 7 May 2025). This suggests that no single scalar metric captures the full objective of temporal-aware spatial compression across domains.

Domain	Temporal-aware spatial mechanism	Reported outcome
Video quality evaluation	Predictability anomaly; temporal metrics beyond VMAF	Non-linear temporal degradation (Zsoldos, 18 May 2026)
Vid2Vid synthesis	Low-resolution keyframes plus motion compensation	Around 20 FPS; around $23.93\%$0 compute saving (Zhuo et al., 2022)
PDE simulation	Physics-aware temporal selector plus neural fields	Up to $23.93\%$1 total compression (Cranganore et al., 10 Apr 2026)
LiDAR streams	Reuse of K-frame plane models on P-frames	$23.93\%$2–$23.93\%$3; up to $23.93\%$4 FPS (Feng et al., 2020)
Video-LLMs	Temporal and spatial budget allocation with memory	Over $23.93\%$5 baseline accuracy at $23.93\%$6 token reduction (Park et al., 19 May 2026)
Streaming 3D reconstruction	Temporal anchor cache plus voxel spatial cache	Nearly $23.93\%$7 less memory; about $23.93\%$8 faster inference (Wang et al., 18 Mar 2026)

Open issues are also consistent across the literature. Objective metrics remain insufficient in some settings; the temporal-consistency study explicitly proposes future Mean Opinion Score validation (Zsoldos, 18 May 2026). Some systems rely strongly on assumptions about the signal regime: SLATE is strongest in low-mobility scenarios (Saini et al., 7 May 2025); LiDAR compression depends on strong planar structure and high overlap (Feng et al., 2020); flow-based event compression depends on accurate optical flow and locally constant motion during the prediction phase (Stumpp et al., 2024). Other limitations concern model internals rather than signal structure: Fast-Vid2Vid depends on effective keyframe selection and motion compensation (Zhuo et al., 2022), STAC must balance anchor preservation against memory budgets (Wang et al., 18 Mar 2026), and the climate framework is sensitive to region-of-interest detection and buffer-zone design (Gong et al., 2024). A plausible implication is that future work will continue to couple temporal selection, spatial representation, and task-specific evaluation more tightly, rather than treating compression as a purely spatial fidelity problem.