Progressive Learning on Gaussian Compaction

• Progressive Learning on Gaussian Compaction is a framework that utilizes hierarchical Gaussian structures to progressively encode high-dimensional data for efficient learning and inference.
• It implements multi-resolution representations and compression techniques, enabling scalable rendering and memory-efficient continual learning across various tasks.
• Empirical results demonstrate near-real-time performance, high compression rates, and improved segmentation accuracy in applications like 3D scene rendering and semantic segmentation.

Progressive Learning on Gaussian Compaction (PLGC) encompasses a suite of methodologies in which models, parameters, or representations are learned, stored, or transmitted in a progressive, multi-resolution fashion using Gaussian-based compaction and filtering. The term spans approaches in neural scene representations, continual learning, and semantic segmentation that leverage hierarchical Gaussian structures, low-rank approximations, and coarse-to-fine encoding to achieve scalable memory, computational efficiency, and enhanced adaptability across tasks or level-of-detail requirements.

1. Foundations of Gaussian Compaction and Hierarchical Representations

Gaussian compaction refers to representing high-dimensional information—such as scene geometry, neural network weights, or semantic features—efficiently using parameterized mixtures of Gaussians or low-rank Gaussian approximations. In 3D Gaussian Splatting (3DGS), a scene is described as a set of $N$ Gaussians ("splats") where each primitive $i$ has parameters $(\mu_i, \Sigma_i, o_i)$ for 3D mean, covariance, and opacity, plus view-dependent color modeled with spherical harmonics (Sario et al., 23 Jan 2025). Aggregated, the scene density along a ray is a mixture model,

$$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$

with $w_i \propto o_i$ and $\mathcal{N}(\cdot\;;\mu, \Sigma)$ the Gaussian density. Such representations can be compressed via codebooks, entropy coding, and aggressive quantization, or further compacted through multi-level binary encoding for semantic tasks (Yang et al., 30 Nov 2025).

In weight-space continual learning, Gaussian compaction manifests as maintaining a diagonal plus low-rank approximation of the posterior covariance of neural network parameters across sequential tasks, allowing tractable Bayesian inference in high dimensions (Sliwa et al., 2024). The weight evolution is governed by a Gaussian state-space model:

$$\theta_{t+1} = \theta_t + \omega_t,\quad \omega_t \sim \mathcal{N}(0,Q)$$

2. Progressive Learning Strategies

PLGC approaches organize Gaussians or parameterizations into hierarchical, multi-resolution structures. In progressive level-of-detail (LoD) for 3DGS, the collection of Gaussians is partitioned into $L$ nested levels:

$$G_0 \subset G_1 \subset \cdots \subset G_{L-1} = \text{full set}$$

where $G_0$ (base) contains the most critical primitives, and each successive set $i$0 adds finer detail (Sario et al., 23 Jan 2025). This hierarchy is constructed by iteratively ranking Gaussians by the $i$1-norm of their loss gradients,

$i$2

and peeling off less important ones into higher LoDs.

For segmentation, progressive learning is achieved by decomposing the labeling task into $i$3 granularity levels, progressively refining from coarse to fine via multi-level binary codes per Gaussian,

$i$4

which are concatenated for inference and segmentation, enabling both compact memory usage and multi-scale semantic control (Yang et al., 30 Nov 2025).

In continual learning, the progressive mechanism is achieved through sequential Laplace–Gaussian filtering, recursively updating parameter posteriors $i$5 for each task $i$6 in a way that accommodates drift (plasticity) and memory (stability) (Sliwa et al., 2024).

3. Compression and Memory Efficiency

PLGC-driven frameworks employ advanced compression steps to support scalability and minimal footprint. In progressive 3DGS (GoDe), after hierarchy construction, two key compression mechanisms are applied (Sario et al., 23 Jan 2025):

• SH culling and fine-tuning: Random sampling over levels and pruning of spherical harmonic bands per Gaussian, typically reducing from 4 bands (9 coefficients each) to 0–3 bands, with per-level fine-tuning for visual fidelity.
• K-means codebook quantization and entropy coding: Each parameter type is discretized using a 256-entry codebook, reducing per-Gaussian parameter storage to 8 bits (codebook index) plus 16-bit (FP16) precision for positions; subsequent ZIP/LZ77 entropy coding further lowers the bitrate.

These steps yield scene models that range in size from $i$73.5 MB (base) up to $i$858 MB (full), corresponding to $i$90.2 dB PSNR drop relative to uncompressed scenes.

For semantic segmentation, per-Gaussian features are compacted to a single $(\mu_i, \Sigma_i, o_i)$0-bit (typically $(\mu_i, \Sigma_i, o_i)$1=32) integer, effecting a 32$(\mu_i, \Sigma_i, o_i)$2 compression over a 32-float vector, while retaining or improving accuracy (Yang et al., 30 Nov 2025). In continual learning, memory efficiency is achieved by storing only diagonal and rank-$(\mu_i, \Sigma_i, o_i)$3 components (with $(\mu_i, \Sigma_i, o_i)$4) of the covariance, obviating the need for dense $(\mu_i, \Sigma_i, o_i)$5 matrices (Sliwa et al., 2024).

4. Progressive Streaming, Scalability, and Real-Time Adaptivity

A central property of PLGC is progressive transmission and adaptation. In streaming scenarios, each LoD layer is stored and transmitted separately, enabling clients with varying compute, memory, or bandwidth budgets to incrementally load and render models up to their constraints (Sario et al., 23 Jan 2025). Each LoD is a valid submodel; no retraining is required when switching LoDs.

High-level pseudocode for progressive streaming supports requests for incremental model enhancement, adapting rendering throughput (FPS) to available system resources: $w_i \propto o_i$0 This architecture ensures amortized download and memory requirements for clients, with empirical results demonstrating $(\mu_i, \Sigma_i, o_i)$6350 FPS at minimal LoD and $(\mu_i, \Sigma_i, o_i)$7200 FPS at intermediate compression.

5. Hierarchical Loss Functions and Optimization

In PLGC-based segmentation, optimization decomposes the loss into hierarchical levels. For each granularity $(\mu_i, \Sigma_i, o_i)$8, a contrastive loss aligns or separates features in binary code space based on mask membership:

• Pull features together if their corresponding pixels share the same mask at level $(\mu_i, \Sigma_i, o_i)$9
• Push features apart if they share the same mask at level $$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$0 but differ at level $$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$1
• No constraint otherwise

A binarization regularizer encourages hard binary (0/1) codes, while a virtual-negative loss penalizes collapse in low-diversity regimes (Yang et al., 30 Nov 2025):

$$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$2

Opacity $$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$3 is fine-tuned, decoupling photometric and segmentation objectives for improved accuracy in semi-transparent and complex cases.

In continual learning, hierarchical loss manifests via the regularized MAP step in Laplace–Gaussian filtering. The quadratic penalty integrates task-sequential prior knowledge while optimizing for the current task, with stability–plasticity trade-offs controlled by the process noise $$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$4 and regularization strength $$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$5 in the filtering equations (Sliwa et al., 2024).

6. Empirical Performance and Practical Benefits

Empirical results across PLGC-based systems show substantial advances:

• 3D Scene Compression and Rendering: Up to 99.8% parameter reduction versus uncompressed scenes, $$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$60.2 dB PSNR drop, and real-time render throughput (Sario et al., 23 Jan 2025).
• Segmentation: With 32-bit per-Gaussian encoding, Binary-Gaussian reaches 87.1% mIoU (coarse) and 86.2% (fine) on LERF-Mask; 94.3% mIoU on SPIn-NeRF, outperforming all 3D-GS baselines. Memory and speed gains are significant: 769 FPS versus 50–100 FPS in clustering-based baselines (Yang et al., 30 Nov 2025).
• Continual Learning: Efficient filtering and smoothing can be performed in $$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$7 (with $$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$8) per task, with no access to past data needed for smoothing. The diagonal plus low-rank format allows for domain/task adaptation via structured $$G(x) = \sum_{i=1}^N w_i \mathcal{N}(x; \mu_i, \Sigma_i)$$9 selection and time-varying dynamics (Sliwa et al., 2024).

Transitions between progressive LoD levels are artifact-free owing to targeted fine-tuning and regularization. Lower LoDs capture coarse structure; higher ones add fine geometric or semantic details.

7. Theoretical and Practical Implications

PLGC frameworks present a unifying abstraction for efficiency in learning, storage, and inference with Gaussian-based models. The diagonal plus low-rank approach in continual learning offers a computationally transparent route to exact Bayesian updating in high-dimensional weight spaces. Importance-based hierarchy construction, codebook quantization, and progressive binary encoding are critical for scalable real-world deployment, supporting on-demand streaming or task adaptation without retraining or manual tuning. Granular control over progressive model evolution (FloD), semantic granularity, and plasticity–stability balancing enables broad applicability—ranging from real-time neural rendering to lifelong learning systems and memory-constrained semantic inference, establishing PLGC as a cornerstone methodology in scalable, adaptive AI.