LightQANet: Efficient Neural Compression

• LightQANet is a neural compression framework that integrates implicit scene representation, low-rank constraints, and quantization-aware training for efficient model compression.
• It employs a single-MLP NeRF formulation and Tensor Train decomposition to significantly reduce parameters while maintaining high visual fidelity.
• The framework achieves superior light field image compression with improved PSNR and enables novel view synthesis for resource-constrained applications.

LightQANet is a framework for neural network compression that integrates low-rank model constraints, efficient quantization, and tensor decomposition for high-dimensional tasks, particularly oriented toward implicit scene representation and light field image compression. The approach synthesizes concepts from neural radiance field modeling, advanced optimization, and quantization-aware network training, resulting in a model that minimizes memory and compute requirements while retaining high visual fidelity and enabling novel-view synthesis.

1. Implicit Scene Representation via Simplified Neural Radiance Field

LightQANet is fundamentally based on a Neural Radiance Field (NeRF) formulation, wherein a single multi-layer perceptron (MLP) maps 5D coordinates—comprising the spatial location $(x, y, z)$ and viewing directions $(\theta, \phi)$—to color and density outputs. Specifically, the representation is expressed as

$$(\mathrm{RGB}, \sigma) = F_{\Theta}(x, y, z, \theta, \phi),$$

with $\Theta = \{W_i, b_i\}$ denoting the set of MLP weight matrices and biases. Unlike standard NeRF implementations that utilize separate coarse and fine networks, LightQANet employs a single-MLP construction for compactness. Training this network over multiple light field sub-aperture images enables both parameter-efficient compression and the neural synthesis of novel views, supplanting the need to transmit full sub-aperture image sets.

2. Low-Rank Constraints via ADMM and Tensor Train Decomposition

Post initial NeRF training, LightQANet imposes a low-rank constraint on the network parameters to facilitate compression. The optimization objective minimizes the $\ell_2$ norm reconstruction loss subject to a rank bound:

$$\min_{W_i, b_i, \ldots} \|c' - c_{gt}\|_2^2 \quad \text{subject to} \quad \operatorname{rank}(W_i) < r,$$

where $c'$ is the rendered pixel value and $c_{gt}$ is the ground truth. To manage the non-convexity inherent in rank constraints, the Alternating Direction Method of Multipliers (ADMM) is employed, introducing auxiliary variables $Z_i$ and enforcing low-rank feasibility via indicator functions and projection:

$Z_i^{(t+1)} = \Pi_r(W_i^{(t+1)} + U_i^{(t)}),$

with $\Pi_r$ denoting projection to the nearest rank-$r$ subspace.

The rank-reduced matrices are subsequently factored using Tensor Train (TT) decomposition, yielding

$W_i = Q_i^1 Q_i^2,$

where $Q_i^1 \in \mathbb{R}^{n_1 \times r}$ and $Q_i^2 \in \mathbb{R}^{r \times n_2}$ for vastly reduced parameter count ($r \ll n_1, n_2$). This TT format preserves the expressivity of the original weight matrix while setting the stage for efficient quantization.

3. Quantization-Aware Training and Codebook Optimization

Following TT decomposition, LightQANet applies rate-constrained quantization to dramatically limit per-parameter storage. The distribution of TT parameters typically localizes within $[-1, 1]$; thus, a global non-uniform codebook $C$ is extracted via $k$-means clustering, mapping each parameter to its nearest centroid. This unified codebook obviates the need for per-layer flags, thereby reducing total bits required.

For rare outlier values outside $[-1, 1]$, standard 16-bit quantization is applied, safeguarding reconstruction accuracy for critical weights. Quantization is performed sequentially, layer-wise, with quantized layers frozen and subsequent layers re-trained to offset propagated errors. The optimization is posed as

$$\min_{W_i, b_i, \forall i > i'} \|c' - c_{gt}\|_2^2 \quad \text{subject to fixed quantized } Q_i \text{ for } i \leq i',$$

permitting quantization-aware adaptation. After quantization, Huffman coding is used across all TT components for further bitrate reduction while retaining efficient codebook referencing.

Due to the complications of optimizing low-rank and quantization constraints jointly, LightQANet includes a network distillation phase wherein the higher-capacity LR-NeRF (teacher) is distilled into a smaller DLR-NeRF (student), initialized directly from the TT components. This separation facilitates quantization without degradation of the learned low-rank structure.

4. Compression Efficiency and Experimental Validation

Empirical analysis demonstrates that LightQANet, realized as QDLR-NeRF, attains higher peak signal-to-noise ratio (PSNR) at moderate bitrates (bits per pixel, bpp) compared with standards such as HEVC, JPEG-Pleno, and deep learning-based codecs (RLVC, HLVC, OpenDVC). On synthetic light field scenes, improvements of approximately 1 dB PSNR over leading competitors are observed.

Rate-distortion metrics highlight the consistency of synthesized view quality; neural approaches outstrip inter-frame prediction codecs due to their implicit spatial representation. Sequential application of constraints—low-rank, distillation, and quantization—yields parameter reductions from 100% (uncompressed) to 3.3% post-quantization with negligible visual loss. Ablation confirms the necessity of each step for optimal compression and fidelity.

5. Applications in Light Field Imaging and Resource-Constrained Deployment

LightQANet is particularly relevant for scenarios demanding aggressive data reduction and flexible viewpoint generation. By transmitting a compact QDLR-NeRF representation rather than dozens or hundreds of sub-aperture images, network-based compression enables:

• Flexible View Synthesis: Generation of novel perspectives from a unified, implicit scene code, critical for VR, AR, and interactive displays.
• Resource Efficiency: Storage and transmission of neural codes at very low bitrates; direct deployment on mobile or embedded hardware with limited memory.

The methodology extends naturally to other NeRF variants and promotes general principles of neural network compression for high-dimensional signal data.

6. Conceptual Adaptations from LR-QAT

Techniques from Low-Rank Quantization-Aware Training (LR-QAT) for LLMs (Bondarenko et al., 10 Jun 2024) provide avenues to further streamline LightQANet:

• Low-Rank Auxiliary Weights: Embedding quantization-aware low-rank adapters within the TT decomposition may allow endogenous compensation for quantization error, as in LR-QAT.
• Advanced Downcasting: Employing fixed-point or double-packed integer representations can minimize memory without significant accuracy loss.
• Checkpointing: Gradient checkpointing can reduce training memory further by recomputing activations as needed.
• General Extended Pretraining: Retaining a general-purpose backbone post-quantization enables broad downstream applicability.

This suggests that LightQANet may adopt LR-QAT design elements to further enhance training and inference efficiency in future iterations.

7. Broader Significance and Research Directions

LightQANet exemplifies a unified approach to neural scene compression, combining low-rank optimization, tensor decomposition, and quantization-aware training within a practical workflow. Its demonstrated advances in light field compression and novel view synthesis underpin ongoing research into neural representations for high-dimensional data, efficient coding strategies, and scalable deployment architectures. The modularity of the methodology allows adaptation across diverse domains requiring implicit modeling and data-efficient transmission. Future research may focus on optimizing TT quantization strategies, improving distillation dynamics, and integrating additional memory-saving protocols.