LLMCodec: Adapting Video Codecs for Efficient Weight Compression of Large Language Models

Abstract: The rapid development of LLMs(LLMs) has led to remarkable advances in natural language processing. However, the increasing scale of these models introduces substantial challenges in terms of storage, transmission, and deployment. Though great efforts have been devoted to model compression and quantization, existing methods often rely on fine-tuning or calibration data, which exhibit limited generalization across different tensor types. In this paper, we argue that video codecs offer a promising solution for LLM compression, due to their inherent compatibility with matrix structured data, configurable compression strategies, and the availability of highly optimized, off-the-shelf implementations. Therefore, we present LLMCodec, a video codec-based LLM compression method that integrates affine quantization with the recent VVC/H.266 video codec. Beyond VVC, we further compare a range of video codecs and encoding profiles to evaluate their impact on compression performance. Experiments on different models demonstrate the robustness and generality of LLMCodec. Notably, on LLaMA-3-8B at 2-bit precision, LLMCodec reduces perplexity by over 1.5x and improves downstream task accuracy by 21% compared with the existing method.

• The paper introduces a novel approach by repurposing video codecs with learnable affine transformations to efficiently compress large language model weights.
• It demonstrates significant improvements in perplexity and downstream accuracy, achieving up to a 36% reduction in perplexity and a 21% increase in accuracy.
• The method’s codec-agnostic design enables seamless integration into deployment pipelines, promoting scalable and resource-efficient AI systems.

LLMCodec: Adapting Video Codecs for Efficient Weight Compression of LLMs

Introduction

The scaling of LLMs has yielded enhanced capabilities across numerous NLP tasks, but the massive parameter counts have dramatically increased the burdens of storage, transmission, and deployment. Classical model compression and quantization approaches—especially post-training quantization (PTQ)—demonstrate practical utility but often suffer from rigidity in bit-width control, reliance on fine-tuning or calibration data, and limited resilience at ultra-low precision. The paper "LLMCodec: Adapting Video Codecs for Efficient Weight Compression of LLMs" (2606.05861) introduces a distinct perspective: repurposing mature video codecs as a vehicle for LLM weight compression, augmented by an outlier mitigation mechanism using learnable affine transformations.

Methodology

The LLMCodec pipeline comprises two major components: (1) a learnable affine transformation to suppress weight outliers prior to quantization, and (2) a structured mapping of quantized weights to a video format, leveraging the sophisticated rate–distortion control and compression efficiency of modern video codecs, specifically VVC/H.266 via VVenC.

The affine transformation is trained to minimize reconstruction error post-quantization, yielding weight distributions with reduced dynamic range and improved homogenization. Subsequently, weights are quantized using round-to-nearest (RTN) to INT8 and mapped exclusively to the Y channel of the YUV420 video format. The U and V (chrominance) channels are filled with constants, ensuring negligible bitrate overhead as codecs can trivially predict their content.

The final video-encoded representations are compressed with VVenC using the All-Intra profile, identified as the optimal profile for weight tensors given intra-frame redundancy properties and precision consistency requirements across layers.

Figure 1: Overview of the LLMCodec framework, including outlier mitigation via learnable affine transformation and weight-to-video mapping followed by video-based compression.

Compression Performance

Across standard LLMs—including LLaMA-3-8B, LLaMA-2-7B, and Qwen-2.5-Instruct-7B—LLMCodec consistently demonstrates superior rate–distortion characteristics under extreme compression. Notably, when compressing LLaMA-3-8B to 2 bits per parameter, LLMCodec reduces WikiText2 perplexity from 41.15 (FlatQuant) to 26.53 (a 36% reduction) and raises average downstream accuracy by 21%. On Qwen-2.5-7B-Instruct, perplexity drops further (14.76 to 14.01 on WikiText2; 36.17 to 24.20 on C4), with an analogous accuracy improvement. Although on LLaMA-2-7B the perplexity slightly increases compared to FlatQuant, the downstream accuracy still improves by 10%, emphasizing the method's practical advantage in application-driven deployments.

Figure 2: Rate-distortion curves for LLaMA-3-8B compression on WikiText2, C4, and average downstream task accuracy, comparing LLMCodec, GPTQ, and FlatQuant.

Figure 3: Downstream reasoning task performance (ARC-Challenge) across different compression rates, with LLMCodec yielding consistently higher accuracy.

Ablation and Codec Selection Analysis

Ablation studies reveal the criticality of affine outlier mitigation: omitting the transformation causes as much as two orders of magnitude deterioration in perplexity and significant accuracy collapse at low bit-width. RTN quantization, rather than GPTQ, is sufficient when paired with the codec, owing to the uniformity requirement of codec inputs and the outlier-removal step.

A further analysis of codec configurations underscores the primacy of VVC/H.266 in weight compression. Among tested codecs—JPEG, WebP, HEVC/H.265, and VVC/H.266—the latter exhibits superior rate–distortion behavior when evaluated in terms of per-plane PSNR and downstream model fidelity. The All-Intra profile emerges as optimal, as inter-frame coding structures (Random-Access, Low-Delay) introduce uneven precision across layers, which is detrimental given the "weakest link" effect in neural network inference.

Figure 4: Per-plane PSNR comparisons highlight the superiority of mapping weights exclusively to the Y channel, ensuring homogenous precision.

Implications and Future Directions

LLMCodec illustrates the viability of transposing established multimedia compression methodologies into neural model deployment, with ready advantages in scalability, compression quality, and practical deployment flexibility. By harmonizing the statistical properties of LLM weights with video codec operating regimes through learnable affine transforms, the approach leverages decades of codec optimization for efficient, rate-controllable model compression.

From a systems perspective, this technique can be seamlessly integrated into model distribution and inference platforms without requiring extensive modification to downstream computational pipelines. Its codec-agnostic modularity offers room for further efficiency gains as new video coding standards emerge.

Theoretically, the work highlights a fruitful cross-pollination between signal processing and neural compression, motivating further research in designing network architectures, quantization schemes, or training objectives explicitly tailored to codec-friendly representations. Future work could target activation compression and full pipeline codec-aware quantization, addressing both the static and dynamic memory bottlenecks of LLM deployment.

Conclusion

LLMCodec demonstrates that sophisticated video codecs, when paired with learnable outlier mitigation, provide an effective, robust mechanism for low-bit-width LLM compression, outperforming established quantization paradigms on perplexity and downstream accuracy at extreme compression ratios. The approach's flexibility and reliance on off-the-shelf codecs imply broad applicability in the dissemination and resource-efficient deployment of LLMs. The results motivate future research at the intersection of neural compression, quantization, and multimedia coding to further improve model scalability and accessibility (2606.05861).