QuantFace: Low-Bit Post-Training Quantization for One-Step Diffusion Face Restoration (2506.00820v1)

Abstract: Diffusion models have been achieving remarkable performance in face restoration. However, the heavy computations of diffusion models make it difficult to deploy them on devices like smartphones. In this work, we propose QuantFace, a novel low-bit quantization for one-step diffusion face restoration models, where the full-precision (\ie, 32-bit) weights and activations are quantized to 4$\sim$6-bit. We first analyze the data distribution within activations and find that they are highly variant. To preserve the original data information, we employ rotation-scaling channel balancing. Furthermore, we propose Quantization-Distillation Low-Rank Adaptation (QD-LoRA) that jointly optimizes for quantization and distillation performance. Finally, we propose an adaptive bit-width allocation strategy. We formulate such a strategy as an integer programming problem, which combines quantization error and perceptual metrics to find a satisfactory resource allocation. Extensive experiments on the synthetic and real-world datasets demonstrate the effectiveness of QuantFace under 6-bit and 4-bit. QuantFace achieves significant advantages over recent leading low-bit quantization methods for face restoration. The code is available at https://github.com/jiatongli2024/QuantFace.

QuantFace: Low-Bit Quantization for Efficient Face Restoration

The paper introduces QuantFace, a novel approach for reducing the computational complexity of face restoration models through low-bit quantization, specifically designed for one-step diffusion processes. This methodology caters to devices with limited resources like smartphones, where deploying traditional diffusion models can be computationally prohibitive.

Key Contributions

QuantFace applies a series of techniques to ensure that low-bit models maintain performance comparable to their full-precision counterparts. The paper outlines several contributions:

1. Rotation-Scaling Channel Balancing: By analyzing activation distributions, the authors find significant variance across channels that hinder quantization fidelity. To address this, QuantFace employs rotation-scaling techniques to balance channels effectively. This technique uniquely combines the Randomized Hadamard Transform with scaling strategies to reduce quantization error while preserving essential facial structure features.
2. Quantization-Distillation Low-Rank Adaptation (QD-LoRA): This method introduces dual low-rank branches to improve alignment between the full-precision and quantized models. SVD is utilized for initialization, enhancing performance by adding more flexibility in the learning process, and thereby better compensating for the quantization-induced distortion.
3. Adaptive Bit-width Allocation: Handled via integer programming, this strategic allocation of bit-width across layers factors in layer sensitivity to quantization, optimizing both perception and reconstruction error. It aids significantly in optimizing computational costs within a constrained bit budget.

Results

The authors validate QuantFace's efficacy through extensive experimentation on both synthetic (CelebA-Test) and real-world datasets (Wider-Test, WebPhoto-Test, LFW-Test). QuantFace exhibits superior performance over existing post-training quantization methods, even under severely low-bit settings like W4A4. For example, at this extreme setting, QuantFace yields up to 84.85% parameter compression and a 82.91% reduction in computational operations compared to full-precision models while maintaining high perceptual quality—as indicated by improved FID, LPIPS, CLIP-IQA scores across datasets.

Implications

QuantFace represents a significant step forward in facilitating the deployment of high-performing AI-driven face restoration models in resource-constrained environments, bridging the gap between sophisticated computational models and practical, real-world applications. By focusing on techniques such as efficient quantization and low-rank adaptations, this framework paves the way for compact AI models that retain the robustness and accuracy typical of larger, more computationally intensive systems.

Future Directions

Looking ahead, the QuantFace framework could inspire further research on quantization strategies tailored for different diffusion-based applications beyond face restoration. Exploration into additional components like layer-wise adaptive quantization algorithms could further enhance the efficiency and robustness of deployed models. Additionally, integrating such quantization methods with advancements in edge AI could expand AI's scope of application, particularly in fields requiring real-time processing and decision-making.

In summary, the QuantFace paper contributes substantially to the field of efficient AI by demonstrating how low-bit quantization can be effectively employed in sophisticated domains like face restoration without compromising quality and utility. Its methodologies and findings are poised to influence future developments in AI model compression and deployment on resource-limited platforms.