Hierarchical Local Quantizer (HLQ)

• HLQ is a hierarchical quantization method that compresses vectors by sequentially encoding residuals with dependent subcodebooks.
• Its coarse-to-fine structure strikes a balance between PQ’s speed and AQ’s low error, making it ideal for large-scale, high-dimensional data.
• Efficient residual k-means updates and sequential codebook refinement enable HLQ to achieve competitive benchmarks on traditional and deep feature datasets.

A Hierarchical Local Quantizer (HLQ) is a quantization methodology for compositional vector compression that imposes a coarse-to-fine hierarchical structure on the quantization codebooks. HLQ achieves low quantization error and fast encoding by sequentially quantizing residuals with a series of dependent subcodebooks, offering a practical middle ground between Product Quantization (PQ) and Additive Quantization (AQ). The approach has demonstrated strong empirical performance and encoding efficiency on benchmarks for both traditional and deep learning-derived features.

1. Mathematical Framework and Hierarchical Residual Encoding

HLQ approximates a real-valued data vector $x \in \mathbb{R}^d$ as an additive composition of quantizations from $m$ subcodebooks: $x \approx \sum_{i=1}^{m} C_i b_i$ where each $C_i \in \mathbb{R}^{d \times h}$ is a subcodebook containing $h$ codewords, and $b_i$ is a one-hot indicator vector with $\lVert b_i \rVert_0 = \lVert b_i \rVert_1 = 1$ selecting a codeword from $C_i$.

Encoding proceeds hierarchically:

• The optimal codeword $b_1$ in $C_1$ approximates $x$: $b_1 = \arg\min_{b_1} \lVert x - C_1 b_1 \rVert^2$.
• The residual $r_1 = x - C_1 b_1$ is then quantized by $C_2$, yielding $b_2 = \arg\min_{b_2} \lVert r_1 - C_2 b_2 \rVert^2$, and so forth.
• Generally, $r_i = r_{i-1} - C_i b_i$ with $r_0 = x$, and: $x \approx C_1 b_1 + C_2 b_2 + \dots + C_m b_m$

This greedy scheme results in fast, sequential encoding and allows codebook updates via residual k-means clustering. Each codebook is refined using residuals computed with partial reconstructions, ensuring hierarchical consistency throughout iterative refinement.

2. Comparative Analysis with PQ and AQ

HLQ's design explicitly positions it between PQ and AQ:

Method	Codebook Dependency	Quantization Error	Encoding Complexity
PQ	Independent (block-diagonal)	Higher (data rarely fits independent subspaces)	$O(h \cdot d)$
AQ	Fully dependent (joint optimization)	Lower	$O(m^3 b h d)$ (NP-hard; beam search)
HLQ	Hierarchical, residual-dependent	Comparable or better than AQ (esp. deep/CNN features)	$O(m h d)$

• PQ encodes by decomposing $x$ into $m$ disjoint subspaces and quantizing each independently.
• AQ encodes $x$ by selecting codewords jointly from $m$ interdependent subcodebooks, minimizing overall error but incurring NP-hard encoding.
• HLQ encodes via residual quantization: the first codebook captures a coarse approximation, subsequent codebooks quantize errors hierarchically. This yields quantization error on par with AQ but at a computational cost orders of magnitude lower.

3. Codebook Refinement and Optimization

After initialization (typically via k-means on $x$ and successive residuals), HLQ employs a top-down codebook refinement strategy. For each $C_i$:

• Compute partial reconstruction excluding $C_i$: $\hat{H}^{(-i)} = \hat{H} - C_i B_i$.
• Update $C_i$ using k-means on $X - \hat{H}^{(-i)}$ with predefined assignments $B_i$.

This preserves hierarchical ordering, avoids breaking greedy encoding, and enables efficient sequential updates. The quadratic scaling in codebook count remains substantially less expensive than joint optimization in AQ.

4. Empirical Performance and Benchmarking

HLQ's efficacy is demonstrated on SIFT1M (128-dim), GIST1M (960-dim), and deep CNN-based features (ConvNet1M-128):

• On SIFT1M and GIST1M, HLQ achieves quantization errors matching AQ.
• On deep features, HLQ sometimes outperforms AQ.
• Encoding 1M vectors can be completed in ~20 seconds using HLQ, whereas AQ encoding (with beam search) may take several hours.

Benchmark evaluations reveal HLQ delivers better recall in nearest neighbor search, reduced quantization error, and significantly lower encoding time compared to PQ, OPQ, and AQ.

5. Computational Efficiency and Scalability

HLQ encoding complexity is nearly linear in the number of codebooks, $O(m h d)$, preserving practical suitability for large-scale deployments. The refinement phase scales quadratically with codebook number but remains tractable relative to the intractability of AQ. This efficiency makes HLQ attractive for large vector databases, object recognition, compression, and retrieval in both classical and deep learning contexts.

6. Significance and Application Domains

HLQ's hierarchical structure combines the expressive low error of dependent codebooks with fast encoding approaching independent (PQ) methods:

• Suited for approximate nearest neighbor search at scale, where quick encoding and low quantization error are essential.
• Effective in representing deep, high-dimensional features, exhibiting particular robustness in convolutional neural networks.
• The mathematical elegance of sequential residual encoding ($x \approx \sum_i C_i b_i$ with $r_i = r_{i-1} - C_i b_i$) offers a general design pattern for quantization schemes balancing speed and accuracy.

A plausible implication is that such hierarchical residual quantization frameworks—when extended with neural or adaptive components—could generalize to new types of structured data and enable efficient, scalable compression with controllable error.

7. Related Hierarchical Quantization Methods

While HLQ (“Stacked Quantizers”) (Martinez et al., 2014) established the coarse-to-fine additive framework for vector compression, related hierarchical quantization concepts appear elsewhere:

• Deep Hierarchical Quantization Compression algorithms in federated learning (Jiang et al., 2022) employ multi-bit hierarchical schemes for model gradient compression, leveraging sparsification followed by multi-bit residual quantization.
• DeepHQ for progressive image coding (Lee et al., 22 Aug 2024) uses learned, channel-wise hierarchical quantization steps applied progressively to latent codes, with selective masking to optimize rate-distortion and model size.
• Multi-layer Hierarchical Federated Learning with Quantization (Azimi-Abarghouyi et al., 13 May 2025) utilizes nested multi-layer aggregation where each layer applies a specific quantizer, optimizing tradeoffs between communication, error, and latency under deadline constraints.

These approaches illustrate the versatility and broad applicability of hierarchical (residual) quantization, extending the HLQ principle to federated optimization, deep image coding, and communication-constrained distributed learning.

---

In summary, HLQ introduced a hierarchical structure for local quantization that achieves competitive or superior quantization error relative to jointly optimized AQ while maintaining nearly linear encoding speed, thus providing a principled solution for large-scale vector compression and serving as inspiration for hierarchical quantization algorithms in modern machine learning systems.