---
title: Block-Householder Quantizer (BHQ)
url: https://www.emergentmind.com/topics/block-householder-quantizer-bhq
type: topic
---

# Block-Householder Quantizer (BHQ)

The Block-Householder Quantizer (BHQ) is an unsupervised, hyperparameter-free quantization strategy for deep image hashing that decouples the learning of similarity-preserving embeddings from the subsequent binarization step. Unlike standard approaches that integrate a quantization penalty into the similarity loss, introducing complex trade-offs and challenging optimization, BHQ decomposes the problem into two separate stages: first, learning continuous embeddings using only similarity-based loss, and second, finding an optimal orthogonal transformation—parametrized as a product of Householder reflectors—to align the embeddings with their binary counterparts before quantization by the sign function. Empirical evidence demonstrates that this separation mitigates performance degradation, enables consistently stronger retrieval results, and requires less tuning compared to conventional methods [2311.04207].

## 1. Two-Stage Decomposition and Formulation

BHQ operates via a structured two-phase procedure:

1. **Similarity Learning Stage**: A standard deep embedding $f_\theta : \mathbb{R}^d \rightarrow \mathbb{R}^k$ is learned by minimizing a similarity-driven objective
$$
L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))
$$
with no quantization term. Here, $s_{ij} \in \{0,1\}$ encodes pairwise similarity, $\ell_S$ and $\ell_D$ represent the chosen similarity and dissimilarity losses. This stage uses frameworks such as contrastive, cross-entropy, or proxy losses.

2. **Householder Quantization Stage**: The learned embeddings $f_\theta$ are normalized for each sample $x_i$ to $\bar{f}_i = \sqrt{k} f_\theta(x_i)/\|f_\theta(x_i)\|_2$ (on the $\sqrt{k}$-radius sphere). Then, one seeks an orthogonal matrix $U \in O(k)$ that makes each $U\bar{f}_i$ as close as possible to its sign vector. The resulting $k$-bit hash is $h(x) = \mathrm{sign}(U f_\theta(x))$.

This formulation leverages similarity-invariant properties of orthogonal transformations, so the choice of $U$ does not affect the preservation of pairwise similarities [2311.04207].

## 2. Quantization Objective and Optimization

The core rounding problem formulated in the quantization stage is:
$$
U^* = \arg\min_{U \in O(k)} \frac{1}{n} \sum_{i=1}^n \| U \bar{f}_i - \mathrm{sign}(U \bar{f}_i) \|_2^2 = \arg\min_{U \in O(k)} L_Q(U)
$$
where the objective $L_Q(U)$ quantifies the mean-squared error between the transformed embeddings and their sign vectors.

This least-squares problem over the orthogonal group $O(k)$ is solved efficiently by parameterizing $U$ via Householder reflections and minimizing $L_Q(U)$ using stochastic gradient descent (SGD).

## 3. Householder Parametrization of Orthogonal Transform

Any orthogonal matrix $U \in O(k)$ can be written as a product of $k$ Householder reflectors:
$$
U = H(v_k) H(v_{k-1}) \cdots H(v_1)
$$
where each Householder reflection $H(v)$ has the form
$$
H(v) = I_k - 2 \frac{v v^\top}{\|v\|_2^2}, \qquad v \in \mathbb{R}^k \setminus \{0\}.
$$
Application of $H(v)$ to a vector $y$ is computed via $H(v)y = y - 2 \frac{v^\top y}{\|v\|_2^2} v$, requiring only $O(k)$ operations per reflection. The full map requires $O(k^2)$ time to apply $k$ reflectors.

This parametrization enables direct optimization of the $k^2$ free parameters $v_1, \ldots, v_k$ and ensures storage costs remain low (at most $k^2$ floats, i.e., $\leq 16$ kB for $k \leq 64$) [2311.04207].

## 4. Stochastic Gradient Descent Updates

With $F_i = U\bar{f}_i$, the quantization loss is $L_Q(v_1, \dots, v_k) = \frac{1}{n} \sum_{i=1}^n \|F_i - \mathrm{sign}(F_i)\|_2^2$. Gradients with respect to $U$ are computed as:
$$
\nabla_U L_Q = \frac{2}{n} \sum_{i=1}^n (F_i - \mathrm{sign}(F_i)) \bar{f}_i^\top.
$$
Backpropagation proceeds through the Householder factorization, computing for each $i$:
$$
\nabla_{v_i} L_Q = \left\langle \nabla_U L_Q, \frac{\partial}{\partial v_i}\left[H(v_k)\cdots H(v_i)\cdots H(v_1)\right] \right\rangle
$$
using the Frobenius inner product. The update for $v_i$ in each SGD iteration is:
$$
v_i \leftarrow v_i - \eta \nabla_{v_i} L_Q, \quad i = 1,\ldots,k
$$
where $\eta$ is the learning rate. These computations are efficiently supported by standard automatic differentiation frameworks.

## 5. Complete BHQ Algorithm and Computational Complexity

The following summary table outlines the procedural workflow:

| Stage            | Input/Action                                                                       | Output                         |
|------------------|------------------------------------------------------------------------------------|--------------------------------|
| I: Similarity    | Learn $f_\theta$ via SGD on $L_S(\theta)$ over data $x_i$, similarities $s_{ij}$   | Embedding $f_\theta(x)$        |
| II: Quantization | 1. Normalize: $\bar{f}_i = \sqrt{k} f_\theta(x_i)/\|f_\theta(x_i)\|_2$ <br>2. Initialize $v_1,...,v_k$ <br> 3. SGD on $L_Q$ <br> 4. Obtain $U^*=H(v_k)...H(v_1)$ | Orthogonal map $U^*$            |
| Hashing          | For any $x$, compute $h(x)=\mathrm{sign}(U^* f_\theta(x))$                       | Binary code $h(x)$             |

The primary computational costs are:  
- Application of a reflector to a vector: $O(k)$.  
- Application of $k$ reflectors (entire $U$): $O(k^2)$.  
- An epoch over $n$ samples: $O(nk^2)$.  
- Empirically for $k \leq 64$, the full quantization stage with 300 epochs and $n \approx 2 \times 10^4$ completes in under 3 minutes on a modern CPU; encoding $10^6$ points requires $< 0.3$ seconds [2311.04207].

## 6. Empirical Performance and Comparative Robustness

Unlike standard quantization-penalty approaches—such as those that jointly optimize $L_S(\theta) + \lambda L_Q(\theta)$ and require manual tuning of $\lambda$ to prevent embedding degradation—BHQ is consistently non-degrading. Experiments report that, across datasets including CIFAR-10, NUS-WIDE, MS-COCO, and ImageNet with 16–64 bit hashes, BHQ improves mean Average Precision (mAP) by 2–7 percentage points over state-of-the-art losses (HyP$^2$, DCH, DHN, DPSH, etc.), and does so uniformly over all tested baseline methods [2311.04207].

Baselines such as Iterative Quantization (ITQ) and Wasserstein-based quantization (HWSD) may degrade performance and generally require additional hyperparameters. In contrast, BHQ is hyperparameter-free and consistently provides positive accuracy gains.

Ablation over multiple rounding losses ($\ell_2$, $\ell_1$, min-entry, bit-variance) reveals that the standard $\ell_2$ loss is essentially optimal, and the particular loss is much less significant than the orthogonal map parametrization.

## 7. Significance and Practical Implications

BHQ exploits the invariance of inner products (and hence similarity measures) under orthogonal transformations, allowing for optimal rounding of embeddings without loss of similarity structure. The method's decoupling of similarity learning and quantization leads to a plug-in quantizer that can be applied atop any pretrained deep embedding network—eliminating performance trade-offs associated with joint objectives, and obviating hyperparameters and costly tuning.

A plausible implication is the potential for extending BHQ to other tasks requiring orthogonally invariant quantization or binary representations, given its modularity, computational efficiency, and robust empirical advantages [2311.04207].

Source: https://www.emergentmind.com/topics/block-householder-quantizer-bhq