Block-Householder Quantizer (BHQ)

• Block-Householder Quantizer is a hyperparameter-free method that decouples similarity learning from binary quantization through a two-stage approach.
• It employs Householder reflectors to form an optimal orthogonal transformation, aligning continuous embeddings with their binary counterparts.
• Empirical results show improved retrieval performance and reduced tuning complexity across benchmarks like CIFAR-10, MS-COCO, and ImageNet.

The Block-Householder Quantizer (BHQ) is an unsupervised, hyperparameter-free quantization strategy for deep image hashing that decouples the learning of similarity-preserving embeddings from the subsequent binarization step. Unlike standard approaches that integrate a quantization penalty into the similarity loss, introducing complex trade-offs and challenging optimization, BHQ decomposes the problem into two separate stages: first, learning continuous embeddings using only similarity-based loss, and second, finding an optimal orthogonal transformation—parametrized as a product of Householder reflectors—to align the embeddings with their binary counterparts before quantization by the sign function. Empirical evidence demonstrates that this separation mitigates performance degradation, enables consistently stronger retrieval results, and requires less tuning compared to conventional methods (Schwengber et al., 2023).

1. Two-Stage Decomposition and Formulation

BHQ operates via a structured two-phase procedure:

1. Similarity Learning Stage: A standard deep embedding $f_\theta : \mathbb{R}^d \rightarrow \mathbb{R}^k$ is learned by minimizing a similarity-driven objective

$$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$

with no quantization term. Here, $s_{ij} \in \{0,1\}$ encodes pairwise similarity, $\ell_S$ and $\ell_D$ represent the chosen similarity and dissimilarity losses. This stage uses frameworks such as contrastive, cross-entropy, or proxy losses.

2. Householder Quantization Stage: The learned embeddings $f_\theta$ are normalized for each sample $x_i$ to $$\bar{f}_i = \sqrt{k} f_\theta(x_i)/\|f_\theta(x_i)\|_2$$ (on the $\sqrt{k}$-radius sphere). Then, one seeks an orthogonal matrix $U \in O(k)$ that makes each $$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$0 as close as possible to its sign vector. The resulting $$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$1-bit hash is $$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$2.

This formulation leverages similarity-invariant properties of orthogonal transformations, so the choice of $$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$3 does not affect the preservation of pairwise similarities (Schwengber et al., 2023).

2. Quantization Objective and Optimization

The core rounding problem formulated in the quantization stage is:

$$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$4

where the objective $$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$5 quantifies the mean-squared error between the transformed embeddings and their sign vectors.

This least-squares problem over the orthogonal group $$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$6 is solved efficiently by parameterizing $$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$7 via Householder reflections and minimizing $$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$8 using stochastic gradient descent (SGD).

3. Householder Parametrization of Orthogonal Transform

Any orthogonal matrix $$L_S(\theta) = \sum_{i,j} s_{ij} \ell_S(\tilde{d}_{ij}(\theta)) + (1-s_{ij}) \ell_D(\tilde{d}_{ij}(\theta))$$9 can be written as a product of $s_{ij} \in \{0,1\}$0 Householder reflectors:

$s_{ij} \in \{0,1\}$1

where each Householder reflection $s_{ij} \in \{0,1\}$2 has the form

$s_{ij} \in \{0,1\}$3

Application of $s_{ij} \in \{0,1\}$4 to a vector $s_{ij} \in \{0,1\}$5 is computed via $s_{ij} \in \{0,1\}$6, requiring only $s_{ij} \in \{0,1\}$7 operations per reflection. The full map requires $s_{ij} \in \{0,1\}$8 time to apply $s_{ij} \in \{0,1\}$9 reflectors.

This parametrization enables direct optimization of the $\ell_S$0 free parameters $\ell_S$1 and ensures storage costs remain low (at most $\ell_S$2 floats, i.e., $\ell_S$3 kB for $\ell_S$4) (Schwengber et al., 2023).

4. Stochastic Gradient Descent Updates

With $\ell_S$5, the quantization loss is $\ell_S$6. Gradients with respect to $\ell_S$7 are computed as:

$\ell_S$8

Backpropagation proceeds through the Householder factorization, computing for each $\ell_S$9:

$\ell_D$0

using the Frobenius inner product. The update for $\ell_D$1 in each SGD iteration is:

$\ell_D$2

where $\ell_D$3 is the learning rate. These computations are efficiently supported by standard automatic differentiation frameworks.

5. Complete BHQ Algorithm and Computational Complexity

The following summary table outlines the procedural workflow:

Stage	Input/Action	Output
I: Similarity	Learn $\ell_D$4 via SGD on $\ell_D$5 over data $\ell_D$6, similarities $\ell_D$7	Embedding $\ell_D$8
II: Quantization	1. Normalize: $\ell_D$9 <br\>2. Initialize $f_\theta$0 <br> 3. SGD on $f_\theta$1 <br> 4. Obtain $f_\theta$2	Orthogonal map $f_\theta$3
Hashing	For any $f_\theta$4, compute $f_\theta$5	Binary code $f_\theta$6

The primary computational costs are:

• Application of a reflector to a vector: $f_\theta$7.
• Application of $f_\theta$8 reflectors (entire $f_\theta$9): $x_i$0.
• An epoch over $x_i$1 samples: $x_i$2.
• Empirically for $x_i$3, the full quantization stage with 300 epochs and $x_i$4 completes in under 3 minutes on a modern CPU; encoding $x_i$5 points requires $x_i$6 seconds (Schwengber et al., 2023).

6. Empirical Performance and Comparative Robustness

Unlike standard quantization-penalty approaches—such as those that jointly optimize $x_i$7 and require manual tuning of $x_i$8 to prevent embedding degradation—BHQ is consistently non-degrading. Experiments report that, across datasets including CIFAR-10, NUS-WIDE, MS-COCO, and ImageNet with 16–64 bit hashes, BHQ improves mean Average Precision (mAP) by 2–7 percentage points over state-of-the-art losses (HyP$x_i$9, DCH, DHN, DPSH, etc.), and does so uniformly over all tested baseline methods (Schwengber et al., 2023).

Baselines such as Iterative Quantization (ITQ) and Wasserstein-based quantization (HWSD) may degrade performance and generally require additional hyperparameters. In contrast, BHQ is hyperparameter-free and consistently provides positive accuracy gains.

Ablation over multiple rounding losses ($$\bar{f}_i = \sqrt{k} f_\theta(x_i)/\|f_\theta(x_i)\|_2$$0, $$\bar{f}_i = \sqrt{k} f_\theta(x_i)/\|f_\theta(x_i)\|_2$$1, min-entry, bit-variance) reveals that the standard $$\bar{f}_i = \sqrt{k} f_\theta(x_i)/\|f_\theta(x_i)\|_2$$2 loss is essentially optimal, and the particular loss is much less significant than the orthogonal map parametrization.

7. Significance and Practical Implications

BHQ exploits the invariance of inner products (and hence similarity measures) under orthogonal transformations, allowing for optimal rounding of embeddings without loss of similarity structure. The method's decoupling of similarity learning and quantization leads to a plug-in quantizer that can be applied atop any pretrained deep embedding network—eliminating performance trade-offs associated with joint objectives, and obviating hyperparameters and costly tuning.

A plausible implication is the potential for extending BHQ to other tasks requiring orthogonally invariant quantization or binary representations, given its modularity, computational efficiency, and robust empirical advantages (Schwengber et al., 2023).