Paired Point Encoding Techniques

• Paired point encoding is a technique that encodes data in pairs, leveraging inter-point dependencies to minimize distortions in pairwise computations such as scalar products and Euclidean distances.
• It employs data-driven linear transformations combined with quantization methods like PQ or OPQ, achieving up to 40–50% reduction in distortion for similarity search tasks.
• In source coding, paired point techniques enable joint encoding for two-dimensional geometric distributions, significantly reducing redundancy compared to traditional symbol-wise methods.

Paired point encoding refers to the class of encoding and quantization strategies that operate not on individual data points or symbols, but on pairs. In both source coding for integer-valued symbols under geometric distributions and lossy compression of high-dimensional vectors, paired point techniques systematically exploit inter-point or inter-symbol relationships to minimize distortions in downstream pairwise computations (e.g., scalar products, Euclidean distances) or reduce coding redundancy compared to symbol-wise approaches. This methodology appears across several domains, notably in optimal prefix code construction for two-dimensional geometric distributions and in advanced quantization pipelines for large-scale similarity search.

1. Principles of Pairwise Quantization and Paired Coding

Paired point encoding in the context of high-dimensional vector quantization, as instantiated in Pairwise Quantization (PairQ), modifies standard quantization objectives from per-point reconstruction fidelity to direct optimization of the pairwise quantities of interest. The core aim is to minimize the distortion in scalar products,

$$D_{\text{sp}} = \mathbb{E}_{i,j}\Big[\big(q_i^\top x_j - q_i^\top \hat{x}_j\big)^2\Big]$$

and in squared Euclidean distances,

$$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$

between compressed and/or uncompressed vectors, as opposed to the classical per-point reconstruction loss $\sum_j \|x_j - \hat{x}_j\|^2$ (Babenko et al., 2016).

In prefix coding for discrete sources, paired coding encodes jointly distributed pairs (e.g., $(X,Y)$ sampled independently from a geometric law), taking advantage of joint statistics to reduce redundancy over symbol-by-symbol codes. The average codeword length for pairs,

$$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$

is compared to the sum of individual code lengths, with reductions in redundancy possible through careful blocking and code construction (Bassino et al., 2011).

2. Linear Transformations and Reduction to Pointwise Problems

Pairwise Quantization achieves its objective via a data-driven linear transformation that maps original vectors into a latent space in which standard pointwise quantization aligns with pairwise preservation goals. In the scalar product case, given dataset vectors $\{x_j\}$ and queries $\{q_i\}$, one estimates the $d \times d$ positive semidefinite matrix $G = \sum_{i=1}^M q_i q_i^\top$ and its matrix square root $C$ (via SVD or Cholesky factorization, such that $$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$0). The transformed vectors are

$$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$1

In this space, quantization minimizing $$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$2 is equivalent to minimizing original pairwise scalar-product distortion (Babenko et al., 2016).

For squared-distance objectives, data vectors $$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$3 are augmented as $$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$4 and queries as $$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$5, leading to a similar linearization approach via $$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$6, its square root $$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$7, and the mapping $$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$8. This canonical reduction translates pairwise distortion minimization to an $$D_{\text{d}} = \mathbb{E}_{i,j}\Big[\big(\|q_i - x_j\|^2 - \|q_i - \hat{x}_j\|^2\big)^2\Big],$$9-reconstruction problem on transformed data.

3. Quantizer Integration and Practical Workflow

Following the pairwise-aware transformation, any standard quantizer—such as Product Quantization (PQ) or Optimized PQ (OPQ)—can be employed on the transformed points. The workflow consists of:

1. Matrix estimation and transformation: Compute $\sum_j \|x_j - \hat{x}_j\|^2$0 (or $\sum_j \|x_j - \hat{x}_j\|^2$1) and form $\sum_j \|x_j - \hat{x}_j\|^2$2 (or $\sum_j \|x_j - \hat{x}_j\|^2$3).
2. Quantizer training in the latent space: Use PQ/OPQ to minimize codebook reconstruction error $\sum_j \|x_j - \hat{x}_j\|^2$4.
3. Indexing and querying: At index time, store compressed codes; at query time, use the appropriate transformed query (e.g., $\sum_j \|x_j - \hat{x}_j\|^2$5 for scalar products), leveraging fast lookup table computation for retrieval.

At inference, pairwise approximations are obtained via the formulas:

• Scalar product: $\sum_j \|x_j - \hat{x}_j\|^2$6,
• Squared distance: $\sum_j \|x_j - \hat{x}_j\|^2$7, but are typically realized via quantizer routines on $\sum_j \|x_j - \hat{x}_j\|^2$8-space (Babenko et al., 2016).

4. Paired Point Prefix Coding for Discrete Geometric Sources

In source coding, paired point (block-of-two) coding for two-dimensional geometric distributions (TDGD) employs joint encoding of symbol pairs $\sum_j \|x_j - \hat{x}_j\|^2$9, for $(X,Y)$0. The objective is to design binary prefix codes $(X,Y)$1 with codeword lengths $(X,Y)$2 minimizing mean codeword length $(X,Y)$3, with entropy given by

$(X,Y)$4

Compared to 1D Golomb coding (with redundancy $(X,Y)$5 per symbol), block coding can yield redundancy $(X,Y)$6 (Bassino et al., 2011).

Notably, optimal codes are "parameter-singular": a prefix code optimal for one $(X,Y)$7 is not optimal for any other, precluding broad universality and requiring characterization at discrete $(X,Y)$8 sequences.

Two code families are prominent:

• Family 1 ($(X,Y)$9, $$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$0): Coding uses a small $$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$1 "top code" $$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$2 (Huffman tree on alphabet $$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$3 with weights $$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$4), concatenated with unary codes for $$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$5, $$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$6.
• Family 2 ($$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$7, $$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$8): Construction uses "L-trees" with infinite support, cyclically layering codewords by signature $$\mathbb{E}[L] = \sum_{i,j \geq 0} P[X = i, Y = j] L(i,j),$$9; codeword lengths $\{x_j\}$0 given in closed form.

As $\{x_j\}$1 ($\{x_j\}$2), the codes converge to a unique "limit tree" $\{x_j\}$3, optimal for the dyadic limit (Bassino et al., 2011).

5. Redundancy Analysis and Computational Complexity

For block-of-two codes on TDGD($\{x_j\}$4):

• For Family 1 ($\{x_j\}$5), the redundancy oscillates for large $\{x_j\}$6, with

$\{x_j\}$7

compared to $\{x_j\}$8–$\{x_j\}$9 for Golomb codes (Bassino et al., 2011).

• Complexity per pair matches symbol-wise Golomb coding: two unary encodings and a single table lookup (Family 1), or fixed prefix and suffix composition (Family 2), both supporting $\{q_i\}$0 coding per symbol pair.
• Empirical curves (Fig. 8–9, (Bassino et al., 2011)) demonstrate up to $\{q_i\}$1 reduction in redundancy over symbol-wise coding for appropriate $\{q_i\}$2.

Pairwise Quantization exhibits reductions in scalar-product or distance distortion of $\{q_i\}$3–$\{q_i\}$4 over state-of-the-art (e.g., OPQ), matching OPQ’s compression ratio and runtime (Babenko et al., 2016). For example, on SIFT1M, 4 bytes/vector under PairQ match the accuracy of 16 bytes/vector under OPQ.

6. Applications and Significance

Paired point encoding is widely applicable in large-scale retrieval, recommendation, and any computation requiring fast, accurate approximate pairwise relations. In high-dimensional settings, Pairwise Quantization preserves scalar products and distances with high fidelity in memory- and computation-constrained scenarios, directly aligning quantizer training with the downstream objectives of ranking or similarity search.

In source coding, block-of-two prefix codes for TDGD($\{q_i\}$5) provide substantial redundancy minimization while retaining practical encoding/decoding complexity, making them suited for efficient lossless coding of quantized transform coefficients or similar data exhibiting geometric statistics. The parameter-singularity implies that code design must carefully match the source $\{q_i\}$6, unlike classical Golomb codes.

A plausible implication is that, by extending the strategy of transforming loss objectives into forms compatible with established encoding or quantization machinery, analogous paired point schemes could be developed for higher-order blocks or more complex dependency structures, subject to computational tractability.

7. Summary Table: Paired Point Encoding Variants

Domain	Core Approach	Key Benefit
High-dim. Quantization	Linear transform + vector quantizer	Minimizes pairwise distortion (Babenko et al., 2016)
Geometric Prefix Codes	Huffman tree/block code for pairs	Reduces redundancy (Bassino et al., 2011)

Both modalities demonstrate that paired point encodings, by adapting coding objectives or codebook construction to the preservation of pairwise information, achieve superior tradeoffs in distortion or redundancy compared to traditional symbol-wise approaches.