Post-Hoc Vector Concatenation

• Post-hoc vector concatenation is a method that fuses outputs from multiple frozen embedding models into a single high-dimensional vector for unified representation.
• It employs a unified decoder, typically a single-layer MLP, to compress the concatenated vector while preserving its relational cosine geometry using Matryoshka loss.
• The approach enhances memory efficiency and inference speed with robust spectral bounds and clustering techniques, achieving competitive retrieval performance even under aggressive quantization.

Post-hoc vector concatenation is a formal strategy for combining the output representations of multiple embedding models or feature generators into a single, higher-dimensional vector, with subsequent compression to a tractable dimension as required by storage, computational, or inference constraints. Distinguished from architectural fusion, post-hoc concatenation operates after all base transformations are “frozen” or finalized, enabling the aggregation and compression of heterogeneous features with mathematical rigor. Modern systems principally leverage this method to realize efficient, robust embeddings for tasks in retrieval, semantic search, recommendation, and multimodal learning.

1. Formal Definition and Mathematical Structure

Let $S_1, S_2, \dots, S_k$ denote $k$ frozen embedding models, each producing a $d_i$-dimensional vector representation for input $x$:

$e_i(x) \in \mathbb{R}^{d_i}, \quad i=1, \dots, k.$

The post-hoc vector concatenation forms the joint high-dimensional embedding:

$$\mathbf{e}_{\text{joint}}(x) = [ e_1(x);\, e_2(x);\, \dots;\, e_k(x) ] \in \mathbb{R}^D, \quad D = \sum_{i=1}^k d_i.$$

For a dataset with $N$ data points, the concatenated feature matrix $C$ is:

$$C = [ E_1\,\,\|\,\,E_2\,\,\|\,\,\dots\,\|\,\,E_k ] \in \mathbb{R}^{N \times D},$$

where $E_i \in \mathbb{R}^{N \times d_i}$ are embeddings from $S_i$. This operation is architecturally agnostic and can be performed after any set of models have been trained, provided their output dimensionalities and types are compatible for horizontal stacking.

2. Unified Decoder and Matryoshka Compression

High-dimensional concatenated vectors ($D$ large) impede efficient inference and storage, motivating architectural compression. The unified decoder $h : \mathbb{R}^D \rightarrow \mathbb{R}^d$ is typically realized as a single-layer MLP (linear transformation):

$$h(\mathbf{z}) = W\mathbf{z} + \mathbf{b}, \quad W \in \mathbb{R}^{d \times D},\; \mathbf{b} \in \mathbb{R}^d,$$

with $d \ll D$ (in practice, $d=768$). Notably, deeper decoders are observed to overfit, and thus a single linear projection suffices for effective geometry preservation (Ayad et al., 6 Oct 2025). The parameter count is $D \cdot d$, e.g., for two 384-dimensional inputs, $D=768$, $d=768$, yielding $0.9$M parameters.

To ensure the compressed output retains the high-dimensional relational structure, Matryoshka Representation Learning (MRL) loss is employed. Defining a batch $Z \in \mathbb{R}^{B \times D}$ and the decoded $H = h(Z) \in \mathbb{R}^{B \times d}$, the loss at each “stop” $d^{(k)}$ is:

$$\ell_{\mathrm{sim}}(H,Z) = \frac{1}{B(B-1)} \sum_{i \neq j} \left[ \cos(h_i, h_j) - \cos(z_i, z_j) \right]^2,$$

with the overall objective averaging over a pre-specified set of stops $\mathcal{D}=\{d^{(1)},\dots,d^{(K)}\}$, e.g., $\{32,64,128,200,256,300,384,512,768\}$. This incentivizes each prefix of the decoder output to approximate the pairwise cosine geometry of the raw concatenation (Ayad et al., 6 Oct 2025).

3. Practical Pipeline: Compression and Quantization

After decoder application, offline-to-online scalar quantization is performed for further dimensionality and size reduction. Calibration on a reference set ($S=100{,}000$ samples) yields per-dimension percentiles:

$$\tau_{j,k} = \text{Percentile}(H_{\text{ref},:,j}, 100k/2^b),\quad k=1,\ldots,2^b$$

for each coordinate $j$. During inference, each new value $H_{i,j}'$ is quantized:

$$q_{i,j} = \sum_{k=1}^{2^b-1} [ H_{i,j}' > \tau_{j,k} ],\quad q_{i,j} \in \{0, \dots, 2^b-1\}$$

Setting $b=1$ to $4$ yields high compression factors (e.g., $48\times$ for 1-bit quantization to 1024 dimensions), while maintaining a substantial proportion of the original ranking performance. For instance, on MTEB tasks, the compressed pipeline with 1-bit quantized decoder recovers approximately 89% of the uncompressed concat performance (nDCG@10 $0.4725$ vs. $0.5076$) (Ayad et al., 6 Oct 2025).

4. Spectral Bounds and Compression Guarantees for Concatenation

The spectrum of the concatenated matrix governs compression fidelity. Global spectral error bounds use Weyl-type monotonicity, ensuring that adding blocks increases (or preserves) the leading singular values. For $K$ blocks $A_1,\dots,A_K$:

$$E_r^2\bigl([A_1,\dots,A_K]\bigr) \le \sum_{j=1}^K\|A_j\|_F^2 - \max_{1 \le j \le K} \sum_{i=1}^r \sigma_i^2(A_j)$$

where $E_r^2(M)$ is the rank-$r$ SVD reconstruction error. Residual-based error bounds further tighten this guarantee by tracking the orthogonal new directions contributed by each block:

$$E_r^2(M_K) \le \sum_{t=1}^K\|A_t\|_F^2 - \sum_{j=1}^r\mu_j^2$$

Here, $\mu_j$ are the leading singular values of the stacked residual matrix, and $M_K = [A_1, \dots, A_K]$ (Shamrai, 12 Jan 2026).

Incremental approximations, via basis-plus-Gram adaptation, allow for efficient estimation of singular values as concatenation proceeds, supporting scalable control over the trade-off between model fusion and reconstruction accuracy.

5. Compression-Aware Clustering Algorithms

Compression-aware clustering determines which vectors (or blocks) to merge without breaching a user-specified error threshold $\varepsilon$, post-concatenation. Three major strategies are formalized (Shamrai, 12 Jan 2026):

• Max-norm clustering: anchors on the largest-norm block, aggregates smaller ones using the (loose) global norm bound. Operates in $O(K\log K)$ time.
• Residual-based clustering: incrementally adds new vectors checking residual norm bounds (tight, but $O(Kdr)$).
• Approximate incremental clustering: leverages plug-in SVD error estimates for rapid, heuristic aggregation; no strict guarantee, but typically suffices in practice.

Applied to single vectors, the max-norm bound between $a$ and $b$ yields:

$E_1^2([a,b]) \le \min\{\|a\|_2^2, \|b\|_2^2\}$

Optimal grouping obeys $$\sum\|v_i\|_2^2 - \sum_{j=1}^r\mu_j^2 \le \varepsilon^2 \sum\|v_i\|_2^2$$ over cluster $C$.

6. Empirical Results and Observations

Empirical evaluations establish that post-hoc concatenation can outperform individual baselines across retrieval tasks:

Model	Avg. nDCG@10
e5-small (33M)	0.4431
No-Ins (33M)	0.5056
bge-small (33M)	0.4909
Arctic-m (109M)	0.5072
[Arctic-m, bge-small] raw (142M)	0.5203

Compression via the unified decoder at 384 dimensions reduces dimensionality with virtually no loss:

Representation	Avg. nDCG@10
[Arctic-m, bge-small] raw (142M, 768-d)	0.5203
Decoder([Arctic-m, bge-small]):384	0.5178

Extreme compression (LSH 1-bit, 1024-d, $48\times$) leads to marginal drops while retaining most utility:

Method	Avg. nDCG@10
[e5-small, No-Ins, gte-small, bge-small], 1536-d	0.5076
LSH$_{1024}$ (1-bit, 1024-d, $48\times$)	0.4725

These results support the conclusion that model concatenation, followed by carefully tuned unified decoding and quantization, achieves much of the performance of large baselines at a small fraction of model and storage footprint (Ayad et al., 6 Oct 2025).

7. Interpretations, Advantages, and Practical Considerations

The efficacy of post-hoc vector concatenation results from the aggregation of complementary knowledge across models trained on different domains or data regimes. Orthogonal axes of semantic similarity are combined; the Matryoshka compression objective ensures preservation of relational geometry even under aggressive dimension reduction. The method’s robustness to scalar quantization is attributed to the distributed nature of the compressed embedding, ameliorating quantization artifacts.

From a systems perspective, post-hoc vector concatenation is notable for:

• Decoupling model training/fusion from compression, enabling plug-and-play composition of heterogeneous models.
• High memory efficiency: e.g., four $33$M parameter models plus decoder $< 140$M, compared to a $335$M monolith.
• Edge readiness: minimal inference overhead, no need for fine-tuning backbones, and quantization reduces communication/compute costs.
• Explicit spectral control: principled, tunable error guarantees via spectral bounds and clustering, applicable in large-scale or federated deployments (Shamrai, 12 Jan 2026).

A plausible implication is the widespread adoption of this method for decentralized or cross-modality scenarios where post-training fusion, not end-to-end retraining, is the normative constraint. The flexible grouping and error analysis also suggest its utility for storage-reduction in scientific computing and multi-view learning contexts.