Mini-vec2vec: Linear Embedding Alignment

• Mini-vec2vec is an unsupervised method that aligns incompatible text embedding spaces using a linear orthogonal map while preserving semantic geometry.
• It uses a multi-stage process including tentative matching, Procrustes transformation, and iterative refinements to achieve efficient alignment.
• The method reduces computational load by replacing adversarial training with efficient linear transformations and clustering-based corrections.

Mini-vec2vec is an unsupervised method for aligning one text embedding space to another when no parallel data are available. It is proposed as a simpler, faster, and more stable alternative to vec2vec, replacing adversarial alignment with a linear transformation learned through three stages: tentative matching of pseudo-parallel embedding vectors, transformation fitting, and iterative refinement. In the reported setting, the two datasets are sampled from the same underlying distribution but have zero overlap, and the goal is to learn a map $f$ such that $f(E_A(\mathbf z)) \approx E_B(\mathbf z)$. The method is explicitly linear, CPU-feasible, and intended to scale universal-geometry alignment without the instability associated with the earlier CycleGAN-style formulation (Dar, 27 Sep 2025).

1. Problem formulation and conceptual basis

Mini-vec2vec addresses unsupervised text embedding alignment in the case where two embedding models $E_A$ and $E_B$ map sentences into incompatible vector spaces, and no sentence pairs are available. The formal setup uses two embedding sets,

$$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$

with

$$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$

The two sentence sets are sampled from the same distribution, but are disjoint. Mini-vec2vec restricts the translator to a linear map,

$f(\mathbf x)=\mathbf x \mathbf W,$

with $\mathbf W$ primarily treated as orthogonal: $$\mathbf{W}\mathbf{W}^T = \mathbf{W}^T\mathbf{W} = \mathbf{I}.$$ This reflects the paper’s assumption that semantic geometry should be preserved under alignment (Dar, 27 Sep 2025).

The method is grounded in the Universal Geometry or Platonic Representation Hypothesis. In the earlier vec2vec formulation, this hypothesis motivated a nonlinear adversarial system with model-specific adapters, a shared latent backbone, adversarial losses at latent and output levels, reconstruction, cycle consistency, and vector-space preservation. That earlier system demonstrated that unsupervised embedding-to-embedding translation could work across model pairs, but it was computationally heavy and unstable. Mini-vec2vec keeps the same target—alignment without parallel data—but replaces the adversarial architecture with a geometric pipeline based on pseudo-matching and orthogonal updates (Jha et al., 18 May 2025).

A central conceptual ingredient is the use of relative representations. Given anchors $A=\{\mathbf a_i\}_{i=1}^k$ and embedding function $f(E_A(\mathbf z)) \approx E_B(\mathbf z)$0, the relative representation of $f(E_A(\mathbf z)) \approx E_B(\mathbf z)$1 is

$f(E_A(\mathbf z)) \approx E_B(\mathbf z)$2

Under the universal-geometry view, if two models represent the same underlying object similarly in relational terms, then

$f(E_A(\mathbf z)) \approx E_B(\mathbf z)$3

Mini-vec2vec operationalizes this idea by discovering pseudo-anchors through matched clusters rather than using known anchor pairs (Dar, 27 Sep 2025).

2. Preprocessing and geometric normalization

The method begins by centering and normalizing both embedding spaces onto the unit sphere. For space $f(E_A(\mathbf z)) \approx E_B(\mathbf z)$4, the paper writes

$f(E_A(\mathbf z)) \approx E_B(\mathbf z)$5

followed by centering and row-wise normalization; the same is done for $f(E_A(\mathbf z)) \approx E_B(\mathbf z)$6. The intended procedure is: compute the mean, subtract it from all vectors, then normalize each row to unit norm. The authors note that removing the mean changes cosine values noticeably because the mean accounts for roughly $f(E_A(\mathbf z)) \approx E_B(\mathbf z)$7–$f(E_A(\mathbf z)) \approx E_B(\mathbf z)$8 of vector norm (Dar, 27 Sep 2025).

This normalization is not a cosmetic preprocessing step. It enforces a geometry in which cosine-based comparisons and orthogonal alignment are meaningful. A plausible implication is that mini-vec2vec treats cross-model alignment less as arbitrary regression and more as recovery of a shared angular structure. That interpretation is consistent with its reliance on cosine similarities among centroids, cosine-based relative representations, and Procrustes updates rather than unrestricted affine fitting.

The paper’s geometric assumptions are explicit. Success depends on shared underlying geometry, stable recurring landmarks, pairwise centroid-similarity structure distinctive enough for quadratic assignment, and averaged nearest neighbors being meaningful despite the absence of exact overlap. These assumptions delimit the regime in which a linear near-isometric map is expected to suffice (Dar, 27 Sep 2025).

3. Tentative matching and pseudo-parallel construction

The first substantive stage is tentative matching of pseudo-parallel vectors. Because there are no paired sentences, the method first constructs shared landmarks by clustering the two spaces independently. K-means is run in both spaces, producing centroids $f(E_A(\mathbf z)) \approx E_B(\mathbf z)$9 and $E_A$0. Their internal similarity structures are then compared through centroid similarity matrices,

$E_A$1

A Quadratic Assignment Problem is solved to find the permutation $E_A$2 that best aligns the two centroid sets: $E_A$3 Implementation uses SciPy’s 2-OPT solver, rerun 30 times, retaining the best solution (Dar, 27 Sep 2025).

Matched centroids then define a relational coordinate system. For each point $E_A$4,

$E_A$5

and for $E_A$6, using the matched ordering,

$E_A$7

To stabilize this stage, the paper ensembles multiple anchor-discovery runs and concatenates the resulting signatures: $E_A$8 and similarly for $E_A$9. The default uses $E_B$0 runs and $E_B$1 clusters. Nearest-neighbor search is then performed in this shared relative space. For each source vector, the method averages the original target-space embeddings of its $E_B$2 nearest neighbors, yielding the pseudo-parallel set

$E_B$3

The use of multiple neighbors rather than a single nearest neighbor is important because there is zero overlap between the sentence sets; exact one-to-one matches are not expected (Dar, 27 Sep 2025).

4. Linear transformation fitting and iterative refinement

Given pseudo-parallel pairs, mini-vec2vec fits an optimal orthogonal transformation by Procrustes analysis. The paper states: $E_B$4 where

$E_B$5

with $E_B$6 and $E_B$7 formed from the pseudo-parallel pairs. This is the method’s initial global alignment (Dar, 27 Sep 2025).

Refinement proceeds in two stages. Refine-1 is ICP-style matching-based refinement. A subsample of $E_B$8 is transformed by the current map, nearest neighbors are found in $E_B$9, target embeddings are averaged, a new orthogonal Procrustes map is estimated, and the current map is updated by exponential smoothing: $$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$0 The paper uses $$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$1–$$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$2 iterations and reports that $$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$3 iterations were enough by a large margin. Each newly estimated $$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$4 is orthogonal, but the smoothed average is only approximately orthogonal. The stated purpose of the smoothing is improved stability and robustness (Dar, 27 Sep 2025).

Refine-2 is clustering-based refinement. The source space is clustered into $$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$5 centroids $$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$6. These centroids are mapped into the target space using the current $$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$7, and k-means is run on $$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$8 initialized with the transformed centroids. A new Procrustes update is then estimated from $$\mathcal{X}_A = \{\mathbf{x}^A_i\}_{i=1}^{n_A}, \qquad \mathcal{X}_B = \{\mathbf{x}^B_j\}_{j=1}^{n_B},$$9 to the adjusted target centroids $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$0, followed by the same exponential smoothing update. Empirically, one iteration of Refine-2 helps after Refine-1, while two or more begin to hurt slightly (Dar, 27 Sep 2025).

The full pipeline is therefore: normalize, construct anchor-based pseudo-matches, fit an initial orthogonal map, refine by repeated nearest-neighbor averaging, and apply one clustering-based correction. The paper presents this as an interpretable replacement for adversarial training rather than as a distillation of the earlier vec2vec architecture.

5. Empirical behavior, efficiency, and robustness

Experiments use Natural Questions. A random subset of size $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$1 sentences is embedded by all encoders; $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$2 sentences are held out for evaluation; the remaining embeddings are split equally into source and target sets, with no overlap between source and target sentences. The evaluated text embedding models are gtr, e5, gte, stella, and granite. Implementation uses scikit-learn for k-means, SciPy 2-OPT for QAP, and Colab CPU runtime; each experiment is repeated 3 times. Default hyperparameters are $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$3, $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$4, $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$5, $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$6, $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$7, $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$8, and $$\mathbf X_A \in \mathbb R^{n_A \times d}, \qquad \mathbf X_B \in \mathbb R^{n_B \times d}.$$9. For $f(\mathbf x)=\mathbf x \mathbf W,$0, the number of clusters in approximate matching is increased to $f(\mathbf x)=\mathbf x \mathbf W,$1 (Dar, 27 Sep 2025).

The paper’s headline efficiency claim is that mini-vec2vec runs in less than ten minutes on a CPU, whereas vec2vec requires 1–7 days on a GPU. It also reports a marked reduction in data requirements, using $f(\mathbf x)=\mathbf x \mathbf W,$2 samples instead of $f(\mathbf x)=\mathbf x \mathbf W,$3 million. This is presented not merely as acceleration, but as a change in optimization regime: k-means, small QAPs, nearest-neighbor search, and closed-form Procrustes updates replace GAN training (Dar, 27 Sep 2025).

Representative alignment results are shown below.

Model pair	vec2vec	mini-vec2vec
granite $f(\mathbf x)=\mathbf x \mathbf W,$4 gte	Top-1 0.95, Rank 1.18	Top-1 0.98, Rank 1.05
gtr $f(\mathbf x)=\mathbf x \mathbf W,$5 e5	Top-1 0.84, Rank 2.88	Top-1 0.98, Rank 1.06
gte $f(\mathbf x)=\mathbf x \mathbf W,$6 gtr	Top-1 0.91, Rank 2.64	Top-1 0.98, Rank 1.04
e5 $f(\mathbf x)=\mathbf x \mathbf W,$7 gtr	Top-1 0.82, Rank 2.56	Top-1 0.96, Rank 1.10
stella $f(\mathbf x)=\mathbf x \mathbf W,$8 gtr	Top-1 1.00, Rank 1.10	Top-1 0.96, Rank 1.10

The paper also reports cosine similarity after each stage. Initial alignment often begins around $f(\mathbf x)=\mathbf x \mathbf W,$9–$\mathbf W$0, Refine-1 raises it to around $\mathbf W$1–$\mathbf W$2, and Refine-2 provides a smaller additional increase. Standard deviations over 3 runs are described as very small for nearly all model pairs. Reported failure modes are extremely rare; when they occur, they are described as around $\mathbf W$3 top-1 and average rank around $\mathbf W$4–$\mathbf W$5, still substantially better than naive baselines (Dar, 27 Sep 2025).

6. Relation to adjacent vec2vec formulations and stated limitations

Mini-vec2vec belongs to a broader family of embedding-to-embedding methods, but it is distinct from both the original vec2vec and other uses of the Vec2Vec name. The earlier vec2vec framework aligned text embeddings without paired data by learning model-specific adapters into and out of a shared latent space, using adversarial losses at latent and output levels together with reconstruction, cycle consistency, and vector-space preservation. That method handled even dimensional mismatch through latent adapters, but its own ablations showed that latent GAN, cycle consistency, and vector-space preservation were all essential, and its training cost was substantial (Jha et al., 18 May 2025).

By contrast, the supervised paper titled “Vec2Vec” learned a compact neural network mapping $\mathbf W$6-dimensional MPNet embeddings to $\mathbf W$7-dimensional text-ada-002 embeddings from paired data. On $\mathbf W$8 unseen reviews, it reported average cosine similarity $\mathbf W$9, but its setting is fully supervised and domain-specific, using $$\mathbf{W}\mathbf{W}^T = \mathbf{W}^T\mathbf{W} = \mathbf{I}.$$0 paired food-review embeddings rather than zero-overlap samples from the same distribution (Gao, 2023). Elsewhere, “Vec2vec” has also denoted a local similarity-preserving dimensionality-reduction method that constructs contexts by random walks on a similarity graph and learns embeddings with a one-hidden-layer skip-gram-like model; that usage is unrelated to universal-geometry alignment between pretrained text encoders (Wang et al., 2021).

The limitations of mini-vec2vec are explicitly tied to its assumptions. It assumes a roughly linear or orthogonal relationship, shared underlying geometry, stable recurring landmarks, and meaningful nearest-neighbor averaging in the absence of true overlap. It is designed for non-overlapping but same-distribution text data, and the paper states that extension to other domains is plausible but not fully established there. This suggests that mini-vec2vec is best understood not as a universal substitute for nonlinear translation, but as a high-efficiency regime within unsupervised text-embedding alignment where a near-isometric map is empirically sufficient (Dar, 27 Sep 2025).