Embedding-Based Sampling Methods

Updated 5 July 2026

Embedding-based sampling is a technique where sampling decisions are guided by explicit or implicit embeddings to improve data coverage and reduce bias.
It is applied in diverse domains such as manifold learning, point clouds, deep metric learning, graph representations, kernel approximations, and quantum annealing.
Practical implementations—like local geometric supersampling, distance-weighted techniques, and adaptive strategies in GNNs—demonstrate improved performance and convergence.

Embedding-based sampling denotes, in the broad sense suggested by recent literature, a family of methods in which sampling decisions are governed by an explicit embedding, an implicit feature map, or an embedding-induced geometry. Across manifold learning, point cloud processing, deep metric learning, graph representation learning, kernel approximation, synthetic data generation, and quantum annealing, the common operation is not merely to sample data points, but to sample with respect to a representation space and often to correct, reweight, or invert those samples afterward. In some settings this improves coverage of sparse regions or stabilizes downstream estimators; in others, especially under hardware minor embedding, it introduces severe sampling bias and logical-subspace suppression [2107.06566], [2104.14769], [1706.07567], [2007.03927], [2603.22294], [1909.12184].

1. Conceptual scope and recurring constructions

The literature covers several distinct meanings of “embedding.” In manifold supersampling, the embedding is a smooth map $f:\mathbb{R}^\delta \supset U \to \mathbb{R}^d$ whose image is the observed manifold [2107.06566]. In point cloud sampling and recovery, it is a locally invertible latent code $z$ that stores discarded local offsets [2104.14769]. In graph and metric learning, it is the learned representation itself, and sampling determines which pairs, walks, or negatives shape that representation [1706.07567], [2010.13023], [2501.12884]. In kernel methods, the embedding is the implicit feature map $\Phi(x)$, and the task is to sample coordinates of that lifted space efficiently [2007.03927]. In quantum annealing, “embedding” means graph minor embedding into hardware, and the central issue is whether thermal samples of the enlarged Hamiltonian preserve the target distribution [1909.12184], [2103.07036].

Setting	Embedding object	Sampling role
Manifold supersampling	Local covariance / Jacobian surrogate	Generate and correct virtual points
Point clouds	Invertible latent residual $z$	Downsample and reconstruct
Deep metric learning	Learned feature geometry	Select informative negatives or curricula
Graph embeddings	Walk, pair, or policy-defined contexts	Control structural signal
Kernel methods	Implicit feature map $\Phi(x)$	Sample feature coordinates
Quantum annealing	Minor-embedded hardware graph	Constrains valid logical samples

This suggests three recurring constructions. First, some methods sample inside an embedding space to densify underrepresented regions, as in local Gaussian perturbation on manifolds or sparse-cell targeting in synthetic data generation [2107.06566], [2603.22294]. Second, some methods reweight the training distribution induced by an embedding so that optimization is not dominated by easy, redundant, or high-frequency examples [1706.07567], [2501.12884]. Third, some methods must project or invert samples back from the embedded domain to the original domain, either constructively, as in PointLIE, or problematically, as in restricted resampling for embedded Ising models [2104.14769], [1909.12184].

2. Local geometric supersampling and invertible reconstruction

Manifold-Embedding Supersampling (MESS) is built on the manifold hypothesis: the high-dimensional dataset $X \subset \mathbb{R}^d$ consists of noisy observations of a lower-dimensional smooth manifold $M$ embedded by a smooth map $f$, so that each observed $x \in X$ corresponds approximately to a parameter vector $\tilde x$ with $f(\tilde x) \simeq x$ [2107.06566]. Because $f$ is generally non-linear, MESS replaces it locally by its Jacobian, estimated through the sample covariance
$$
\Sigma_x \coloneqq \frac{1}{k}\sum_{i=1}^k (x_i-x)(x_i-x)^T,
$$
which approximates $J_f(\tilde x)\,C\,J_f(\tilde x)^T$ in small neighborhoods. Raw virtual samples are drawn as
$$
p_{\text{raw}} = x + L_x z,\qquad z \sim \mathcal N(0,I_d),\qquad L_xL_x^T=\Sigma_x,
$$
so $p_{\text{raw}}\sim \mathcal N(x,\Sigma_x)$. These raw samples are then corrected by local chart maps $C_1$ or $C_2$ and aggregated using inverse-Mahalanobis weights
$$
w_u = (\Delta^{T\Sigma_u^{{-1}\Delta)^{-1/2},}}
$$
normalized to sum to one. The aim is an extended point set $X_{\text{corr}}$ whose density along $M$ is higher by a factor ext, while remaining near the manifold [2107.06566].

The theoretical intuition is explicitly local. In the infinitesimal limit, for a $\delta$-dimensional parameter ball of radius $\epsilon$,
$$
\operatorname{Cov}[f(B_\epsilon)] \to \frac{\epsilon^{2}{\delta+2}J_fJ_f^T.}
$$
Sampling ext points from $\mathcal N(x,\Sigma_x)$ yields locally ext-fold increased density along tangent directions, while chart correction removes orthogonal components introduced by curvature or noise. Under mild smoothness assumptions, the bias of each corrected sample is $O(\epsilon^2)$ when neighborhoods of radius $\epsilon$ are used and ext is finite. In practice, MESS reduces the required neighborhood radius for a given local sampling density by approximately a factor ext^{1/\delta} [2107.06566].

The empirical validation targets intrinsic-dimension estimation. On synthetic toy manifolds m1–m13, geometry-based estimators such as ABID and expansion-based estimators such as Hill stabilize with much smaller neighborhood sizes after supersampling. For m4 with $d=8,\delta=4$, ABID reaches median ID $4.05$ with IQR $0.35$ under MESS versus $4.37$ with IQR $0.58$ in the baseline; Hill reaches median ID $4.02$ with IQR $1.2$ under MESS versus $4.9$ with IQR $2.4$ in the baseline. On the Möbius strip m11, ABID error $|ID-2|$ is reduced by $\sim 50\%$ in median and the IQR is halved. On MNIST with $k=20$, baseline ABID gives median $\approx 3.8$ and IQR $\approx 1.6$, whereas MESS with ext=50,k1=20 gives median $\approx 3.9$ and IQR $\approx 0.6; 3D point-cloud scans show local ID variance reductions of up to40–60%` [2107.06566]. The paper also contrasts MESS with SMOTE, arguing that SMOTE’s linear interpolations do not respect non-convex local curvature and often introduce off-manifold samples, whereas MESS uses $\Sigma_x$ to preserve local tangent structure [2107.06566].

PointLIE addresses a related but distinct problem: point cloud sampling and recovery without storing per-point adjacency or side information [2104.14769]. Its core map is a single invertible network
$$
f_\theta:\hat{\mathcal Q}\longmapsto (\mathcal P,z),\qquad z\sim p(z),
$$
where $\hat{\mathcal Q}\subset\mathbb R^3$ is the dense cloud, $\mathcal P$ is the adaptively sampled sparse cloud of size $N/r$, and $z$ is a latent embedding that captures all lost local offsets. Running the same network backward yields
$$
f_\theta^{{-1}:(\mathcal} P,z^{\star)\longmapsto} \mathcal Q,\qquad z^\star\sim p(z),
$$
thereby recovering a dense cloud $\mathcal Q\approx\hat{\mathcal Q}$ [2104.14769].

The architecture is recursive and multiscale. Each Rescale Layer halves the point count via Farthest-Point Sampling, forms local offset tensors from discarded neighbors, and processes the pair of branches through stacked Point-Invertible blocks:
$$
\begin{aligned}
\mathcal Q_s^{l+1} &= \mathcal Q_s^l\odot \exp(\mathcal Q_r^l)+\mathcal F(\mathcal Q_r^l),\
\mathcal Q_r^{l+1} &= \mathcal Q_r^l\odot \exp(\mathcal G(\mathcal Q_s^{{l+1}))+\mathcal} H(\mathcal Q_s^{l+1}).
\end{aligned}
$$
Typical settings are $k=3$ nearest neighbors, $M=8$ for $4\times$ sampling, $M=4$ for $8\times$ and $16\times$, and feature dimension $d=64$–$128$ per point. Training uses a sum of sampling loss $\mathcal L_{\rm sam}$ based on EMD, reconstruction loss $\mathcal L_{\rm rec}$ with EMD plus repulsion and uniformity terms, and distribution-fitting loss $\mathcal L_{\rm dis}$ that forces $z$ toward a Gaussian prior [2104.14769]. On the PU-147 benchmark, PointLIE achieves state-of-the-art CD and HD at 8× and 16×, and as an upsampler it matches or beats PU-GAN on standard sparse-to-dense benchmarks [2104.14769].

Taken together, these methods show two geometric interpretations of embedding-based sampling. One interpretation treats the embedding as a local tangent geometry from which one can draw faithful virtual neighbors; the other treats it as an invertible residual store that makes aggressive sampling reversible. A plausible implication is that both approaches replace naive resampling by a representation-aware mechanism that preserves local topology more explicitly than interpolation alone.

3. Pair, distance, and curriculum sampling in deep embedding learning

In deep metric learning, the basic problem is that the usefulness of a sampled pair depends strongly on where it lies in embedding space. “Sampling Matters in Deep Embedding Learning” models normalized embeddings as roughly uniform on the unit sphere $\mathbb S^{D-1}$ and derives the distance density
$$
q(d)\propto d^{{D-2}\Bigl(1-\frac{d^{2}{4}\Bigr)^{{\frac{D-3}{2}}}}}
$$
for pairwise Euclidean distance $d=|f(x_i)-f(x_j)|$ [1706.07567]. In high dimensions, $q(d)$ concentrates near $d\approx\sqrt2$, so uniformly sampled negatives are usually easy and yield no gradient, while the hardest negatives are unstable because the gradient direction becomes dominated by noisy embedding estimates. The proposed remedy is distance-weighted sampling:
$$
w(D_{an})=\min!\Bigl(\lambda,\frac{1}{q(D_{an})}\Bigr),\qquad \Pr(n^\star=n\mid a)\propto w(D_{an}),
$$
with clipping constant $\lambda=10^4$. This is paired with the margin loss
$$
\ell_{\rm margin}(i,j)=\max\bigl(0,\alpha+y_{ij}(D_{ij}-\beta)\bigr),
$$
where $\beta$ is learned [1706.07567].

The reported gains are substantial. On Stanford Online Products, margin loss with distance-weighted sampling reaches R@1 = 72.7, compared with 66.7 for triplet plus semi-hard sampling; on CARS196 it reaches R@1 = 86.9 and NMI = 77.5; on CUB200 it reaches R@1 = 63.9 and NMI = 69.8; on LFW it attains 98.37% accuracy with 256-d embeddings [1706.07567]. The paper also reports that on LFW validation, margin plus distance-weighted sampling converges in ≈40 K iterations, whereas triplet plus semi-hard sampling takes ≈120 K to reach the same accuracy [1706.07567].

“Dynamic Sampling for Deep Metric Learning” addresses the same training signal problem through an easy-to-hard schedule rather than a stationary distance law [2004.11624]. Positive pairs are retained when $s_{ij}<\tau_p$ and negative pairs when $s_{ij}>\tau_n$ and $s_{ij}>\min_{k:y_{ik}=1}s_{ik}-\tau_b$, with typical thresholds $\tau_p=0.9$, $\tau_n=0.1$, and $\tau_b\approx0$. Their hardness weights evolve linearly with the epoch index:
$$
w^{+{ij}(E_c)=\frac{2E_c}{E_t}(\tau_p-s{ij})^2,\qquad}
w^{-{ij}(E_c)=\frac{2E_c}{E_t}(s{ij}-\tau_n)^2.}
$$
These terms are inserted into several standard losses, including binomial deviance and triplet loss, so that early training is dominated by easy pairs and later training by hard pairs [2004.11624].

The empirical pattern is consistent across tasks. On DeepFashion2, binomial deviance improves from 40.2% Recall@1 to 44.0% when both thresholds and dynamic weights are used; on In-Shop, binomial deviance improves from 87.3 to 89.8; on Cars-196, triplet loss improves from 55.4 to 63.2; on Market-1501, multiple-similarity improves from 62.0 to 63.0 mAP/Recall@1 [2004.11624]. The paper frames this as avoiding the damage that static hard-mining can inflict on a general boundary.

These two lines of work converge on a shared principle: in a learned embedding manifold, the sampling distribution is not a peripheral implementation detail but a primary determinant of optimization behavior. One method estimates informativeness from ambient distance statistics on $\mathbb S^{D-1}$; the other schedules informativeness by similarity thresholds over training time. This suggests that embedding-based sampling in metric learning is fundamentally a problem of controlling gradient variance through geometric selectivity.

4. Sampling for graph and knowledge-graph embeddings

Graph embedding methods expose several distinct sampling objects: random walks, node-context pairs, pair frequencies, central nodes, and corrupted triples. MLANE formalizes random-walk context generation as a small MDP whose state is the source node and current hop distance, and whose actions are a_f (forward), a_s (same), and a_b (back) [2010.13023]. Its policy network $\pi_\phi(a\mid s)$ is optimized jointly with SkipGram parameters $\theta$ in a meta-learning loop: sampled trajectories define contexts, SkipGram is trained on the inner loss
$$
L_{\rm task}(\theta;C)=-\sum_{v\in V}\sum_{u\in C_v}\log p_\theta(u\mid v),
$$
and the outer objective maximizes a downstream task metric $M_T(Z)$ through REINFORCE,
$$
\nabla_\phi J(\phi)\approx R\cdot \nabla_\phi\log \rho(\Psi;\phi).
$$
Because the policy is conditioned on node identity and hop distance, it can adapt between homophily-preserving DFS-style moves and structural-equivalence-preserving BFS-style moves [2010.13023].

Across six real networks, MLANE outperforms uniform walks, fixed-bias walks, purely homophily models, and purely structural models. On Cora node classification, Micro-F1 rises from 0.811 for DeepWalk and 0.816 for node2vec to 0.832; on Citeseer, Macro-F1 rises from 0.531 and 0.543 to 0.573; on Amazon, Macro-F1 rises from 0.120 for LINE and 0.125 for node2vec to 0.172; on Cora clustering, purity rises from 0.760 for node2vec to 0.913, and NMI from 0.425 to 0.677 [2010.13023].

Smooth Pair Sampling modifies not the walk policy but the frequency with which co-occurring node pairs enter optimization [2501.12884]. If $(u,v)$ appears $#(u,v)$ times in the walk corpus, the smoothed relative frequency is
$$
f_\beta(u,v)=\frac{[#(u,v)]^{\beta}{M_\beta},\qquad} 0<\beta\le 1,
$$
and the implementation accepts an observed candidate pair with probability
$$
p_{\rm sample}(u,v)=[#(u,v)]^{\beta-1}.
$$
A two-pass Frequent sketch gives O(M) time, O(1) amortized work per pair, and memory O(b+n·d) [2501.12884]. The point is to flatten the heavy head of the empirical Zipf-like pair distribution so that learning is not dominated by a tiny fraction of co-occurrences. On eight real graphs, SmoothDeepWalk raises Precision@100 in link prediction from ≈0.75→0.90 on average, with up to +200% gain on sparse Pubmed, Git, and Flickr; training is only ×1.2–2.8 slower than DeepWalk [2501.12884].

TGE-PS attacks redundancy even more aggressively. Its Pairs Sampling procedure replaces random-walk windows by a depth-limited pair generator over $o$th-order neighborhoods, so that RW-based sampling on the order of
$$
(2L-W-1)WT|V|
$$
pairs is replaced by
$$
NO\bar d|V|
$$
pairs [1809.04234]. On real graphs the resulting ratio corresponds to a ~99% reduction in sample size, while link-prediction performance remains competitive or improves. On SNOMED, the paper computes a ratio of about 409, meaning PS uses only ∼0.25% of RW samples. Coupled with its text-driven encoder, TGE-PS achieves state-of-the-art results on both standard and zero-shot link prediction, including AUC_pair = 0.781 on HepTh zero-shot compared with 0.672 for Text Matching [1809.04234].

Centrality-weighted sampling changes which nodes matter most when generating Skip-Gram updates. The objective weights each positive example by $\lambda_i=C(v_i)$, where $C(v_i)$ may be degree, PageRank, closeness, betweenness, or load centrality [1907.08793]. Under a unified framework over DeepWalk, node2vec, LINE, and NBNE, all centralities outperform the uniform baseline on Cora and Citeseer. On Cora, DeepWalk improves from 0.5805 Micro-F1 to 0.6380 with degree centrality, and node2vec improves from 0.5952 to 0.6375; on Citeseer, DeepWalk improves from 0.4019 to 0.4536 with load centrality [1907.08793]. The paper also reports roughly 2× faster convergence in the number of samples needed to reach a given accuracy [1907.08793].

Knowledge graph embedding introduces another variant: negative sampling of corrupted triples. A PyKEEN extension implements Uniform Random, Bernoulli, Corrupt, Typed, Relational, Nearest-Neighbor, and Adversarial samplers under the standard margin-ranking loss
$$
L=\sum_{(h,r,t)\in\mathcal T^{+}\sum_{(h',r,t')\in\mathcal} N(h,r,t)}
\max(0,\gamma+s(h,r,t)-s(h',r,t')).
$$
Bernoulli sampling biases head versus tail corruption according to relation arity; Typed sampling restricts corruptions by domain and range; dynamic samplers use auxiliary embeddings and $k$-NN pools [2508.05587]. On FB15K at $\eta=1$, average Hits@10 over 12 models is 0.640 for uniform, 0.650 for Bernoulli, 0.648 for typed, and 0.612 for adversarial; on WN18, Bernoulli reaches 0.948 versus 0.943 for uniform [2508.05587]. The paper emphasizes the trade-off between harder negatives and the overhead of nearest-neighbor search.

A common misconception is that graph embedding quality is determined mainly by the loss or encoder architecture. These works collectively contradict that view: the walk policy, pair-frequency law, sample redundancy, centrality prior, and corruption rule all alter what structural evidence the model ever sees. That conclusion is explicit in MLANE, Smooth Pair Sampling, TGE-PS, and the PyKEEN extension, even though the underlying embedding objectives differ substantially [2010.13023], [2501.12884], [1809.04234], [2508.05587].

5. Adaptive sampling in implicit kernel embeddings and sampled GNN training

In kernel approximation, the relevant embedding is the lifted feature map itself. “Near Input Sparsity Time Kernel Embeddings via Adaptive Sampling” considers polynomial kernels with
$$
\Phi(x)=x^{\otimes q}\in\mathbb R^{{d^q},\qquad}
k(x,y)=\langle x,y\rangle^q=\langle \Phi(x),\Phi(y)\rangle,
$$
and truncated Gaussian-kernel embeddings obtained from Taylor expansion [2007.03927]. The objective is to construct a sketching matrix $S$ so that $S\Phi$ is a subspace embedding in near input-sparsity time. Rows of $\Phi$ are sampled either by row norms,
$$
p_i \asymp \frac{|\phi_i|^{2}{|\Phi|_F^2},}
$$
or by ridge leverage scores,
$$
\ell_i^{\lambda=\phi_i^{T(\Phi^{T\Phi+\lambda}}} I)^{{-1}\phi_i,\qquad} p_i\propto \ell_i^\lambda.
$$
A recursive outer algorithm decreases the regularization parameter geometrically, while a specialized RowSampler uses Gaussian or SRHT sketches together with the oblivious multilinear sketch of Ahle et al. to sample rows of tensor-product features rapidly [2007.03927].

The resulting guarantee is spectral:
$$
Z^TZ+\lambda I \;\approx_{1\pm\epsilon}\; K+\lambda I,
$$
with target dimension $s=O(s_\lambda\epsilon^{-2}\log n)$ and running time
$$
\tilde O!\left(s_\lambda^2n\cdot \operatorname{poly}(\epsilon^{-1},q,\log n)+\operatorname{nnz}(X)\cdot q^{5/2}\log⁴ n\right).
$$
The paper states that this improves the dependence on the polynomial degree by a factor of $q^{{5/2}/\epsilon^2$} over the recent oblivious sketching method of Ahle et al., and that in large-scale regression tasks the method outperforms state-of-the-art kernel approximation baselines [2007.03927]. It also derives a kernel ridge regression risk bound showing that the approximate predictor retains the exact KRR statistical rate up to $(1\pm O(\epsilon))$ factors [2007.03927].

For GNNs, the embedding-based sampling problem appears as variance control over intermediate node embeddings. “Minimal Variance Sampling with Provable Guarantees for Fast Training of Graph Neural Networks” decomposes the total gradient error into embedding-approximation variance and stochastic-gradient variance:
$$

E|\tilde{\mathbf g}-\nabla f(\theta)|²

E|\tilde{\mathbf g}-\mathbf g|²
+
E|\mathbf g-\nabla f(\theta)|^2.
$$
The first term arises because sampled neighborhoods approximate inner-layer embeddings; the second is the usual minibatch SGD variance [2006.13866]. Minimizing the backward-stage variance under a minibatch budget $B$ yields the optimal inclusion probabilities
$$
p_i=\min\Bigl{1,\frac{\bar g_i}{\mu}\Bigr},\qquad \sum_i p_i=B,
$$
where $\bar g_i$ is an upper bound or estimate of the gradient norm [2006.13866].

MVS-GNN implements this in decoupled outer and inner loops. Outer iterations compute a large exact subgraph pass, store historical embeddings $\bar H_i^{(\ell)}$ and fresh gradient norms $\bar g_i$, and inner iterations sample a small minibatch under the probabilities above, mixing current and historical embeddings in the forward pass and updating parameters with importance weighting [2006.13866]. The paper reports that on Reddit and PPI, MVS-GNN reaches target accuracy 6–10× faster in wall-clock time than GraphSage, VRGCN, LADIES, ClusterGCN, or GraphSaint, and remains stable even with minibatch sizes as small as 32 on Cora and Citeseer [2006.13866].

These two papers represent a computational branch of embedding-based sampling. The sampled object is neither a data point nor a semantic pair, but a coordinate or node contribution in an implicit representation space. A plausible implication is that once the feature map itself is too large to materialize exactly, sampling becomes a way of approximating the embedding operator rather than merely selecting training examples.

6. Embedding-space targeting for synthetic data generation

Embedding-based sampling has recently been used to steer synthetic data generation toward weak regions of a model’s representation space. “Efficient Embedding-based Synthetic Data Generation for Complex Reasoning Tasks” analyzes a student model’s embedding distribution and reports a near-linear relation between local example density and accuracy in that region, with Pearson’s r≈0.813 (p<10⁻¹¹) and Spearman’s ρ≈0.806 (p<10⁻¹¹) [2603.22294]. The method embeds each seed example into a low-dimensional space $\mathcal E\subset\mathbb R^K$ using the student model’s own embedding and attention weights, with K=2 or 3 in practice, overlays a regular grid, and defines the count
$$
D(c)=|{x\in D:e(x)\in c}|.
$$
Cells satisfying
$$
0<D(c)<T
$$
are treated as candidate sparse regions; empty cells are skipped [2603.22294].

Sampling is uniform over candidate sparse cells,
$$
p(c)=\frac{1}{|S|},
$$
and after choosing a cell, the method samples two seed examples on opposite boundaries of that cell, interpolates their embeddings using a midpoint construction based on linear dimensionality reduction, decodes that interpolated embedding into a partial prompt via the student model, and asks a teacher model to generate a full QA example from the two original seeds plus the decoded midpoint prompt [2603.22294]. Because PCA or TruncatedSVD is linear, the interpolated embedding lies at the midpoint of the two seed embeddings in $\mathcal E$ [2603.22294].

The reported gains are consistent across all three student models and both benchmarks. On GSM8K, Granite 3 8B Code Instruct improves from 0.761 with RandomSDG to 0.782 with EmbedSDG, versus 0.55 for the base model; Granite 3.1 8B Instruct improves from 0.786 to 0.824, versus 0.74 base; Mistral 7B improves from 0.354 to 0.620, versus 0.354 base. On MATH, Granite 3 8B Code Instruct improves from 0.225 to 0.249, Granite 3.1 8B Instruct from 0.280 to 0.342, and Mistral 7B from 0.214 to 0.244 [2603.22294]. The paper states that gains persist up to k=4 500 synthetic examples [2603.22294].

This work is notable because the embedding is not only an analysis tool but a generator-conditioning substrate. The sparse-cell criterion defines where new samples should be requested, and interpolation defines what semantic region they should target. The authors also recommend choosing grid resolution so that most cells hold on the order of 30–50 points, then setting $T$ near the 10–20th percentile of the cell-count histogram, with T≈10 as an example when typical cell counts are 30–50 [2603.22294]. A plausible implication is that embedding-based sampling can act as a low-cost acquisition strategy even when no explicit uncertainty model is available.

7. Failure modes: sampling bias under hardware minor embedding

In statistical physics and quantum annealing, embedding-based sampling refers to minor embedding of a target Hamiltonian into limited-connectivity hardware, and the results are largely cautionary. For classical thermal sampling, a logical Ising model
$$
H(s)=\sum_{i<j}J_{ij}s_is_j+\sum_i h_is_i
$$
is embedded by replacing each logical spin with a connected subtree or chain of physical spins tied by strong ferromagnetic couplings $J_F<0$ [1909.12184]. Logical configurations are those with no broken chains. Although the logical subspace reproduces the correct Boltzmann weights when restricted to unanimous chains, the probability of drawing a sample directly in that subspace is
$$
P_0=(1+e^{2\beta J_F})^{-(K-1)N},
$$
which decays exponentially in system size $N$ and chain length $K$ [1909.12184]. The paper notes that for N≈120, K=3, and βJ_F≈−1.2, one already expects fewer than 10^−9 logical samples per draw [1909.12184].

This is the basis for a central warning: an embedding that is adequate for optimization need not be adequate for sampling. The paper states explicitly that the effect of embedding is more pronounced for sampling than for optimization, because sampling depends on states at all energies rather than only the ground state [1909.12184]. Standard post-processing by majority vote produces a distorted distribution: the fitted inverse temperature falls below the true $\beta$, the KL divergence to the target Boltzmann law grows with $N$, and states of equal logical energy can occur with frequencies differing by orders of magnitude [1909.12184]. To mitigate this, the paper introduces restricted resampling (RRS), which keeps unbroken logical spins fixed and re-thermalizes only the broken chains under the original Hamiltonian. Empirically, RRS yields much smaller KL divergence and fitted inverse temperature close to the true $\beta$ [1909.12184].

The quantum extension studies transverse-field Ising models under minor embedding [2103.07036]. The logical-subspace probability is
$$
P_L=\frac{\sum_{z_L\in\Omega_0}\langle z_L|e^{-\beta \hat H}|z_L\rangle}{\operatorname{Tr}[e^{-\beta \hat H}]},
$$
and in the large-transverse-field limit satisfies $P_L\approx 2^{-N(K-1)}$ [2103.07036]. More generally, the paper models it as
$$
P_L(\Gamma,T,N)\approx [p_\ell(\Gamma,T)]^{N(K-1)}.
$$
It also introduces an LC-QMC algorithm that enforces one Trotter slice to remain in the logical subspace, enabling much more efficient simulation of embedded quantum Hamiltonians [2103.07036].

The observed bias extends beyond sample validity. In the intermediate-field regime 1≲Γ/Δ≲10, embedded sampling yields a downward shift in diagonal energy and an upward shift in absolute antiferromagnetic magnetization relative to the native model; for larger embedding size $K$, the magnetization histogram increasingly “locks in” order [2103.07036]. The critical transverse field shifts to larger values as $K$ increases, roughly linearly for the uniform and random embeddings studied, and for random embeddings a clean Binder-cumulant crossing reappears only after averaging over ≥30 embedding realizations [2103.07036].

These papers establish an important controversy within the broader topic. In most machine-learning contexts, embedding-based sampling is introduced to improve efficiency or fidelity. In minor-embedded thermal and quantum sampling, by contrast, the embedding can be the primary source of distortion. The corrective techniques—restricted resampling in the classical case and logically constrained QMC for simulation in the quantum case—are therefore not mere implementation details but necessary responses to a representation mismatch between the target model and the sampling hardware [1909.12184], [2103.07036].