Vector Calibrated Sampling (VCS)

• Vector Calibrated Sampling (VCS) is a technique that corrects sampling drift by keeping updates orthogonal to invariant data manifolds, ensuring perceptual consistency.
• It employs a projection and renormalization step to remove velocity components along the invariant direction during ODE-based sample generation in LP-CFM.
• Empirical results on speech datasets demonstrate that VCS improves data efficiency and quality, particularly in low-step and low-data regimes.

Vector Calibrated Sampling (VCS) is a sampling correction technique introduced in the context of Linear Projection Conditional Flow Matching (LP-CFM) for generative modeling of perceptually invariant data, particularly in speech domains. VCS addresses the challenge of keeping the generated samples strictly within the manifold of perceptually equivalent variants during ODE-based sample generation, robustly aligning the sampling process with the desired invariance manifold and correcting drifts that arise in conventional flow-matching sampling methods (Kwak et al., 23 Dec 2025).

1. Motivation and Theoretical Foundations

Conventional generative models, including standard Conditional Flow Matching (CFM), transport samples from a simple prior to target data points (e.g., speech spectrograms) by numerically integrating a drift field learned to match an optimal path between source and target. These models enforce the endpoint distribution to be narrowly centered around each data instance, disregarding inherent data invariances. In speech, imperceptible transformations such as amplitude re-scaling or sub-frame temporal shifts produce distinct point representations that are perceptually equivalent.

LP-CFM addresses this by treating each training datum as a representative of an equivalence manifold—specifically, a line $L(n; x_1)=a(x_1)n + b(x_1)$ of all variants perceptually identical to $x_1$. The flow then targets an elongated Gaussian spread along this line rather than a point, inducing more semantically meaningful and robust transport paths. However, inference-time sampling using the learned field $f_\theta(t,x\mid y)$ may unintentionally steer samples off this line. Vector Calibrated Sampling is introduced to enforce that the model-generated velocity remains orthogonal to the equivalence manifold, thereby preventing drift along the manifold's invariant direction (Kwak et al., 23 Dec 2025).

2. Mathematical Description of VCS

For a given direction $a \in \mathbb{R}^d$ defining the invariance manifold and its associated projection matrix $P = \frac{a a^\top}{a^\top a}$, the raw neural vector field $v = f_\theta(t, x \mid y)$ at each ODE step can have a nonzero parallel component $P v$. This parallel component drives the generated sample away from the nearest point on the invariant line $L$, thereby degrading sample fidelity.

The VCS correction step removes this parallel component while preserving the vector's original norm: $$v_\perp = (I - P)v, \quad v' = \frac{\|v\|}{\|v_\perp\|} v_\perp.$$ Euler updates are then performed using $v'$: $x_{t-\Delta t} = x_t + \Delta t v'.$ This ensures the update is strictly orthogonal (in parameter space) to the invariance direction, so that samples remain aligned with the equivalence manifold throughout the integration trajectory. This correction is straightforward to implement for any linear manifold specified by $a$; $P$ is computed either analytically or from data-specific invariance knowledge (e.g., amplitude scaling, time shift).

3. Algorithmic Integration and Sampling Workflow

The prototypical VCS-based sampling workflow operates as follows:

• Input: Initial point $x_T \sim p_0$, conditioning target $y$ (the data sample), step size $\Delta t$, projection matrix $P$.
• For $t = T, T-1, \dots, 1$:

1. Compute predicted flow $v = f_\theta(t, x_t \mid y)$. 2. Project out the parallel component: $v_\perp = (I - P) v$. 3. Renormalize to original norm: $v' = (\|v\|/\|v_\perp\|) v_\perp$. 4. Euler update: $x_{t-\Delta t} = x_t + \Delta t v'$.

• Return: $x_0$ as the generated sample.

This update rule is identical in both uni- and multi-dimensional settings as long as the manifold is locally linear. VCS does not modify the generative field at training, but recalibrates its effect during inference.

Step	Operation	Purpose
1	$v = f_\theta(t, x_t \mid y)$	Predicts drift
2	$v_\perp = (I - P) v$	Removes parallel component
3	$v' = (\|v\|/\|v_\perp\|) v_\perp$	Normalizes to preserve step size
4	$x_{t-\Delta t} = x_t + \Delta t v'$	Sample update

Employing VCS requires only knowledge of the invariance direction $a$ and can be universally combined with any flow-matching network $f_\theta$ trained for elongated manifold targets.

4. Implementation and Integration with LP-CFM

In practical instantiations for speech, LP-CFM defines $a$ for amplitude invariance as the all-ones vector in log-magnitude space, or as the negative group delay vector for phase shift invariance. The projection $P$ and the corresponding $v'$ are computed per-sample at each integration step.

The VCS module is computationally negligible, as it involves only vector projection and normalization at inference. Network architectures and training regimes are agnostic to VCS, except that the underlying LP-CFM must have been trained with an elongated (manifold-aware) target rather than the isotropic target used in OT-CFM (Kwak et al., 23 Dec 2025).

A critical empirical finding is that, when used with OT-CFM (which lacks manifold alignment in its learned flow), VCS calibration is detrimental, confirming that the VCS correction is only appropriate when the vector field has incorporated the invariance direction during training.

5. Empirical Findings and Analysis

VCS, as deployed in LP-CFM, supports significantly improved robustness and perceptual quality in speech generation tasks relative to standard CFM or OT-CFM, particularly in low-resource and few-step regimes.

Key results (Kwak et al., 23 Dec 2025):

• On the LJSpeech dataset, LP-CFM with VCS demonstrates consistent improvements in M-STFT, PESQ, MCD, and UTMOS across model scales (UNet-16, UNet-32, UNet-64).
• LP-CFM with VCS achieves comparable or superior performance to full-data OT-CFM even with $33\%$ or $66\%$ of the data, highlighting enhanced data efficiency.
• Superior UTMOS is retained with as few as $3$–$12$ sampling steps, with LP-CFM+VCS outperforming all baselines in rapid-generation regimes.
• Subjective preference is confirmed by CMOS studies in low-step, low-data, and compact-model settings ($p<0.05$).
• Ablation confirms that VCS is essential for constraining sampling to the learned manifold; however, it is only effective if the network has adopted projection-aligned flows.

A plausible implication is that VCS, by enforcing strict adherence to perceptual invariance, enables aggressive reduction in step count during sampling and enhances fidelity under tight resource or data constraints.

6. Broader Implications and Extensions

VCS defines a simple, universal post-processing scheme to enforce manifold-aligned sampling for any linear invariance. Potential generalizations include:

• Application to arbitrary linear invariances: e.g., frequency scaling, pitch shifting, or image rotation (by defining relevant $a$ and $P$).
• Extension to non-linear manifolds by local linearization and dynamically computed $P$.
• Joint handling of multiple invariances via summation or stacking of multiple projection matrices $(P_1, P_2, ...)$ to accommodate composite invariance manifolds.
• Possible use outside speech, such as for images, style transfer, or 3D generative modeling with known symmetry groups.

By doctrinally separating the transport field learning from the invariance calibration during sampling, VCS presents an efficient, generalizable strategy for integrating manifold-awareness into sample generation (Kwak et al., 23 Dec 2025).