A Flow Matching Algorithm for Many-Shot Adaptation to Unseen Distributions

Abstract: While generative modeling has achieved remarkable success on tasks like natural language-conditioned image generation, enabling model adaptation from example data points remains a relatively underexplored and challenging problem. To this end, we propose Function Projection for Flow Matching (FP-FM), an algorithm that directly conditions generation on samples from the target distribution. FP-FM learns basis functions to span the velocity fields corresponding to a set of training distributions, and adapts to new distributions by computing a simple least-squares projection onto this basis. This enables efficient generation of samples from diverse target distributions without additional training at inference time. We further introduce multiple variants of FP-FM that provide a trade-off in expressivity and compute by enriching the coefficient calculation, e.g., by making the coefficients dependent on time. FP-FM achieves greatly improved precision and recall relative to baselines across synthetic and image-based datasets, with especially strong gains on unseen distributions.

• The paper introduces FP-FM, which uses function space projection to adapt generative models using sample data without gradient-based finetuning.
• It leverages basis functions to span velocity fields and employs least-squares to compute coefficients for modeling unseen distributions efficiently.
• Dynamic FP-FM demonstrates superior performance on metrics like precision, recall, and FID across datasets like MNIST and ImageNet.

Flow Matching for Many-Shot Adaptation to Unseen Distributions

Problem Overview and Motivation

Generative models, particularly those based on diffusion and flow matching, have achieved strong empirical results, but the problem of adapting generative models to novel target distributions given only example data points (not explicit conditioning variables) remains underexplored. In real-world settings, new target distributions frequently appear for which only samples, but no formal descriptions, are accessible. Current solutions predominantly rely on explicit conditioning variables (e.g., class labels or text) or require computationally intensive finetuning, which fail to generalize efficiently when only samples from the target distribution are available.

To address this, "A Flow Matching Algorithm for Many-Shot Adaptation to Unseen Distributions" (2605.06272) introduces Function Projection for Flow Matching (FP-FM), a methodology that learns to span the velocity fields of known distributions and then generates from unseen distributions via projection in a function space. The approach is general, requiring only samples (not labels or textual descriptions) and achieves adaptation without gradient-based optimization at inference.

FP-FM: Methodological Framework

FP-FM recasts the adaptation problem as one in function space, specifically focusing on the velocity fields defined within the flow matching framework. During training, basis functions parameterized by neural networks are learned to span empirical observations of velocity fields on a set of training distributions. For a novel target (unseen) distribution, a simple least-squares projection using only sample data yields coefficients for generating with these basis functions.

The core insight driving FP-FM is the exploitation of the Hilbert space structure of function spaces: inner products and least-squares projections can be well-defined and empirically estimated with i.i.d. samples from the target. The architecture is agnostic to data modality and does not require target distribution labels.

Figure 1: Basis functions are learned to span velocity fields for training distributions; a new target field is represented as a linear combination whose coefficients are computed via least-squares, with three FP-FM variants differing by the dependency of those coefficients.

FP-FM Variants

FP-FM is instantiated in three principal variants, trading off expressivity of the modeling and computational cost during adaptation:

• Static FP-FM: Coefficients for basis function linear combinations are fixed for a given target distribution.
• Temporal FP-FM: Coefficients vary as a function of time in the flow process.
• Dynamic FP-FM: Coefficients are fully dependent on state and time, maximizing flexibility at the expense of increased computation.

This progression reflects increasing functional expressivity. Static FP-FM is efficient but restrictive, while Dynamic FP-FM allows nuanced, sample-dependent behaviors necessary for complex, unseen distributions.

Theoretical Foundations

The method is justified by leveraging the mathematical properties of function encoders [Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spaces, ICML 2025] and modern flow matching (FM) [Flow Matching for Generative Modeling, ICLR 2023]. For each target distribution, a velocity field is defined, and its empirical estimation reduces to weighted least-squares projection given sample pairs. A crucial technical tool is a derivation of an estimator for the conditional expectation $\mathbb{E}[X_1 - X_0 \mid X_t = x]$ using importance weighting, facilitating tractable projections required for Dynamic FP-FM (see Theorem \ref{thm: sampling} and Figure 2).

Figure 3: Visualization of the estimation method for conditional expectations in the dynamic coefficient computation; the expected velocity field aligns with higher-probability point pairs.

Algorithmic Details and Ablations

Explicit algorithms for all variants are provided and analyzed. Notably:

• Adaptation for a new distribution involves only an empirical least-squares solve, avoiding additional neural network training.
• Computational complexity scales with the expressivity of the chosen variant: Dynamic FP-FM computes coefficients per sample and per time step, whereas Static FP-FM requires just a single solve per distribution.

An ablation on the number of provided shots (example samples) demonstrates FP-FM is robust to the number of examples, with performance improving gracefully as more samples are available.

Figure 5: Precision, recall, and FID as the number of shots varies—the approach rapidly achieves accurate generative modeling with increasing samples.

Empirical Results

Comprehensive experiments are conducted on synthetic low-dimensional distributions (2D Arcs), MNIST digits, and large-scale ImageNet subsets. Baselines include unconditional and conditional FM, classifier-guided diffusion, distribution-guided generation, and finetuning. Metrics focus on precision, recall, Fréchet Inception Distance (FID), and generation time.

Key findings:

• Dynamic FP-FM consistently obtains the highest precision and recall, especially on Unseen Distributions (UD) and Unseen Supports (US), outperforming training-based adaptation and traditional conditional approaches.
• FP-FM methods do not require expensive gradient-based finetuning at inference; adaptation is fast and efficient.
• Static and Temporal FP-FM undergo the anticipated expressivity-accuracy tradeoff but remain competitive, especially when computational cost is at a premium.

Notably, on ImageNet, Dynamic FP-FM achieves both qualitatively and quantitatively superior samples for unseen classes, with lower FID and improved alignment to target distributions.

Figure 2: Schematic of the distribution-guided baseline; FP-FM sidesteps the inefficiencies of this two-stage mapping approach.

Figure 6: FP-FM's robustness to the number of basis functions; increasing the basis yields diminishing but stable returns in generative fidelity.

Theoretical and Practical Implications

The FP-FM approach exposes a principled pathway for scaling generative modeling to highly flexible, sample-conditioned adaptation, moving beyond the constraints of explicit conditioning variables and slow finetuning. Function space projections afford fine-grained expressivity control, suggesting new directions for model regularization (e.g., expressing coefficient smoothness or Lipschitz properties).

Limitations include the dependence on adequate sample coverage and a computation cost—though still lower than finetuning—that grows with model expressivity. In very extreme low-shot regimes, approximation quality could degrade if the basis functions' expressivity or the sample set is insufficient.

Broader impact considerations note that efficient adaptation to arbitrary target distributions may amplify ethical risks related to impersonation or tailored synthetic data generation.

Conclusion

FP-FM provides a general and theoretically grounded method for generative adaptation via function space projections. It achieves state-of-the-art adaptation to novel distributions using only samples, outperforming conditional models and training-based adaptation in precision and recall. The explicit tradeoff between expressivity and computational efficiency allows practitioners to balance practical considerations against modeling power. Open research directions include interpolating the space of coefficient dependencies to control adaptation smoothness and extending FP-FM to further high-dimensional and structured modalities.

---