Directionally-Pruned Sampling
- Directionally-pruned sampling is a framework where pruning or sampling decisions are based on the geometry of evolution rather than uniform or scalar criteria.
- In flow-matching text-to-speech, EPSS exemplifies this approach by retaining dense, high-curvature timesteps early and pruning near-linear later steps to cut computational cost.
- The concept spans diverse applications—from dataset and directed graph sampling to neural network pruning—each using geometry-aware metrics to inform efficient sample retention.
Directionally-pruned sampling is best understood as an Editor’s term for pruning or sampling schemes in which retention, omission, or step allocation is informed by the geometry of evolution in a state space rather than by uniform allocation or purely scalar ranking. In the most direct instance among the cited works, Empirically Pruned Step Sampling (EPSS) for flow-matching text-to-speech prunes timesteps after inspecting whether the sampling trajectory is highly curved or nearly linear (Zheng et al., 26 May 2025). Closely related but non-identical uses of the idea appear in dataset pruning through class-relative feature geometry (Grosz et al., 2024), node selection on directed graphs through direction-sensitive smoothness priors (Li et al., 2022), motion-planning samplers that restrict samples to cost-improving informed sets and use directional proposals (Yi et al., 2017), and neural-network pruning methods that move along flat directions of a loss valley rather than by raw magnitude alone (Chao et al., 2020, Li et al., 2021). This suggests that the phrase names a family resemblance across methods rather than a single canonical algorithm.
1. Conceptual scope and taxonomy
The most precise common core is pruning guided by structure in how a system changes: curvature of a generation trajectory, class-relative geometry in feature space, directional flow on a graph, or flat versus high-curvature directions in parameter space. The cited literature does not define one universal formalism spanning all these settings. A careful reading therefore requires distinguishing direct matches from partial analogues.
| Domain | Mechanism | Relation to the term |
|---|---|---|
| Flow-matching TTS | Non-uniform timestep pruning from observed trajectory shape | Directly relevant (Zheng et al., 26 May 2025) |
| Dataset pruning | Geometry-aware scalar scores and adaptive importance sampling | Partially relevant (Grosz et al., 2024, Wu et al., 11 Jun 2026) |
| Directed graph sampling | Node selection under directed smoothness and Gershgorin bounds | Direction-aware sampling (Li et al., 2022) |
| Motion planning | Informed-set restriction with MCMC, especially Hit-and-Run | Indirectly directional (Yi et al., 2017) |
| DNN pruning | Flat-direction-aware sparsification in parameter space | Directional pruning, not sampling (Chao et al., 2020, Li et al., 2021) |
A recurring misconception is to equate any non-uniform sampler with directional pruning. The papers do not support that equivalence. In several cases, the method is geometry-aware without computing an explicit direction vector, angle threshold, or directional derivative. Conversely, some methods are explicitly directional but are pruning schemes rather than sampling schemes.
Another recurring misconception is to assume that “directional” always means runtime adaptation. The strongest direct example, EPSS, uses a fixed empirical schedule, not online direction-adaptive control (Zheng et al., 26 May 2025). Likewise, SIMS and UGIES are geometry-aware subset-selection methods, but their actual retention rules are scalarized through scores or marginal gains rather than through explicit vector-valued directional operators (Grosz et al., 2024, Wu et al., 11 Jun 2026).
2. Trajectory-geometry pruning in flow-matching text-to-speech
The clearest instantiation of directionally-pruned sampling in the supplied literature is EPSS in flow-matching TTS. Flow matching learns a time-dependent vector field
that induces
In the TTS setting, the model mainly generates Mel spectrograms. Under the OT path,
the conditional flow matching loss is written as
$\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$
with
At inference, generation solves
Because the integral is not available in closed form, discrete ODE solvers such as Euler are used over timesteps (Zheng et al., 26 May 2025).
The acceleration problem arises because each step requires at least one evaluation of the neural vector field, so the number of function evaluations directly drives inference time. The paper notes that Voicebox used 64 steps, while E2 TTS and F5-TTS brought this down to 32 steps, but even 32 iterative neural evaluations remain expensive (Zheng et al., 26 May 2025).
The baseline schedule is uniform,
$t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$
while F5-TTS also uses sway sampling,
with coefficient in experiments (Zheng et al., 26 May 2025). EPSS is introduced as a training-free, non-uniform time-step sampling strategy that prunes unnecessary steps based on empirical observations of the sampling trajectory. The authors inspect PCA projections of the 100-dimensional Mel trajectory and per-dimension trajectories of selected Mel features, concluding that the early phase is highly curved and uncertain, whereas the later phase is nearly linear. The pruning criterion is therefore not a formal local truncation-error estimate, not a norm threshold on 0, and not a perceptual sensitivity model; it is a qualitative trajectory analysis followed by hand-crafted timestep selection and empirical validation (Zheng et al., 26 May 2025).
This geometric asymmetry yields the characteristic EPSS allocation: denser, smaller intervals in the early phase and sparser, larger intervals later. A representative 7-NFE schedule is
1
The corresponding F5-TTS implementation is
2
The method is therefore best characterized as timestep pruning informed by trajectory geometry rather than as explicit directional-vector pruning (Zheng et al., 26 May 2025).
The empirical evidence is unusually direct. For F5-TTS, 32 NFE gives RTF 0.123 and LibriSpeech-PC WER 2.37. Plain 7-step reduction without EPSS preserves the same RTF advantage, 0.030, but degrades LibriSpeech-PC WER to 4.16. Fast F5-TTS, defined as F5-TTS + EPSS at 7 NFE, retains RTF 0.030 while recovering LibriSpeech-PC WER to 2.45, with comparable SIM-o and UTMOS across LibriSpeech-PC and Seed-TTS test-en/test-zh (Zheng et al., 26 May 2025). The paper states that this 7-step EPSS configuration reduces computational cost by 78% relative to 32-step sampling while maintaining comparable quality, and is about 4 times faster than original F5-TTS on an NVIDIA RTX 3090 GPU (Zheng et al., 26 May 2025). Transfer to E2 TTS shows the same pattern: baseline 32 NFE yields LibriSpeech-PC WER 2.63, plain 7 NFE degrades to 4.04, and E2 TTS + EPSS recovers to 2.73 at the same 0.036 RTF (Zheng et al., 26 May 2025).
The most revealing ablation compares two 6-step schedules. The 6a variant prunes a later timestep at 3, while 6b prunes an earlier timestep at 4. Since 6a substantially outperforms 6b, the paper’s central claim is supported directly: later steps are more dispensable, early steps are more critical (Zheng et al., 26 May 2025). This is why EPSS is the strongest concrete example of directionally-pruned sampling “in spirit”: it preserves high-curvature regions of the trajectory and prunes nearly straight regions.
3. Geometry-aware sample retention in dataset pruning
In dataset pruning, the relation to directionally-pruned sampling is weaker but still technically substantive. SIMS defines a pruning metric from separability, integrity, and model uncertainty, then performs adaptive importance sampling over the resulting scalar scores (Grosz et al., 2024). The setup is
5
with pruning ratio
6
The key geometric component is class-relative separability:
7
8
9
Integrity is
0
and certainty is derived from ensemble JSD:
1
2
3
After aggregation and normalization, the combined score is
4
5
Sampling then uses
6
with Gaussian score distributions and pruning-ratio-dependent parameters (Grosz et al., 2024).
This method is not explicitly directional. It does not compute directional vectors such as 7, does not project onto margin normals, and does not use directional gradients. Its closest analogy to directionality is the separability term, which compares a sample to its own class center and nearest impostor class center in angular space (Grosz et al., 2024). The sampling stage itself is score-based and scalar.
Empirically, SIMS is strongest at high pruning ratios. On CIFAR-100 with 8, the paper reports SIMS 33.83 versus Forgetting 21.72, Prototype 7.52, and EL2N 6.37. On Tiny-ImageNet with 9, SIMS achieves 27.60 versus Forgetting 25.11. On iNaturalist with $\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$0, SIMS achieves 22.42 versus Random 17.61 and Forgetting 4.57 (Grosz et al., 2024). The authors report that in the high-$\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$1 regime, SIMS beats Forgetting Scores by 22.1% average test accuracy improvement across datasets, and that incorporating importance sampling improves average performance by 30.87% for SIM and 8.57% for Forgetting Scores over $\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$2 (Grosz et al., 2024). These results support a broader interpretation in which geometry-aware pruning can be paired with sampling procedures, even when the geometry has been scalarized.
A more recent graph-based formulation, UGIES, goes further in unifying intrinsic and extrinsic criteria (Wu et al., 11 Jun 2026). It models the dataset as a weighted graph with node weights
$\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$3
and optimizes
$\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$4
The greedy marginal gain is
$\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$5
Under non-negative $\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$6 and non-positive $\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$7, the paper proves submodularity and invokes the $\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$8 greedy guarantee (Wu et al., 11 Jun 2026). The framework is not directional by default, but the paper explicitly notes that adapted gradient similarity can be used as an extrinsic term. This suggests a direct route to “directionally diversified sampling” if $\mathcal{L}_{\text{CFM}(\theta) = \mathbb{E} \left \Vert m \odot (v_t(x_t, c; \theta) - (x_1 - x_0)) \right \Vert ^2,$9 is chosen in gradient space rather than in generic feature space.
4. Direction-aware sampling on directed graphs
In graph signal processing, the term is closest to its literal meaning when the graph itself is directed. The directed-graph sampling problem chooses a node subset through
0
producing samples
1
The goal is high-fidelity reconstruction of 2 from those retained node values (Li et al., 2022).
The directed graph model is
3
with positive edge weights, no self-loops,
4
out-degree matrix 5, directed Laplacian
6
normalized adjacency
7
and random-walk Laplacian
8
The paper avoids directed graph Fourier constructions and instead uses graph shift variation,
9
as the smoothness prior (Li et al., 2022). This is direction-sensitive because it measures mismatch between a signal and its one-hop random-walk shift along directed edges.
Reconstruction is posed as
0
with coefficient matrix
1
so that
2
Sampling is then formulated as maximizing the smallest eigenvalue,
3
subject to binary diagonal 4 with trace 5 (Li et al., 2022).
To avoid eigendecomposition, the paper adapts Gershgorin Disc Alignment Sampling. For a symmetric matrix, the smallest disc left-end
6
lower-bounds 7. The directed method, GDA-Direct, optimizes a lower bound involving
8
rather than repeatedly handling 9 spectrally (Li et al., 2022). The practical complexity is
0
which the paper describes as roughly linear in 1 for sparse graphs and small 2 (Li et al., 2022).
This is genuinely direction-aware sampling, but not because it prunes directions in a latent space. Rather, node retention is optimized under a reconstruction model whose regularizer depends on directed diffusion. The paper reports that GDA-Direct achieves the lowest reconstruction MSE for all three tested signal classes and all sampling budgets 3, and compared with non-GDAS baselines reduces MSE by 8.6% for GS1 at 4 and by 11.9% for GS3 at 5; at 6, GDA-Direct is 1.4× faster than GDA-Balance (Li et al., 2022). The directional content is therefore encoded in the signal prior and the resulting optimization criterion.
5. Informed-set restriction and directional proposals in motion planning
In asymptotically optimal motion planning, pruning is applied not to coordinates or timesteps but to the admissible sampling region after an initial solution is found. The informed set is
7
where 8 is the incumbent solution cost (Yi et al., 2017). The central idea is to sample uniformly only from states that can still improve the current solution, rather than from the full state space.
The paper’s key reformulation is to regard informed sampling as uniform sampling from a sublevel set of an implicit non-convex function. This is not explicit directional pruning in the sense of heading-based or control-direction-based elimination. The pruning operation is cost-bounded subset restriction:
9
The directional aspect arises secondarily, through the geometry of the informed set induced by dynamics and through the MCMC transition kernel (Yi et al., 2017).
The target distribution is uniform over $t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$0. Two MCMC realizations are emphasized. Metropolis-Hastings uses local Gaussian proposals and accepts according to the standard ratio. Hit-and-Run is more directionally explicit: from current state $t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$1, sample a random unit direction $t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$2, define the line
$t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$3
sample $t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$4 uniformly on a bounded interval, and shrink the interval until a point inside $t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$5 is found (Yi et al., 2017). Thus, the chain explores the pruned set by directional line-based moves.
The paper proves that Informed RRT* running with MCMC-based informed sampling is asymptotic optimal, because any region of $t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$6 has nonzero probability of being sampled (Yi et al., 2017). Empirically, the comparison among rejection sampling, hierarchical rejection sampling, Metropolis-Hastings, and Hit-and-Run shows that as the informed-set volume ratio decreases, MH and HNR become much more efficient per sample than RS and HRS, with HNR scaling better than HRS and HRS better than RS. In planning efficiency, HNR performs comparably to RS/HRS in 6D but significantly better in 12D and 14D, whereas MH performs worst because of strong sample correlation and covariance sensitivity (Yi et al., 2017).
A plausible implication is that informed-set methods occupy an intermediate position in the taxonomy of directionally-pruned sampling. Their primary pruning criterion is not direction but admissible improvement. Yet in kinodynamic settings the informed set is typically anisotropic, skewed, or non-convex, and Hit-and-Run exploits that structure through random directional exploration.
6. Parameter-space directional pruning and its relevance to sampling
Two neural-network pruning papers supply the strongest formal use of “directional pruning,” although neither is a sampling method in the strict algorithmic sense. Both start from the premise that SGD tends to find a flat minimum valley of the training loss, and that sparsification should proceed within or close to that flat region so that training loss remains nearly unchanged (Chao et al., 2020, Li et al., 2021).
For unstructured pruning, the directional pruning objective is
$t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$7
with
$t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$8
where $t_k = \frac{k}{\text{NFE}, \quad k = 0,1,\dots,\text{NFE},$9 projects onto the flat subspace 0 (Chao et al., 2020). The coordinatewise solution is
1
Since computing flat directions directly is expensive, the practical algorithm uses a tuned generalized regularized dual averaging update,
2
with
3
The theorem states that, under assumptions and with 4, gRDA asymptotically performs directional pruning based on 5 (Chao et al., 2020).
For structured pruning, the analogous formulation is
6
where the direction factor 7 comes from orthogonal projection of groupwise normalized perturbations onto the flat minimum subspace (Li et al., 2021). The group solution is
8
The fast solver AltSDP alternates
9
and
0
with the same threshold scaling 1 (Li et al., 2021).
These works are not sampling papers. They do not define a sampling distribution over directions, masks, or subnetworks. Their relevance is foundational: they provide the strongest explicit notion of “directional pruning” in the supplied corpus, showing that the retained or removed parameters should be determined by alignment with flat directions rather than by magnitude alone. This suggests that any future directionally-pruned sampling scheme in parameter space would naturally use flat-subspace projections, Hessian geometry, or groupwise direction factors as proposal or importance mechanisms.
The empirical results reinforce the distinction between directional pruning and ordinary pruning. The unstructured method achieves about 73% top-1 accuracy with only 8% active parameters on ResNet50/ImageNet, i.e. about 92% sparsity, without retraining (Chao et al., 2020). The structured method reports 93.97% on VGG16/CIFAR-10 without retraining and shows that AltSDP and SGD can lie in the same minimum valley (Li et al., 2021). These are not sampling metrics, but they establish why “directional” matters: movement along flat directions preserves the objective more effectively than naive shrinkage.
7. Limits, definitions, and open interpretive boundaries
Across the cited literature, the most accurate encyclopedic characterization is that directionally-pruned sampling is not yet a single settled technical term. EPSS is a direct example of timestep pruning informed by trajectory geometry in generative inference (Zheng et al., 26 May 2025). Directed-graph sampling is a literal direction-aware sampling problem because the graph operator is asymmetric and the smoothness prior depends on directionality (Li et al., 2022). In motion planning, directionality is mostly implicit in the informed-set geometry and in Hit-and-Run proposals (Yi et al., 2017). In dataset pruning, geometry is often compressed into scalar scores or marginal gains, which makes the relation partial rather than exact (Grosz et al., 2024, Wu et al., 11 Jun 2026). In neural-network compression, the directionality is explicit but the methods are pruning optimizers rather than samplers (Chao et al., 2020, Li et al., 2021).
Several boundaries should therefore be kept explicit.
First, trajectory-aware is not the same as online direction-adaptive. EPSS uses a global, hand-crafted, static schedule inferred from observed trajectory shape, not per-sample runtime directional measurements (Zheng et al., 26 May 2025).
Second, geometry-aware is not the same as directionally explicit. SIMS uses own-center versus nearest-impostor geometry, but the pruning and sampling law are scalar (Grosz et al., 2024). UGIES can host directional discrepancies through the choice of 2, but its base graph is undirected and symmetric (Wu et al., 11 Jun 2026).
Third, direction-aware sampling does not require a frequency-domain theory. The directed-graph method avoids directed spectral ordering altogether by using graph shift variation and Gershgorin lower bounds (Li et al., 2022).
Fourth, pruning by admissible subsets is distinct from pruning by preferred directions. In informed motion planning, the primary mechanism is restricting to 3, even though the geometry of that set can be highly anisotropic (Yi et al., 2017).
A plausible synthesis is that future work on directionally-pruned sampling would combine three ingredients already present separately in the literature: explicit geometric or directional diagnostics, pruning or subset restriction tied to those diagnostics, and sampling or solver policies that exploit the resulting reduced domain. The cited papers demonstrate each ingredient in isolation or in pairs, but not yet as one unified formal doctrine.