Quality-Filtered Trajectory Distillation

• Quality-Filtered Trajectory Distillation is a framework that filters teacher trajectories based on quality to better match the student model's capacity.
• It adapts various methods like adaptive step selection, latent brightness filtering, and distribution-level alignment across multiple domains.
• Practical implementations demonstrate improvements in efficiency, performance metrics, and robustness in tasks such as dataset distillation and diffusion modeling.

Quality-Filtered Trajectory Distillation denotes a family of distillation procedures in which a student model, a synthetic dataset, or a downstream generator is trained against teacher trajectories, while the supervisory signal is restricted, reweighted, or regularized so that only trajectories, trajectory segments, or latent states judged to be high-quality dominate optimization. In the cited literature, the trajectory may be a parameter-space training path in dataset distillation, a decoding trajectory in diffusion LLMs, a PF-ODE denoising path in diffusion and rectified-flow models, or a forecast trajectory in motion prediction and text-to-3D generation (Wang et al., 2024, Zhou et al., 10 May 2026, Luo et al., 9 Mar 2025, Wang et al., 2023). Quality filtering may be explicit, as in data pruning, temporal partitioning, brightness masking, or PF-ODE segmentation, or implicit, as in objectives that favor smooth, self-consistent, mode-seeking, or trajectory-aligned behavior (Li et al., 2024, Tang et al., 25 Nov 2025, Shen et al., 2023).

1. Conceptual scope

A common core appears across otherwise disparate applications. WiDistill compresses a large Wi-Fi CSI dataset $\mathcal{D}_{\text{real}}$ into a much smaller synthetic dataset $\mathcal{D}_{\text{syn}}$ with $|\mathcal{D}_{\text{syn}}| = spc \times C$, and optimizes $\mathcal{D}_{\text{syn}}$ so that training on it reproduces the expert model’s parameter trajectory on the full dataset (Wang et al., 2024). ATT identifies the Accumulated Mismatching Problem (AMP) in fixed-length long-range dataset distillation and replaces a fixed matching step with an adaptive step

$$N_{\mathrm{opt}} = \arg\min_t \|\theta'_{i,t} - \theta^*_{i,N_T}\|_2^2,$$

thereby filtering trajectory supervision by alignment quality (Liu et al., 2024). TAD constructs teacher decoding trajectories for diffusion LLMs under privileged information, partitions masked positions into near and distant subsets according to how many decoding steps remain, and applies hard cross-entropy to near tokens and soft KL divergence to distant tokens (Zhou et al., 10 May 2026). TDM matches the distribution of student few-step trajectories to the teacher’s diffusion distributions across time, rather than forcing pointwise trajectory identity, and makes the objective sampling-steps-aware so that one student can support multiple deterministic step budgets (Luo et al., 9 Mar 2025).

These examples show that “quality-filtered” does not denote a single algorithmic primitive. It denotes a design principle: trajectory information is useful only insofar as it is compatible with the student’s capacity, the deployment sampler, the domain’s structure, and the intended fidelity–efficiency trade-off. This suggests that quality filtering is best understood as a control layer on trajectory distillation rather than as a replacement for trajectory distillation itself.

2. Canonical mathematical forms

In parameter-space trajectory matching, the teacher supplies a recorded sequence of parameters

$\{\theta_t^{D_{\text{real}}}\}_{t=0}^T,$

and the student is initialized from a teacher checkpoint and unrolled on distilled data. WiDistill uses the normalized trajectory loss

$$L_{\text{traj}}= \frac{\left\|\theta^{D_{\text{syn}}}_{t_0+K}-\theta^{D_{\text{real}}}_{t_0+J}\right\|_2^2} {\left\|\theta^{D_{\text{real}}}_{t_0}-\theta^{D_{\text{real}}}_{t_0+J}\right\|_2^2},$$

with gradients backpropagated through $K$ student updates to optimize both the synthetic CSI values and a trainable learning rate $\alpha$ (Wang et al., 2024). AST keeps the same trajectory-matching paradigm but argues that expert smoothness materially affects student alignment, and therefore regularizes expert generation so that average parameter variation

$$\text{Avg\_Var}=\mathbb{E}_t\left[\|\theta_t-\theta_{t+1}\|_2^2\right]$$

remains moderate while expert accuracy stays high (Shen et al., 2023).

In sequence generation, the trajectory is not a weight path but a decoding path. TAD defines a teacher token-by-token joint distribution

$\mathcal{D}_{\text{syn}}$0

and distills it into a factorized student by temporally splitting masked positions into $\mathcal{D}_{\text{syn}}$1 and $\mathcal{D}_{\text{syn}}$2, with

$\mathcal{D}_{\text{syn}}$3

The loss uses hard labels where the teacher is about to reveal a token and soft distributions where the teacher is still planning further ahead (Zhou et al., 10 May 2026).

In diffusion models, the trajectory is a denoising path. TDM replaces pointwise teacher-student trajectory matching with distribution-level alignment:

$\mathcal{D}_{\text{syn}}$4

where student trajectories are sampled deterministically and then diffused to intermediate times $\mathcal{D}_{\text{syn}}$5 so that the student’s trajectory distribution can be matched to the teacher’s diffusion distributions via score distillation (Luo et al., 9 Mar 2025). SegmentDreamer introduces a segment-indexed consistency function

$\mathcal{D}_{\text{syn}}$6

and enforces consistency only within PF-ODE sub-intervals $\mathcal{D}_{\text{syn}}$7, thereby localizing the distillation target (Zhu et al., 7 Jul 2025).

These formulations differ in object type, but they share one structure: a teacher trajectory defines target dynamics; a student trajectory is induced by a compressed substrate; and a quality-aware loss determines which parts of the teacher dynamics should be trusted, emphasized, or ignored.

3. Mechanisms of quality filtering

Quality filtering appears in at least five distinct forms. First, it can be implemented by trajectory construction under privileged or regularized conditions. TAD conditions the teacher on both prompt and ground-truth answer, retains only valid trajectories, and reports that this “with GT” setting raises training-task correctness for trajectory construction on GSM8K from $\mathcal{D}_{\text{syn}}$8 to $\mathcal{D}_{\text{syn}}$9 for LLaDA and from $|\mathcal{D}_{\text{syn}}| = spc \times C$0 to $|\mathcal{D}_{\text{syn}}| = spc \times C$1 for Dream, before subsequent filtering to roughly $|\mathcal{D}_{\text{syn}}| = spc \times C$2k LLaDA and $|\mathcal{D}_{\text{syn}}| = spc \times C$3k Dream samples (Zhou et al., 10 May 2026). AST smooths expert trajectories with clipped cross-entropy and gradient penalty so that high expert accuracy is not achieved at the price of student-incompatible parameter jumps (Shen et al., 2023).

Second, it can be implemented by adaptive segment or step selection. ATT directly searches over student trajectory positions and uses the closest one for supervision, turning trajectory length into a per-expert, per-iteration quality decision rather than a fixed hyperparameter (Liu et al., 2024). SegmentDreamer partitions the PF-ODE trajectory into sub-trajectories and proves that the local distillation error in a segment scales as $|\mathcal{D}_{\text{syn}}| = spc \times C$4, tightening the bound relative to global consistency formulations (Zhu et al., 7 Jul 2025).

Third, it can be implemented by sample or parameter pruning. PAD prunes the target dataset according to compressing ratio using EL2N difficulty and retains only deep layers of the agent model for matching, arguing that shallow layers contribute low-level, redundant signals while deep layers carry the high-level semantic information more suitable for distilled datasets (Li et al., 2024). In the mapless forecasting setting, feature and output distillation are weighted by learned variances, so difficult feature dimensions or poorly matched output modes are effectively downweighted rather than forced, which serves as a probabilistic form of quality filtering (Wang et al., 2023).

Fourth, it can be implemented by trajectory-state filtering. TBCM operates entirely in generation space rather than diffusion space and introduces a brightness filter in latent space to mask trajectories whose final latents are too close to the latent of a fully black image, thereby discarding low-brightness, low-quality teacher trajectories from student training (Tang et al., 25 Nov 2025). This is an explicitly hard filter. By contrast, WiDistill performs no explicit sample selection; synthetic CSI values are continuously modified so that samples that improve trajectory alignment are reinforced and those that hurt alignment are reshaped (Wang et al., 2024).

Fifth, it can be implemented by objective-level geometric filtering. TraFlow enforces endpoint fidelity, straightness, and self-consistency simultaneously through output reconstruction, a velocity-alignment term, and a consistency loss. In that formulation, trajectories that are endpoint-accurate but non-straight, or self-consistent but poor at reaching teacher outputs, are penalized; quality is therefore the intersection of three geometric criteria rather than a single distance threshold (Wu et al., 24 Feb 2025).

A recurrent misconception is that quality filtering must mean hard rejection of bad samples. The literature shows both hard and soft variants. Hard filtering appears in ATT’s step selection and TBCM’s brightness mask; soft filtering appears in TAD’s hard/soft near–distant supervision, PAD’s deep-layer masking, AST’s smoothness regularization, and TraFlow’s combined geometric constraints (Liu et al., 2024, Tang et al., 25 Nov 2025, Zhou et al., 10 May 2026, Li et al., 2024, Shen et al., 2023, Wu et al., 24 Feb 2025).

4. Major domains and representative instantiations

The phrase spans multiple technical domains, and the trajectory object changes accordingly.

Domain	Trajectory object	Representative quality filter
Dataset distillation	Parameter-space training path	Adaptive step choice, deep-layer matching, smooth expert trajectories
Diffusion LLMs	Decoding trajectory over masked states	Near/distant token partition, reverse-KL mode seeking
Diffusion and rectified flow	PF-ODE or denoising trajectory	Trajectory-distribution matching, self-consistency, brightness filtering
Motion and 3D generation	Forecast or PF-ODE latent trajectory	Privileged teacher signals, segment-wise consistency, variational reweighting

In Wi-Fi human activity recognition, WiDistill applies trajectory matching to CSI-based datasets such as Widar3.0, XRF55, and MM-Fi, reducing dataset size while preserving training dynamics and showing robust performance in cross-network environments (Wang et al., 2024). In image dataset distillation, ATT treats the matching step as a variable to avoid AMP, PAD filters the real dataset and the agent parameters before matching, AST regularizes the expert trajectory itself, and progressive trajectory matching for medical data uses a curriculum over trajectory length plus a dynamic overlap mitigation module to stabilize training and preserve synthetic diversity (Liu et al., 2024, Li et al., 2024, Shen et al., 2023, Yu et al., 2024).

In autonomous-driving motion forecasting, a map-based teacher distills map-consistent trajectory knowledge into a mapless student through feature and output distillation, with learned variances acting as dimension-wise or mode-wise quality controls (Wang et al., 2023). In long-horizon trajectory prediction, Di-Long distills a short-term teacher that observes longer context and predicts a shorter remaining horizon; the shorter horizon is treated as less uncertain, so distillation is effectively confined to the more reliable part of the trajectory (Das et al., 2023).

In diffusion LLMs, TAD and T3D are complementary examples. TAD filters teacher trajectories temporally and uses mixed hard/soft supervision conditioned on how soon a token will be revealed (Zhou et al., 10 May 2026). T3D uses teacher-generated trajectories under full-step decoding and applies Direct Discriminative Optimization, a reverse-KL-style objective that concentrates student mass on high-probability teacher modes, with an additional path-consistency term that weights earlier decoded tokens more heavily (Zhang et al., 12 Feb 2026).

In diffusion image, video, and text-to-3D generation, TDM, TraFlow, TBCM, Hierarchical Distillation, and SegmentDreamer all treat quality filtering as a control problem on denoising paths: match trajectory distributions rather than exact paths, enforce self-consistency and straightness, filter teacher-generated states in generation space, use trajectory distillation as a structure-preserving sketch before detail refinement, or partition the PF-ODE into shorter segments with tighter consistency constraints (Luo et al., 9 Mar 2025, Wu et al., 24 Feb 2025, Tang et al., 25 Nov 2025, Cheng et al., 12 Nov 2025, Zhu et al., 7 Jul 2025).

5. Empirical behavior, benefits, and failure modes

Reported benefits are substantial but domain-specific. In diffusion LLMs, TAD improves the accuracy–parallelism trade-off on LLaDA from average accuracy $|\mathcal{D}_{\text{syn}}| = spc \times C$5 to $|\mathcal{D}_{\text{syn}}| = spc \times C$6 with the Quality model and from AUP $|\mathcal{D}_{\text{syn}}| = spc \times C$7 to $|\mathcal{D}_{\text{syn}}| = spc \times C$8 with the Speed model; on Dream, it raises accuracy from $|\mathcal{D}_{\text{syn}}| = spc \times C$9 to $\mathcal{D}_{\text{syn}}$0 for the Quality model and AUP from $\mathcal{D}_{\text{syn}}$1 to $\mathcal{D}_{\text{syn}}$2 for the Speed model (Zhou et al., 10 May 2026). T3D shows that mode-seeking trajectory self-distillation can substantially narrow the few-step gap while largely preserving full-step behavior, and that naive forward-KL trajectory distillation can damage full-step performance where the DDO-based objective does not (Zhang et al., 12 Feb 2026).

In diffusion image and video generation, TBCM reports one-step performance of $\mathcal{D}_{\text{syn}}$3 FID and $\mathcal{D}_{\text{syn}}$4 CLIP on MJHQ-30k while reducing training time by approximately $\mathcal{D}_{\text{syn}}$5 compared to Sana-Sprint and cutting GPU memory substantially (Tang et al., 25 Nov 2025). TDM distills PixArt-$\mathcal{D}_{\text{syn}}$6 into a 4-step generator that outperforms its teacher on real user preference at $\mathcal{D}_{\text{syn}}$7 resolution using $\mathcal{D}_{\text{syn}}$8 iterations and $\mathcal{D}_{\text{syn}}$9 A800 hours, and also improves CogVideoX-2B on VBench from $$N_{\mathrm{opt}} = \arg\min_t \|\theta'_{i,t} - \theta^*_{i,N_T}\|_2^2,$$0 to $$N_{\mathrm{opt}} = \arg\min_t \|\theta'_{i,t} - \theta^*_{i,N_T}\|_2^2,$$1 with only $$N_{\mathrm{opt}} = \arg\min_t \|\theta'_{i,t} - \theta^*_{i,N_T}\|_2^2,$$2 NFE (Luo et al., 9 Mar 2025). Hierarchical Distillation reports that trajectory-based methods preserve global structure but act as a “lossy compressor,” whereas a subsequent distribution-based refinement stage can restore detail and reach teacher-level or near-teacher-level quality in a single step (Cheng et al., 12 Nov 2025).

In dataset distillation, AST consistently improves over MTT and FTD on CIFAR-10, CIFAR-100, and Tiny ImageNet; its motivating evidence is that expert accuracy can increase while distilled performance collapses if the expert trajectory becomes too steep under momentum or Adam, making smoothness itself a quality criterion (Shen et al., 2023). PAD reports that filtering the target dataset by difficulty and restricting matching to deep layers yields state-of-the-art results across multiple IPC regimes, and that removing shallow-layer parameters is beneficial while removing deep-layer parameters is harmful (Li et al., 2024). Progressive trajectory matching for medical dataset distillation reports an average improvement of $$N_{\mathrm{opt}} = \arg\min_t \|\theta'_{i,t} - \theta^*_{i,N_T}\|_2^2,$$3 over previous state of the art, and $$N_{\mathrm{opt}} = \arg\min_t \|\theta'_{i,t} - \theta^*_{i,N_T}\|_2^2,$$4 improvement when $$N_{\mathrm{opt}} = \arg\min_t \|\theta'_{i,t} - \theta^*_{i,N_T}\|_2^2,$$5 (Yu et al., 2024).

The failure modes are equally consistent. ATT attributes one family of failures to AMP: fixed trajectory length forces the student to match at a non-optimal step, accumulates mismatch over distillation iterations, and particularly harms cross-architecture generalization (Liu et al., 2024). SegmentDreamer identifies a different failure mode in CD-based score distillation: if self-consistency and cross-consistency are imbalanced, the result is either weak conditional guidance or over-guidance with overexposure and artifacts (Zhu et al., 7 Jul 2025). In mapless forecasting and long-horizon prediction, teacher quality and privileged-input quality remain limiting factors; poor teacher predictions or weak scene signals can propagate bias or reduce the effectiveness of distillation (Wang et al., 2023, Das et al., 2023).

A second misconception is that more global matching always implies better quality. The surveyed results do not support that view. Global hard labeling in TAD degenerates to a less favorable regime than temporally localized hard/soft supervision; fixed-length global matching in ATT creates AMP; and full-trajectory consistency without segmentation in text-to-3D produces looser error bounds than segment-wise consistency (Zhou et al., 10 May 2026, Liu et al., 2024, Zhu et al., 7 Jul 2025).

6. Design principles and open directions

Several recurring design principles emerge directly from the cited methods. High-quality teacher trajectories are rarely obtained for free. TAD uses privileged information and validity filtering; AST regularizes expert trajectories; TBCM constructs training pairs directly from the teacher’s backward generation trajectory to bridge the diffusion-generation space discrepancy; and WiDistill normalizes trajectory loss by expert movement so that late-stage near-stationary epochs do not produce misleadingly small losses (Zhou et al., 10 May 2026, Shen et al., 2023, Tang et al., 25 Nov 2025, Wang et al., 2024).

The form of the quality signal is strongly domain-dependent. In ATT it is parameter-space distance to an expert endpoint; in PAD it is sample difficulty and layer depth; in TAD it is temporal distance to token revelation; in TBCM it includes trajectory sampling strategy and latent brightness; in TraFlow it is endpoint fidelity, straightness, and self-consistency; and in operator-merging theory it is Wasserstein-2 signal fidelity relative to a surrogate teacher composite (Liu et al., 2024, Li et al., 2024, Zhou et al., 10 May 2026, Tang et al., 25 Nov 2025, Wu et al., 24 Feb 2025, Gao et al., 21 May 2025).

Theoretical work further suggests that the optimal amount of trajectory compression depends on data geometry. The operator-merging analysis of diffusion trajectory distillation shows a phase transition governed by covariance structure: low-variance regimes favor sequential BOOT-like merging, large-variance regimes favor vanilla one-shot merging, and mixed spectra induce hybrid optimal merge plans (Gao et al., 21 May 2025). This suggests that “quality filtering” can also be spectral: one filters merge aggressiveness by signal direction rather than only by timestep or sample.

Taken together, these works suggest three open directions. First, explicit quality scoring remains underdeveloped: several papers propose confidence-weighted, uncertainty-based, or curriculum-based extensions, but most current systems still rely on implicit criteria such as temporal locality, smoothness, or reverse-KL mode seeking (Wang et al., 2023, Wu et al., 24 Feb 2025). Second, quality filtering and diversity control remain in tension: PAD removes misaligned low-level information, AST and medical progressive matching add stabilization, while Hierarchical Distillation and medical overlap mitigation reintroduce diversity or detail through additional stages (Li et al., 2024, Shen et al., 2023, Yu et al., 2024, Cheng et al., 12 Nov 2025). Third, broader generalization remains unresolved: several papers note overfitting to teacher architecture, teacher schedule, or teacher data regime, and propose ensembles, cross-architecture matching, or multi-domain trajectories as mitigations rather than complete solutions (Wang et al., 2024, Gao et al., 21 May 2025).

Quality-Filtered Trajectory Distillation is therefore best understood not as a single method, but as a trajectory-centric doctrine for compression: the teacher trajectory is the supervision substrate, and quality control determines which parts of that substrate are preserved, softened, segmented, normalized, or discarded. Across dataset distillation, diffusion modeling, language generation, forecasting, and 3D synthesis, the common thesis is that compressed learning systems do not merely need shorter trajectories; they need trajectories whose information content is filtered to match what the student can reliably absorb.