Human-Guided Conditional Diffusion

• Human-guided conditional diffusion is a generative framework where human signals, including perceptual scores and structural priors, steer the reverse diffusion process.
• It uses diverse inputs like guide images, parsing masks, and physical constraints to condition the denoising step and improve output fidelity in tasks from imaging to pose reconstruction.
• By integrating learnable priors with optimization-based guidance, the approach balances realism and controllability while addressing challenges in iterative sampling and conflicting conditions.

to=arxiv_search.search 全民彩票天天送钱json {"query":"all:(\"human-guided conditional diffusion\" OR \"score-guided diffusion\" OR \"guided conditional diffusion\")", "max_results": 10, "sort_by": "relevance"} to=arxiv_search.search 天天中彩票和json {"query":"id:(Ueda et al., 2023) OR id:(Stathopoulos et al., 2024) OR id:(Li et al., 9 Sep 2025) OR id:(Kang et al., 15 Jul 2025) OR id:(Xu et al., 5 Feb 2025) OR id:(Zhong et al., 2022) OR id:(Zhang et al., 2024) OR id:(Guo et al., 5 Feb 2026)", "max_results": 20, "sort_by": "relevance"} Human-guided conditional diffusion denotes a family of diffusion-based generative and inverse-problem methods in which denoising is conditioned or steered by signals that encode human intent, human structure, human preference, or human-related constraints. In the literature, these signals range from guide images and class labels, human perceptual evaluations, clinical metadata, semantic part labels, parsing masks, trajectories, and textual descriptions to explicitly human-centric physical objectives such as contact, support, and collision avoidance. The unifying pattern is a diffusion prior—unconditional, conditional, or bridge-based—whose reverse dynamics are altered so that sampling remains plausible under the learned distribution while moving toward externally specified human criteria (Busaranuvong et al., 2024, Ueda et al., 2023, Stathopoulos et al., 2024, Li et al., 9 Sep 2025, Xu et al., 5 Feb 2025, Guo et al., 5 Feb 2026).

1. Conceptual scope and definitions

The term does not denote a single standardized formalism. Instead, the surveyed literature exhibits several mathematically distinct but conceptually related uses. In "HumanDiffusion" the target distribution is a human-acceptable distribution,

$p_{\text{human}}(x)=\frac{1}{Z}D(x),$

where $D(x)$ is a human perceptual evaluation of naturalness, so human judgment directly defines the density being sampled (Ueda et al., 2023). In "Guided Conditional Diffusion Classifier (ConDiff)" a real guide image $x_0$ is perturbed and then conditionally denoised under candidate labels, so classification is performed by comparing the original guide image with label-conditioned reconstructions in embedding space (Busaranuvong et al., 2024). In "Score-Guided Human Mesh Recovery" and "ScoreHOI", diffusion acts as a conditional prior over human pose or human-object interaction parameters, while denoising is further guided by reprojection, temporal, contact, floor-support, and penetration objectives (Stathopoulos et al., 2024, Li et al., 9 Sep 2025). In "TruePose", human parsing is not merely an auxiliary input but a mechanism for reweighting attention during diffusion-based pose transfer (Xu et al., 5 Feb 2025).

A second distinction concerns whether the “human” component is a human feedback signal or a human-centered structural prior. Binary expert preference labels are used as classifier-free guidance conditions in CBCT-to-MDCT translation, where they explicitly steer a Schrödinger Bridge sampler toward clinically preferred artifact suppression (Kang et al., 15 Jul 2025). By contrast, one-shot human image synthesis via 3D priors uses 3D normal maps and color prompts derived from SMPL as geometry and appearance conditions; this is human-centric conditioning rather than human preference feedback (Gong et al., 18 Jun 2026). Guided Motion Diffusion likewise focuses on controllable human motion synthesis conditioned by natural language and spatial constraints such as pre-defined motion trajectories, obstacles, and sparse keyframes, with feature projection, a new imputation formulation, and dense guidance from sparse signals (Karunratanakul et al., 2023).

This breadth suggests that “human-guided” is best understood as a spectrum. At one end, human evaluation defines the target energy or probability model directly; at the other, human-related structure—body models, parsing maps, trajectories, or metadata—defines the conditional variables through which diffusion operates.

2. Mathematical mechanisms of guidance

A recurrent formulation is to decompose a conditional score into a learned prior term and an externally specified guidance term. In ScoreHOI, denoising is guided by physical plausibility constraints $\mathcal{P}$ through

$$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$

with the second term approximated on the denoised prediction $\hat{\mathbf{x}}_0(\mathbf{x}_t)$, and the resulting gradient added to the noise prediction as an energy-guidance correction (Li et al., 9 Sep 2025). ScoreHMR uses the same logic for inverse problems in 3D human recovery, replacing contact energies with task losses such as 2D reprojection, multi-view consistency, or temporal smoothness (Stathopoulos et al., 2024).

Another important mechanism is classifier-free guidance. ConDiff trains a conditional diffusion model with label dropout and samples with

$$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$

while initializing the reverse process from a noisy version of a specific guide image rather than from pure noise (Busaranuvong et al., 2024). The CBCT-to-MDCT Schrödinger Bridge model uses the same principle with binary human feedback labels, combining a conditional score and an unconditional score through

$$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$

so that sampling can be pushed toward the “good” preference class without introducing a separate reward model (Kang et al., 15 Jul 2025).

A third family comprises training-free plug-and-play guidance. FreeDoM defines a time-independent energy $\mathcal{E}(c,x_0)$ using off-the-shelf networks and applies guidance through the gradient of $\mathcal{E}$ evaluated on the estimated clean sample $D(x)$0 (Yu et al., 2023). MPGD modifies this pattern by applying guidance in clean space and then performing a shortcut DDIM-style update, with optional manifold-preserving projection through a pretrained autoencoder in data or latent space (He et al., 2023). ADMMDiff goes further by decoupling generation and guidance into two variables,

$D(x)$1

and solving the resulting constrained problem with ADMM, treating the diffusion reverse step as an approximate proximal operator of the unconditional prior (Zhang et al., 2024).

The hard-constraint setting is different again. Under Doob’s $D(x)$2-transform, conditional generation under an event $D(x)$3 modifies the pretrained reverse dynamics to

$D(x)$4

where $D(x)$5. Here the constraint is not a soft preference but a target event that should hold with probability one in the ideal conditional law (Guo et al., 5 Feb 2026). Tempered Guided Diffusion occupies an intermediate position: it keeps the unconditional diffusion prior fixed, defines tempered posteriors

$D(x)$6

and allocates compute across multiple trajectories via sequential Monte Carlo rather than by modifying a single trajectory alone (Makris et al., 5 May 2026).

3. Sources of human guidance

The surveyed works differ primarily in the form of the external signal that enters the diffusion process.

Guidance source	Role in diffusion	Representative papers
Human perceptual score or preference label	Defines target density or CFG condition	(Ueda et al., 2023, Kang et al., 15 Jul 2025)
Guide image or reference appearance	Anchors conditional editing to a specific sample	(Busaranuvong et al., 2024, Xu et al., 5 Feb 2025, Gong et al., 18 Jun 2026)
Human-body structure, motion, or physical constraints	Steers denoising toward pose, contact, and feasibility	(Karunratanakul et al., 2023, Stathopoulos et al., 2024, Li et al., 9 Sep 2025)
Clinical metadata or semantic labels	Conditions denoising on structured human context	(Shi et al., 2024, Stone et al., 21 Sep 2025)
User-authored logical rules or hard events	Defines differentiable robustness or exact conditional law	(Zhong et al., 2022, Guo et al., 5 Feb 2026)

At the preference end, HumanDiffusion estimates both $D(x)$7 and $D(x)$8 from human evaluations, using periphery samples and NES-like finite differences to approximate a perceptual score field that is wider than the empirical data distribution (Ueda et al., 2023). The CBCT-to-MDCT Schrödinger Bridge model uses binary expert labels only—“good” and “bad”—but couples them to CFG and iterative fine-tuning, so the model internalizes clinically preferred outcomes through tournament-based selection of generated candidates (Kang et al., 15 Jul 2025).

At the structural-conditioning end, several methods guide diffusion with human-body or human-scene representations. Guided Motion Diffusion incorporates natural-language descriptions together with pre-defined motion trajectories, obstacles, and sparse keyframes, and explicitly addresses the problem that sparse signals can be ignored during reverse diffusion by turning them into denser guidance signals (Karunratanakul et al., 2023). ScoreHMR conditions a diffusion prior on image features and then guides sampling with losses defined on SMPL pose, camera projection, and temporal or multi-view relations (Stathopoulos et al., 2024). ScoreHOI extends this to joint human-object parameters, with contact masks, floor support, and SDF-based penetration avoidance as human-centric physical constraints (Li et al., 9 Sep 2025). TruePose uses human parsing maps to specify which source regions are semantically relevant for the target pose, and uses those masks to reweight attention so that clothing details and facial identity are preserved through pose transfer (Xu et al., 5 Feb 2025).

A different line of work treats structured human context as metadata. mbVDiT conditions latent-space diffusion on the observed portion of a microbiome profile and on patient metadata embedded by modality-specific MLPs, then fuses those embeddings through cross-attention in a DiT-like backbone (Shi et al., 2024). In 3D point-cloud generation, guided diffusion holds per-point semantic labels fixed while only diffusing geometry; because those labels can be specified by a human before sampling, the method supports direct semantic composition control over generated object parts (Stone et al., 21 Sep 2025). This suggests that “human guidance” can be instantiated by explicit user constraints, by expert annotations, or by structured human-side covariates, provided those signals are differentiably connected to denoising.

4. Architectural and training patterns

A common architectural pattern is to separate a pretrained prior from task-specific conditioning modules. ScoreHOI first trains an affordance-aware regressor and contact predictor, then freezes the image backbone and affordance network while training a diffusion model over the 331-dimensional HOI parameter vector $D(x)$9. Conditioning is injected through an IG-Adapter with an extra cross-attention block over image features and geometry/affordance features, plus timestep-dependent scaling and shifting of the state (Li et al., 9 Sep 2025). ScoreHMR similarly learns a task-agnostic image-conditional diffusion prior over SMPL pose and then reuses it across single-frame fitting, multi-view refinement, and video smoothing without retraining the prior itself (Stathopoulos et al., 2024).

Another recurring pattern is two-stage learning. ConDiff first fine-tunes a conditional latent diffusion backbone, then freezes it and trains a separate EfficientNet-B0 embedding network with triplet loss on real and synthetic images; the diffusion model is thus used as a conditional editor, and the embedding model as the discriminative decision rule (Busaranuvong et al., 2024). mbVDiT likewise pretrains a VAE on public microbiome datasets, freezes or reuses that latent model, and then trains a metadata-conditional latent diffusion model for imputation (Shi et al., 2024). In both cases, the separation of generative modeling and downstream decision-making reduces overfitting and allows the guidance signal to be modified without relearning the entire pipeline.

Human-guided diffusion also motivates specialized attention design. TruePose employs a human-parsing-aware Siamese architecture with dual identical UNets, a Human-Parsing-Guided Fusion Attention module, and a CLIP-Guided Attention Alignment module. The crucial intervention is not an auxiliary loss but attention reweighting itself: source features are reweighted by target-relevant parsing masks before cross-attending into the target denoising path, and low-attended regions are then refined through CLIP region embeddings (Xu et al., 5 Feb 2025). In one-shot novel-view and pose synthesis, a separate line uses a reference encoder and a noise encoder, injecting 3D normal maps and a target-view color prompt as image-like conditions, with layer-wise cross-attention at $x_0$0, $x_0$1, and $x_0$2 resolutions (Gong et al., 18 Jun 2026). These designs indicate that human-structured conditions are often too localized or too geometry-dependent to be handled well by simple global conditioning.

Training-free methods replace learned conditional modules with optimization layers around pretrained priors. FreeDoM computes guidance through differentiable energies evaluated on $x_0$3 and can combine text, segmentation, landmarks, identity, style, or low-pass constraints with no retraining of the diffusion model (Yu et al., 2023). MPGD adds autoencoder-based manifold projection to keep guidance tangent to an approximate data manifold (He et al., 2023). ADMMDiff and Tempered Guided Diffusion further replace single-trajectory heuristics with optimization or particle-based outer loops that adaptively balance prior realism and condition satisfaction (Zhang et al., 2024, Makris et al., 5 May 2026).

5. Representative applications and empirical behavior

Medical imaging provides several of the clearest demonstrations. ConDiff reports an accuracy of $x_0$4 and an $x_0$5-score of $x_0$6 for diabetic foot ulcer infection prediction, outperforming the compared state-of-the-art models by at least $x_0$7 while using guided conditional synthesis plus distance-based classification rather than a conventional discriminative head (Busaranuvong et al., 2024). In CBCT-to-MDCT translation, the Schrödinger Bridge model with human feedback achieves RMSE $x_0$8, SSIM $x_0$9, LPIPS $\mathcal{P}$0, and Dice $\mathcal{P}$1, and does so with only $\mathcal{P}$2 sampling steps; on the bad-case subset it reaches ARR $\mathcal{P}$3 and ARSR $\mathcal{P}$4 (Kang et al., 15 Jul 2025). In microbiome imputation, mbVDiT attains PCC $\mathcal{P}$5 on STAD, $\mathcal{P}$6 on COAD, and $\mathcal{P}$7 on HNSC, and is reported to be best across PCC, cosine similarity, RMSE, and MAE on all three cancer datasets (Shi et al., 2024).

Human-body reconstruction and interaction are another major domain. ScoreHMR improves HMR 2.0 on 3DPW from PA-MPJPE $\mathcal{P}$8 to $\mathcal{P}$9 in single-frame model fitting, and in video refinement reduces acceleration error on 3DPW from $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$0 to $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$1 while also lowering PA-MPJPE to $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$2 (Stathopoulos et al., 2024). ScoreHOI improves contact plausibility on BEHAVE from F-score $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$3 for CONTHO to $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$4, while also improving $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$5 from $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$6 to $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$7 and $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$8 from $$\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c},\mathcal{P}) = \nabla_{\mathbf{x}_t}\log p(\mathbf{x}_t\mid \mathbf{c}) + \nabla_{\mathbf{x}_t}\log p(\mathcal{P}\mid \mathbf{c},\mathbf{x}_t),$$9 to $\hat{\mathbf{x}}_0(\mathbf{x}_t)$0; it runs at $\hat{\mathbf{x}}_0(\mathbf{x}_t)$1 FPS, compared with $\hat{\mathbf{x}}_0(\mathbf{x}_t)$2 FPS for CHORE, and a reduced-iteration variant reaches $\hat{\mathbf{x}}_0(\mathbf{x}_t)$3 FPS with only a small drop in accuracy (Li et al., 9 Sep 2025). These results reflect the value of combining a learned conditional prior with task-specific human-body energies rather than relying on pure regression or pure iterative fitting.

Pose- and image-conditioned synthesis shows a related pattern. TruePose achieves LPIPS $\hat{\mathbf{x}}_0(\mathbf{x}_t)$4, SSIM $\hat{\mathbf{x}}_0(\mathbf{x}_t)$5, and PSNR $\hat{\mathbf{x}}_0(\mathbf{x}_t)$6 on DeepFashion $\hat{\mathbf{x}}_0(\mathbf{x}_t)$7, and LPIPS $\hat{\mathbf{x}}_0(\mathbf{x}_t)$8, SSIM $\hat{\mathbf{x}}_0(\mathbf{x}_t)$9, and PSNR $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$0 on DeepFashion $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$1, with clear gains over CFLD and PCDM; on the in-the-wild WPose benchmark it obtains LPIPS $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$2 and PSNR $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$3, substantially better than the compared diffusion baselines (Xu et al., 5 Feb 2025). One-shot human image synthesis with 3D priors reports, on RenderPeople novel view, PSNR $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$4, SSIM $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$5, LPIPS $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$6, and FID $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$7, and on RenderPeople novel pose, PSNR $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$8 and LPIPS $$\tilde{\epsilon}_\theta(x_t,t,y) = (1-\omega)\,\epsilon_\theta(x_t,t) + \omega\,\epsilon_\theta(x_t,t,y),$$9, outperforming SHERF, Champ, and PIDM on the listed metrics (Gong et al., 18 Jun 2026).

The same logic extends beyond explicitly human imagery. In controllable traffic generation, user-defined STL rules such as speed limits, goal waypoints, collision avoidance, off-road avoidance, and stop-sign compliance act as differentiable logical guidance. Under the speed-limit rule, CTG reports rule violation $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$0, realism deviation $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$1, and fail $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$2, compared with $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$3, $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$4, and $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$5 for BITS+opt, illustrating the controllability–realism tradeoff that rule-guided diffusion can navigate more effectively than purely optimization-based baselines (Zhong et al., 2022). This supports a broader interpretation of human-guided diffusion as externally constrained conditional generation rather than only human-image synthesis.

6. Limitations, tensions, and emerging directions

A persistent tension is between guidance strength and fidelity to the learned prior. CTG explicitly reports a controllability–realism tradeoff: stronger STL guidance reduces rule violation but can worsen realism deviation and failure rate (Zhong et al., 2022). FreeDoM observes that large guidance strength $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$6 can overshoot and produce artifacts or off-manifold images, especially when multiple conditions conflict (Yu et al., 2023). ADMMDiff reframes this as a balancing problem between a prior term and a guidance term, which it solves with dual variables instead of a fixed scalar weight, but the need to tune $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$7, $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$8, and the number of inner optimization steps remains (Zhang et al., 2024). This suggests that human-guided diffusion is not only a modeling problem but also a control problem over the reverse trajectory.

A second limitation is dependence on external priors and intermediate predictors. ScoreHOI depends on predefined canonical object templates and on the quality of contact masks; under extreme occlusion, wrong masks can misguide optimization (Li et al., 9 Sep 2025). TruePose depends on accurate human parsing, since parsing masks directly reweight attention (Xu et al., 5 Feb 2025). One-shot human image synthesis is limited by SMPL estimation quality, and performance degrades when ground-truth SMPL parameters are replaced by HMR2.0 estimates (Gong et al., 18 Jun 2026). Hard-constraint guidance under Doob’s $$\tilde{\mathbf{s}}(\mathbf{z}_t) = (1+w)\,\mathbf{s}_\theta(\mathbf{z}_t\mid \mathbf{z}_0,t,r) - w\,\mathbf{s}_\theta(\mathbf{z}_t\mid \varnothing),$$9-transform avoids heuristic weighting, but its guarantees depend on estimating $\mathcal{E}(c,x_0)$0 and $\mathcal{E}(c,x_0)$1 accurately, which becomes difficult for rare events or noisy labels (Guo et al., 5 Feb 2026).

Sampling cost remains substantial. ConDiff requires about $\mathcal{E}(c,x_0)$2 minutes for $\mathcal{E}(c,x_0)$3 test images on an A100, approximately $\mathcal{E}(c,x_0)$4 seconds per image, compared with less than $\mathcal{E}(c,x_0)$5 seconds total for the compared CNN/ViT baselines (Busaranuvong et al., 2024). ScoreHOI is much faster than Adam-based optimizers but still heavier than pure regression (Li et al., 9 Sep 2025). By contrast, the Schrödinger Bridge CBCT system shows that strong human-guided performance can be obtained with $\mathcal{E}(c,x_0)$6 sampling steps when the bridge structure is analytically exploited (Kang et al., 15 Jul 2025). A plausible implication is that future work will increasingly combine stronger inductive structure—bridges, manifolds, latent priors, or exact constraints—with preference guidance, in order to reduce the cost of iterative denoising.

Several papers point toward richer forms of interaction. ScoreHOI explicitly suggests modifying contact masks or adding user-specified constraints such as “the right hand must touch the handle,” and also suggests language-guided HOI, multi-person and multi-object interaction, and temporal diffusion over motion sequences (Li et al., 9 Sep 2025). The structured point-cloud work suggests interactive editing by painting or manipulating per-point labels, and proposes soft labels, continuous guidance strength, and hierarchical semantics as open directions (Stone et al., 21 Sep 2025). ConDiff identifies adversarial diffusion distillation such as SDXL Turbo as a promising route to one-step sampling, which would make guide-image-based decision pipelines much more practical (Busaranuvong et al., 2024). Together these works indicate that the field is moving from static conditional generation toward iterative, constraint-rich, and potentially interactive human guidance over diffusion trajectories.