High-dimensional behavior of CFG weighting toward private regions

Establish whether, for any dimension D ≥ 2 in masked discrete diffusion models with classifier-free guidance (CFG), the sampled distribution q_T^{z_1,w} under Assumption 1 (the full data distribution p is a mixture of class-conditional distributions {p(·|z_k)} with weights {a_k}) consistently places larger weights on more private regions of the target class z_1, where privacy is defined via non-overlap of supports and marginal supports with other classes, thereby leveraging the geometric information of the full data distribution.

Background

The paper derives explicit formulas for guided reverse dynamics in masked discrete diffusion and characterizes how classifier-free guidance reshapes sampled distributions in low-dimensional settings. In 1D, the guided sampling distribution matches the tilted distribution exactly, while in 2D, the sampled distribution deviates from the tilt and reweights probability mass depending on overlap and marginal overlap with other classes, with larger weights in regions unique to the target class.

Based on these 2D insights, the authors hypothesize a general high-dimensional pattern: that CFG leverages geometric information (supports and marginal supports) to emphasize private regions of the target class. This conjecture seeks to formalize whether the same weighting behavior holds for any D ≥ 2 under the mixture-model assumption.