No-Clipping-Point Policy

• No-Clipping-Point Policy is a strategy for ensuring prediction–loss alignment that stabilizes training on discrete binary manifolds.
• It contrasts signal prediction with velocity matching, detailing when to apply x-loss (MSE/BCE) versus v-loss to control gradient divergence.
• Empirical results on Binary MNIST and MIMO detection demonstrate that proper alignment significantly improves convergence and performance.

Below is a consolidated, self‐contained summary of the paper’s theoretical and empirical developments on manifold‐aligned prediction targets, organized to address your five points. Where helpful, key LaTeX formulas are included in‐line.

1. Definitions of x‐prediction and velocity‐based v‐loss on binary manifolds

• Forward interpolation on a standardized binary manifold We map each bit x∈{−1,1}^D into “analog” form and define z_t = t * x + (1−t)·ε with ε∼N(0,I), t∈[0,1]. The true instantaneous velocity is u_t = ẋ_t = x − ε.
• Velocity‐matching loss (v‐loss) Standard flow matching directly parameterizes v_θ(z_t,t)≈u_t and minimizes L_vel(θ) = E_{t,x,ε} [‖v_θ(z_t,t) − (x−ε)‖²] .
• Signal‐prediction (x‐prediction) Instead of predicting velocity, predict the clean signal: x_pred = ˆx_θ(z_t,t) . Then algebraically recover velocity as v_θ(z_t,t) := (x_pred − z_t)/(1−t).
• Induced v‐loss under x‐prediction Substituting v_θ yields the mismatched loss L_vel(θ) = E_{t,x,ε} [ ‖ (x_pred − z_t)/(1−t) − (x−ε) ‖² ] = E_{t,x,ε} [ (1−t)^{−2}‖x_pred − x‖² ] . We write the singular weight λ(t) = (1−t)^{−2}.

2. Prediction–loss alignment condition and singular weighting

• Alignment condition “Prediction‐loss alignment’’ means the model’s prediction target and its loss space coincide. Concretely, one must use either
  • v‐prediction + v‐loss, or
  • x‐prediction + x‐loss. The problematic mix is x‐prediction + v‐loss.
• Mismatch derivation of singular gradient Under x‐pred + v‐loss, the per‐t loss is ℓt(θ) = λ(t)·‖δ‖², δ = x_pred(z_t,t)−x. Its gradient w.r.t. θ is g_t(θ) = ∂θℓt = 2λ(t)·δ^⊤·Jθ, J_θ=∂θ x_pred . Hence ‖g_t(θ)‖² = 4 λ(t)² ‖δ^⊤Jθ‖² ≳ c·(1−t)^{−4} R(t) by a non‐degeneracy assumption ‖δ^⊤J_θ‖² ≥ c‖δ‖², and R(t)=E[‖δ‖²].
• Divergence of gradient variance Define the integrated second moment I = ∫₀¹ E[‖g_t‖²] dt ≳ ∫₀¹ (1−t)^{−4} R(t) dt. On a binary manifold R(t)→Ω(1) as t→1, so the integrand ∼(1−t)^{−4} and I diverges like (1−t)^{−3}, making uniform‐time training catastrophically unstable.

3. Proof sketch of how x‐loss alignment removes the singularity

• Signal‐space (x‐loss) objective Define aligned loss L_x(θ) = E_{t,x,ε}[‖x_pred(z_t,t) − x‖²] with no extra λ(t) factor. The instantaneous gradient is g_t(θ) = 2(x_pred − x)^⊤·J_θ.
• Uniform boundedness under mild assumptions
  • Lipschitz capacity: ‖J_θ‖≤K.
  • Signal in [−1,1]^D ⇒ ‖x_pred−x‖ ≤ 2√D. ⇒ ‖g_t‖ ≤ 4K√D for all t. Hence I = ∫₀¹ E[‖g_t‖²] dt = O(1), so uniform‐time sampling is stable without any timestep‐clipping or importance sampling distribution.

4. Topology‐induced choice between probabilistic vs. geometric losses on {−1,1}^D

Once alignment is secured (so that numerical stability is guaranteed), the choice of loss becomes a modeling decision reflecting signal topology:

• Binary Cross‐Entropy (BCE) – local, factorized probability Interpret each coordinate x_i∈{−1,1} via p_i=σ(a_i) where a_i is the logit output. L_BCE(x,a) = − Σ_i [ (1+x_i)/2 * log p_i + (1−x_i)/2 * log(1−p_i) ]. This assumes conditional independence of bits given the model output and is ideal for i.i.d. symbolic data (e.g. MIMO symbols).
• Mean Squared Error (MSE) – global, geometric embedding L_MSE(x,ˆx) = ‖x − ˆx‖². This is equivalent to an isotropic Gaussian likelihood over the entire vector and preserves Euclidean distances and spatial correlations, making it preferable for structured images (e.g. Binary MNIST).

1. Practical guidelines for discrete generative modeling

   1. Always enforce prediction–loss alignment:
      • If you predict velocity v_θ, use L_vel.
      • If you predict signal x_θ, use L_x (MSE or BCE). Avoid x‐prediction + velocity‐loss under uniform time sampling.
   2. Sampling schedule:
      • With proper alignment, you may safely sample t∼Uniform[0,1].
      • Heuristic schedules (Logit‐Normal) may temporarily mask instability but do not resolve the structural singularity.
   3. Loss based on data topology:
      • For spatially correlated binary images, use x‐prediction + MSE.
      • For i.i.d. binary sequences (e.g. digital communications), use x‐prediction + BCE.
   4. Empirical evidence:
      • Toy experiments show gradient norms of x‐pred+v‐loss blow up under uniform sampling but remain bounded under x‐pred+x‐loss.
      • On Binary MNIST, aligned x‐pred+MSE yields the lowest FID and monotonic convergence, whereas x‐pred+v‐loss is unstable even with Logit‐Normal sampling.
      • In MIMO detection, aligned x‐pred+BCE outperforms misaligned objectives in bit‐error‐rate under uniform‐time training.

Together, these principles—prediction–loss alignment plus topology‐aware loss choice—form the core of “manifold‐aligned prediction target design,” ensuring both numerical robustness and fidelity to the discrete data’s geometric or probabilistic structure.