Soft or probabilistic supervision as partial responsibility constraints

Develop a formal treatment of soft or probabilistic supervision within the implicit expectation-maximization framework by modeling labels as partial constraints on responsibilities, extending the constrained regime beyond hard labels and providing a unified approach to semi-supervised learning and label smoothing.

Background

In the paper’s constrained regime (cross-entropy), responsibilities are clamped to targets via hard labels, which may be unrealistic when labels are noisy, partial, or uncertain.

A principled extension that treats supervision as partial constraints on responsibilities would align training with real-world label conditions and could unify several practical techniques (semi-supervised learning, label smoothing, crowd-sourced labels) under the implicit EM perspective.

References

Several directions remain open. The constrained regime analysis assumes hard labels that clamp responsibilities exactly; a fuller treatment would model soft or probabilistic supervision as partial constraints on the responsibility structure.

— Gradient Descent as Implicit EM in Distance-Based Neural Models  (2512.24780 - Oursland, 31 Dec 2025) in Discussion, Open Directions (Section 7, Open Directions)