Principled volume control for distance-based log-sum-exp objectives

Develop a principled mechanism for volume control in neural training under distance-based log-sum-exp objectives by either deriving implicit covariance-volume terms from architectural components or designing objectives that include explicit log-determinant-like penalties, in order to prevent collapse of the learned metric during implicit expectation-maximization dynamics.

Background

The paper highlights a collapse risk inherent to distance-based objectives optimized with softmax/log-sum-exp: unlike Gaussian mixture models that include a log-determinant term to penalize shrinking component volumes, standard neural losses lack explicit volume control. This absence can allow a component (or metric) to degenerate, capturing probability mass through positive feedback in responsibility-weighted updates.

The authors note that practical training often relies on heuristic stabilizers (e.g., weight decay, normalization layers), but these are not principled replacements for a theoretically motivated volume term. A formal approach to embedding volume control in the objective or deriving it from architectural elements would make the implicit EM dynamics more robust and theoretically grounded.