Core–wall mechanism in broader underparameterized landscapes

Determine whether the core–wall exponential-sum mechanism explaining quadratic cores, superquadratic walls, and outer rollovers in confident cross-entropy loss slices also explains basin- and valley-like loss landscapes in broader underparameterized neural-network settings.

Background

The paper proposes that confident cross-entropy produces a directional core–wall landscape: an approximately quadratic core near a minimum, a superquadratic wall at finite displacement, and an outer rollover. This geometry is used to explain the near-linear Adam stability boundary observed in several model–task settings.

The authors explicitly leave unresolved whether the same exponential-sum mechanism applies to broader basin- and valley-like landscapes beyond the settings studied in the paper, particularly in underparameterized neural networks.