Exhaustiveness of architecture-induced fixed points and invariant manifolds

Determine whether the fixed points and invariant manifolds identified in this work—particularly those associated with Theorem 3 and related constructions—exhaust all architecture-induced fixed points and invariant manifolds for networks defined by Equation (1), and specify conditions under which such exhaustiveness holds.

Background

Sections 3 and 4 construct embedded fixed points and invariant manifolds arising from permutation symmetry across units. While these structures explain saddle-to-saddle dynamics across architectures, it is unclear whether they account for all such stationary structures permitted by the architecture.

Resolving exhaustiveness would turn plateaus into diagnostics for effective-width behavior and clarify whether data-specific structures can induce additional fixed points or invariant manifolds beyond the architecture-agnostic ones.

References

Although we have not identified any fixed points or invariant manifolds beyond Theorems 3 and 5, it remains an open question whether these are exhaustive.

— Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architectures (2512.20607 - Zhang et al., 23 Dec 2025) in Section 7 — Exhaustiveness of fixed points and invariant manifolds