Prove the tight $(d,d+1,d+1)$ architecture bound for spheres

Prove that the explicitly defined ReLU network with architecture $(d,d+1,d+1)$ realizes a space homeomorphic to the sphere $S^d$ for arbitrary dimension $d$, and prove that no architecture of shape $(d,d,d+1)$ can express such a sphere.

Background

The paper constructs ReLU networks whose images are homotopy equivalent to spheres, wedge sums of spheres, and products of spheres. Its general constructive sphere bound uses architecture (d,Wd,d+1)(d,W_d,d+1), where Wd=min⁡(2d(d+1),3d+2)W_d=\min(2d(d+1),3d+2). The authors report an alternative architecture with only d+1d+1 hidden neurons, yielding (d,d+1,d+1)(d,d+1,d+1), and formulate this as a conjectured tight upper bound; they also conjecture that reducing the hidden width to dd cannot express SdS^d.

The proposed construction is supported empirically for dimensions d=2d=2 and d=3d=3 using sampled meshes and persistent homology, but the authors do not establish it mathematically for arbitrary dimensions. Proving the conjecture would substantially improve the paper’s wedge-sum and product architectural bounds.

References

While we are unable to prove this bound for spheres of arbitrary dimension in this work, we provide some empirical evidence to support its validity. For $d=2$ we applied the network to $1512$ points from a regular mesh on $[-1,2]2$ and for $d=3$ we used $813$ points on $[-1,2]3$. We then applied the construction on the mesh and computed the persistent homology of the output using Ripser with 800 landmarks. As one can see, the birth-death diagrams in panel B of Figure \ref{fig:tight construction} clearly reflect the structure of the sphere with $d=2,3$. While this is not a proof of the conjecture, it demonstrates that this construction likely works for $d<4$. We believe that the conjecture can be rigorously proven in future work.

— Constructive Bounds for the Topological Expressivity of Neural Networks  (2610.13931 - Beshkov, 9 Oct 2026) in Section 3, subsection “Conjectured tight and lower bounds” (following Conjecture; discussion immediately thereafter)