Generalization of P=2, D=2 MMCR intuition to high-dimensional regimes
Determine the extent to which the closed-form intuition for Maximum Manifold Capacity Representations (MMCR) in the P=2, D=2 case—namely, that maximizing the norm of each mean vector and orthogonalizing the centers maximizes the nuclear norm—extends to general settings with arbitrary numbers of manifolds P and embedding dimensions D in the large-data, high-dimensional regime.
References
Yerxa et. al (2023) \citep{yerxa2023learning} note that no closed form solution exists for singular values of an arbitrary matrix, but when $P=2, D=2$, a closed form solution exists that offers intuition: $|C|_*$ will be maximized when (i) the norm of each mean is maximized i.e., $|_p|_2 = 1$ (recalling that $0 \leq |_p| < 1$ since the embeddings live on the hypersphere), and (ii) the means $_1, _2$ are orthogonal to one another. While we commend the authors for working to offer intuition, it is unclear to what extent the $P=2, D=2$ setting sheds light on MMCR in general, as MMCR was theoretically derived and numerically implemented in the large data and high dimension regime.