CPU–GPU Conjecture

Prove or refute the CPU–GPU conjecture that for any deep learning task outcome (e.g., model performance on classification or regression), a model trained on a central processing unit (CPU) can achieve the same result in comparable training time as a model trained on a graphical processing unit (GPU), provided that the CPU-trained model is designed with sufficiently strong mathematical insight.

Background

The paper motivates this conjecture by emphasizing that deep learning training can be computationally expensive and that better mathematical understanding of decision boundaries—such as modeling them by low-degree algebraic hypersurfaces—may lead to more efficient models. The authors suggest that principled, insight-driven model design could offset the raw computational advantages typically attributed to GPUs.

They argue that widespread reliance on trial-and-error hyperparameter searches and brute-force compute may be reducible through mathematically informed architectures and training strategies, raising the question of whether GPU performance advantages are fundamentally necessary for achieving state-of-the-art results.