How Inductive Bias in Machine Learning Aligns with Optimality in Economic Dynamics

Abstract: This paper examines the alignment of inductive biases in ML with structural models of economic dynamics. Unlike dynamical systems found in physical and life sciences, economics models are often specified by differential equations with a mixture of easy-to-enforce initial conditions and hard-to-enforce infinite horizon boundary conditions (e.g. transversality and no-ponzi-scheme conditions). Traditional methods for enforcing these constraints are computationally expensive and unstable. We investigate algorithms where those infinite horizon constraints are ignored, simply training unregularized kernel machines and neural networks to obey the differential equations. Despite the inherent underspecification of this approach, our findings reveal that the inductive biases of these ML models innately enforce the infinite-horizon conditions necessary for the well-posedness. We theoretically demonstrate that (approximate or exact) min-norm ML solutions to interpolation problems are sufficient conditions for these infinite-horizon boundary conditions in a wide class of problems. We then provide empirical evidence that deep learning and ridgeless kernel methods are not only theoretically sound with respect to economic assumptions, but may even dominate classic algorithms in low to medium dimensions. More importantly, these results give confidence that, despite solving seemingly ill-posed problems, there are reasons to trust the plethora of black-box ML algorithms used by economists to solve previously intractable, high-dimensional dynamical systems -- paving the way for future work on estimation of inverse problems with embedded optimal control problems.