Value of physical inductive biases at scale

Determine whether physical inductive biases for neural surrogates remain valuable at sufficient scale or are subsumed by model capacity and data diversity.

Background

The thesis compares neural architectures that encode different physical priors, including locality in convolutional networks, periodicity in Fourier Neural Operators, and resolution hierarchies in UNets. Vanilla transformers introduce substantially less physical structure and instead rely primarily on model capacity and data.

The unresolved issue is whether explicitly encoding physical inductive biases continues to improve neural PDE surrogates as models become larger and are trained on more diverse data. This question is relevant to the design of scientific foundation models.