2000 character limit reached
On the inductive bias of infinite-depth ResNets and the bottleneck rank (2501.19149v1)
Published 31 Jan 2025 in cs.LG, cs.AI, and stat.ML
Abstract: We compute the minimum-norm weights of a deep linear ResNet, and find that the inductive bias of this architecture lies between minimizing nuclear norm and rank. This implies that, with appropriate hyperparameters, deep nonlinear ResNets have an inductive bias towards minimizing bottleneck rank.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.