Theory for Last-Layer Fine-Tuning

Develop theoretical guarantees for last-layer fine-tuning strategies that are analogous to the guarantees established for the two-stage offset transfer learning framework.

Background

The two-stage offset framework is conceptually related to the practically common strategy of pretraining a neural network and then fine-tuning only its final layer for each target group. The paper includes Top-FT as an empirical competitor but does not provide a corresponding theoretical analysis.

A theory for last-layer fine-tuning would connect the paper's additive offset analysis to a widely used neural-network transfer-learning procedure and clarify when fine-tuning achieves rates comparable to or better than the proposed two-stage estimator.