Determine Whether the Group-Proportion Factor Is a Proof Artifact

Determine whether the factor \(\underline{\pi}_{\ell}\) appearing in the convergence bound of Theorem 3 for the non-sample-split deep ReLU pretraining estimator is merely an artifact of the proof rather than an intrinsic feature of the estimator's convergence rate.

Background

The paper compares two-stage transfer learning with and without sample splitting. Corollary 1, which assumes that the first-stage estimator is computed from data independent of the second-stage data, removes a factor of $\underline{\pi}_{\ell}$ from the denominator of the first-stage contribution to the rate. By contrast, Theorem 3 for the usual shared-data setting retains this factor.

The authors conjecture that this discrepancy is caused by the proof technique because empirical practice shows better performance without sample splitting. Establishing whether the factor can be removed theoretically would clarify the sharpness of the non-independent two-stage bound, especially when the target-group proportion tends to zero.