Handling heterogeneous noise variances across tasks

Develop and analyze methods for the multivariate random-effects linear meta-learning model when noise variances differ across tasks, i.e., when each task ℓ has its own variance σℓ^2 rather than a common σ^2. Specifically, extend the proposed hyper-covariance estimation and generalized ridge regression framework to the heteroscedastic setting by constructing consistent estimators for {σℓ^2} and the hyper-covariance Ω, and characterize the resulting predictive risk on new tasks.

Background

Throughout the paper, the noise variance is assumed to be common across tasks (σ^2), which enables use of method-of-moments or Dicker-type estimators for σ^2 and simplifies hyper-covariance estimation and predictive risk analysis.

The authors note that allowing different noise variances across tasks (heteroscedasticity) is more challenging. Addressing this would require both methodological developments (e.g., simultaneous estimation of task-specific noise variances and the hyper-covariance) and theoretical analysis (e.g., high-dimensional consistency and predictive risk characterization under heteroscedastic noise).