Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Transfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedure (2401.12272v1)

Published 22 Jan 2024 in stat.ML and cs.LG

Abstract: Transfer learning for nonparametric regression is considered. We first study the non-asymptotic minimax risk for this problem and develop a novel estimator called the confidence thresholding estimator, which is shown to achieve the minimax optimal risk up to a logarithmic factor. Our results demonstrate two unique phenomena in transfer learning: auto-smoothing and super-acceleration, which differentiate it from nonparametric regression in a traditional setting. We then propose a data-driven algorithm that adaptively achieves the minimax risk up to a logarithmic factor across a wide range of parameter spaces. Simulation studies are conducted to evaluate the numerical performance of the adaptive transfer learning algorithm, and a real-world example is provided to demonstrate the benefits of the proposed method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks. Physical and Engineering Sciences in Medicine, 43(2):635–640.
  2. Analysis of representations for domain adaptation. Advances in Neural Information Processing Systems, 19:137.
  3. Learning bounds for domain adaptation. In Proc. Conf. Empirical Methods in Natural Language, pages 120–128.
  4. Breiman, L. (2001). Random forests. Machine learning, 45(1):5–32.
  5. Supplement to “Transfer learning for nonparametric regression: Non-asymptotic minimax rate and adaptive procedure”.
  6. Transfer learning for nonparametric classification: Minimax rate and adaptive classifier. The Annals of Statistics, 49(1):100–128.
  7. Adaptive transfer learning. In proceedings of the AAAI Conference on Artificial Intelligence, volume 24, pages 407–412.
  8. Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association, 74(368):829–836.
  9. Locally weighted regression: an approach to regression analysis by local fitting. Journal of the American Statistical Association, 83(403):596–610.
  10. Modeling wine preferences by data mining from physicochemical properties. Decision support systems, 47(4):547–553.
  11. Daumé III, H. (2009). Frustratingly easy domain adaptation. arXiv preprint arXiv:0907.1815.
  12. Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. The Annals of Statistics, pages 196–216.
  13. Variable bandwidth and local linear regression smoothers. The Annals of Statistics, 20:2008–2036.
  14. A distribution-free theory of nonparametric regression, volume 1. Springer.
  15. Correcting sample selection bias by unlabeled data. Advances in Neural Information Processing Systems, 19:601–608.
  16. Transfer learning for high-dimensional linear regression: Prediction, estimation, and minimax optimality. Journal of the Royal Statistical Society: Series B, (to appear).
  17. Transfer learning in large-scale gaussian graphical models with false discovery rate control. Journal of the American Statistical Association, (to appear).
  18. Estimation and inference for high-dimensional generalized linear models with knowledge transfer. Technical Report.
  19. Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430.
  20. Inferring air pollution by sniffing social media. In 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2014), pages 534–539. IEEE.
  21. Learning inverse dynamics: a comparison. In European Symposium on Artificial Neural Networks, number CONF.
  22. Computed torque control with nonparametric regression models. In Proc. American Control Conference, pages 212–217. IEEE.
  23. Adaptive transfer learning. The Annals of Statistics, 49(6):3618–3649.
  24. Shimodaira, H. (2000). Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 90(2):227–244.
  25. Stone, C. J. (1977). Consistent nonparametric regression. The Annals of Statistics, 5:595–620.
  26. Direct importance estimation with model selection and its application to covariate shift adaptation. In NIPS, volume 7, pages 1433–1440. Citeseer.
  27. Tsybakov, A. B. (1986). Robust reconstruction of functions by the local-approximation method. Problemy Peredachi Informatsii, 22(2):69–84.
  28. Adversarial discriminative domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7167–7176.
  29. Statistical learning for humanoid robots. Autonomous Robots, 12(1):55–69.
  30. Nonparametric risk and stability analysis for multi-task learning problems. In IJCAI, pages 2146–2152.
  31. A survey of transfer learning. Journal of Big data, 3(1):1–40.
  32. Robust learning under uncertain test distributions: Relating covariate shift to model misspecification. In International Conference on Machine Learning, pages 631–639. PMLR.
  33. More efficient local polynomial estimation in nonparametric regression with autocorrelated errors. Journal of the American Statistical Association, 98(464):980–992.
  34. Learning inverse dynamics by gaussian process begrression under the multi-task learning framework. In The Path to Autonomous Robots, pages 1–12. Springer.
  35. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1):43–76.
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets