Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Active Learning for Regression based on Wasserstein distance and GroupSort Neural Networks (2403.15108v1)

Published 22 Mar 2024 in cs.LG, math.ST, stat.ML, and stat.TH

Abstract: This paper addresses a new active learning strategy for regression problems. The presented Wasserstein active regression model is based on the principles of distribution-matching to measure the representativeness of the labeled dataset. The Wasserstein distance is computed using GroupSort Neural Networks. The use of such networks provides theoretical foundations giving a way to quantify errors with explicit bounds for their size and depth. This solution is combined with another uncertainty-based approach that is more outlier-tolerant to complete the query strategy. Finally, this method is compared with other classical and recent solutions. The study empirically shows the pertinence of such a representativity-uncertainty approach, which provides good estimation all along the query procedure. Moreover, the Wasserstein active regression often achieves more precise estimations and tends to improve accuracy faster than other models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. (2007). Modeling slump flow of concrete using second-order regressions and artificial neural networks. Cement and Concrete Composites, 29(6):474–480.
  2. (2021). Pool-based unsupervised active learning for regression using iterative representativeness-diversity maximization (irdm). Pattern Recognition Letters, 142:11–19.
  3. (2023). Active learning for regression by inverse distance weighting. Information Sciences, 626:275–292.
  4. Sorting out lipschitz function approximation.
  5. A theory of learning from different domains. Machine Learning, 79:151–175.
  6. Curriculum learning. International Conference on Machine Learning.
  7. Some Theoretical Properties of GANs. Annals of Statistics, 48(3):1539–1566.
  8. Some theoretical insights into wasserstein gans. 22(1).
  9. An iterative algorithm for computing the best estimate of an orthogonal matrix. SIAM Journal on Numerical Analysis, 8(2):358–364.
  10. Active learning for regression based on query by committee. In Yin, H., Tino, P., Corchado, E., Byrne, W., and Yao, X., editors, Intelligent Data Engineering and Automated Learning - IDEAL 2007, pages 209–218, Berlin, Heidelberg. Springer Berlin Heidelberg.
  11. Cédric, V. (right 2009). Optimal transport : old and new / Villani Cédric. Grundlehren der mathematischen Wissenschaften. Springer, Berlin.
  12. Dudley, R. M. (1969). The Speed of Mean Glivenko-Cantelli Convergence. The Annals of Mathematical Statistics, 40(1):40 – 50.
  13. On the rate of convergence in Wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162(3-4):707.
  14. Geometry, resistance and stability of the delft systematic yacht hull series. International shipbuilding progress, 28:276–297.
  15. Generative adversarial nets. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Weinberger, K., editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc.
  16. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 110(2):393–416.
  17. Improved training of wasserstein gans. CoRR, abs/1704.00028.
  18. Hedonic housing prices and the demand for clean air. Journal of Environmental Economics and Management, 5(1):81–102.
  19. Adam: A method for stochastic optimization. International Conference on Learning Representations.
  20. A sequential algorithm for training text classifiers. In Croft, B. W. and van Rijsbergen, C. J., editors, SIGIR ’94, pages 3–12, London. Springer London.
  21. One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell., 28(4):594–611.
  22. An invitation to statistics in wasserstein space.
  23. A survey of deep active learning. arXiv preprint arXiv:2009.00236.
  24. Toward optimal active learning through sampling estimation of error reduction. In International Conference on Machine Learning.
  25. Settles, B. (2012). Active Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning Series. Morgan & Claypool.
  26. Shannon, C. E. (2001). A mathematical theory of communication. SIGMOBILE Mob. Comput. Commun. Rev., 5(1):3–55.
  27. Deep active learning: Unified and principled method for query and training. In International Conference on Artificial Intelligence and Statistics, pages 1308–1318. PMLR.
  28. Curriculum learning: A survey. Int. J. Comput. Vision, 130(6):1526–1565.
  29. Optimal 1-wasserstein distance for wgans. ArXiv, abs/2201.02824.
  30. Approximating lipschitz continuous functions with groupsort neural networks. ArXiv, abs/2006.05254.
  31. Airfoil self-noise and prediction. Technical report, NASA RP-1218.
  32. Effective data augmentation with diffusion models.
  33. Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy and Buildings, 49:560–567.
  34. Active learning for regression using greedy sampling. Information Sciences, 474:90–105.
  35. A probability analysis on the value of unlabeled data for classification problems. In Proc. 17th International Conf. on Machine Learning, pages 1191–1198. Morgan Kaufmann, San Francisco, CA.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets