Papers
Topics
Authors
Recent
Search
2000 character limit reached

Active Learning for Regression based on Wasserstein distance and GroupSort Neural Networks

Published 22 Mar 2024 in cs.LG, math.ST, stat.ML, and stat.TH | (2403.15108v1)

Abstract: This paper addresses a new active learning strategy for regression problems. The presented Wasserstein active regression model is based on the principles of distribution-matching to measure the representativeness of the labeled dataset. The Wasserstein distance is computed using GroupSort Neural Networks. The use of such networks provides theoretical foundations giving a way to quantify errors with explicit bounds for their size and depth. This solution is combined with another uncertainty-based approach that is more outlier-tolerant to complete the query strategy. Finally, this method is compared with other classical and recent solutions. The study empirically shows the pertinence of such a representativity-uncertainty approach, which provides good estimation all along the query procedure. Moreover, the Wasserstein active regression often achieves more precise estimations and tends to improve accuracy faster than other models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. (2007). Modeling slump flow of concrete using second-order regressions and artificial neural networks. Cement and Concrete Composites, 29(6):474–480.
  2. (2021). Pool-based unsupervised active learning for regression using iterative representativeness-diversity maximization (irdm). Pattern Recognition Letters, 142:11–19.
  3. (2023). Active learning for regression by inverse distance weighting. Information Sciences, 626:275–292.
  4. Sorting out lipschitz function approximation.
  5. A theory of learning from different domains. Machine Learning, 79:151–175.
  6. Curriculum learning. International Conference on Machine Learning.
  7. Some Theoretical Properties of GANs. Annals of Statistics, 48(3):1539–1566.
  8. Some theoretical insights into wasserstein gans. 22(1).
  9. An iterative algorithm for computing the best estimate of an orthogonal matrix. SIAM Journal on Numerical Analysis, 8(2):358–364.
  10. Active learning for regression based on query by committee. In Yin, H., Tino, P., Corchado, E., Byrne, W., and Yao, X., editors, Intelligent Data Engineering and Automated Learning - IDEAL 2007, pages 209–218, Berlin, Heidelberg. Springer Berlin Heidelberg.
  11. Cédric, V. (right 2009). Optimal transport : old and new / Villani Cédric. Grundlehren der mathematischen Wissenschaften. Springer, Berlin.
  12. Dudley, R. M. (1969). The Speed of Mean Glivenko-Cantelli Convergence. The Annals of Mathematical Statistics, 40(1):40 – 50.
  13. On the rate of convergence in Wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162(3-4):707.
  14. Geometry, resistance and stability of the delft systematic yacht hull series. International shipbuilding progress, 28:276–297.
  15. Generative adversarial nets. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Weinberger, K., editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc.
  16. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 110(2):393–416.
  17. Improved training of wasserstein gans. CoRR, abs/1704.00028.
  18. Hedonic housing prices and the demand for clean air. Journal of Environmental Economics and Management, 5(1):81–102.
  19. Adam: A method for stochastic optimization. International Conference on Learning Representations.
  20. A sequential algorithm for training text classifiers. In Croft, B. W. and van Rijsbergen, C. J., editors, SIGIR ’94, pages 3–12, London. Springer London.
  21. One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell., 28(4):594–611.
  22. An invitation to statistics in wasserstein space.
  23. A survey of deep active learning. arXiv preprint arXiv:2009.00236.
  24. Toward optimal active learning through sampling estimation of error reduction. In International Conference on Machine Learning.
  25. Settles, B. (2012). Active Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning Series. Morgan & Claypool.
  26. Shannon, C. E. (2001). A mathematical theory of communication. SIGMOBILE Mob. Comput. Commun. Rev., 5(1):3–55.
  27. Deep active learning: Unified and principled method for query and training. In International Conference on Artificial Intelligence and Statistics, pages 1308–1318. PMLR.
  28. Curriculum learning: A survey. Int. J. Comput. Vision, 130(6):1526–1565.
  29. Optimal 1-wasserstein distance for wgans. ArXiv, abs/2201.02824.
  30. Approximating lipschitz continuous functions with groupsort neural networks. ArXiv, abs/2006.05254.
  31. Airfoil self-noise and prediction. Technical report, NASA RP-1218.
  32. Effective data augmentation with diffusion models.
  33. Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy and Buildings, 49:560–567.
  34. Active learning for regression using greedy sampling. Information Sciences, 474:90–105.
  35. A probability analysis on the value of unlabeled data for classification problems. In Proc. 17th International Conf. on Machine Learning, pages 1191–1198. Morgan Kaufmann, San Francisco, CA.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 2 tweets with 37 likes about this paper.

Sign Up for Free