Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Active Learning Guided by Efficient Surrogate Learners (2301.02761v2)

Published 7 Jan 2023 in cs.LG and cs.CV

Abstract: Re-training a deep learning model each time a single data point receives a new label is impractical due to the inherent complexity of the training process. Consequently, existing active learning (AL) algorithms tend to adopt a batch-based approach where, during each AL iteration, a set of data points is collectively chosen for annotation. However, this strategy frequently leads to redundant sampling, ultimately eroding the efficacy of the labeling procedure. In this paper, we introduce a new AL algorithm that harnesses the power of a Gaussian process surrogate in conjunction with the neural network principal learner. Our proposed model adeptly updates the surrogate learner for every new data instance, enabling it to emulate and capitalize on the continuous learning dynamics of the neural network without necessitating a complete retraining of the principal model for each individual label. Experiments on four benchmark datasets demonstrate that this approach yields significant enhancements, either rivaling or aligning with the performance of state-of-the-art techniques.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Active learning: a survey. In Aggarwal, C. C., ed., Data Classification Algorithms and Applications, 571–605. CRC Press.
  2. Hierarchical subquery evaluation for active learning on a graph. In CVPR, 564–571.
  3. Gone fishing: neural active Learning with Fisher embeddings. In NeurIPS.
  4. Deep batch active learning by diverse, uncertain gradient lower bounds. In ICLR.
  5. Sequential graph convolutional network for active learning. In CVPR, 9583–9592.
  6. Selection via proxy: efficient data selection for deep learning. In ICLR.
  7. A convex optimization framework for active learning. In ICCV, 4321–4328.
  8. Selecting influential examples: active learning with expected model output changes. In ECCV, 562–577.
  9. Deep active learning over the long tail. In arXiv:1711.00941.
  10. Discriminative active learning. In arXiv:1907.06347.
  11. Caltech-256 object category dataset. Technical report, California Institute of Technology.
  12. Guo, Y. 2010. Active instance sampling via matrix partition. In NIPS, 802–810.
  13. Deep residual learning for image recognition. In CVPR, 770–778.
  14. Bayesian active learning for classification and preference learning. In arXiv:1112.5745.
  15. Task-aware variational adversarial active learning. In CVPR, 8166–8175.
  16. BatchBALD: efficient and diverse batch acquisition for deep Bayesian active learning. In NeurIPS.
  17. Krizhevsky, A. 2009. Learning multiple layers of features from tiny images. Technical report, University of Toronto.
  18. Tiny ImageNet visual recognition challenge. Technical Report CS231N Course, Stanford University.
  19. Active learning using pre-clustering. In ICML, 623–630.
  20. Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press.
  21. A survey of deep active learning. In arXiv:2009.00236.
  22. Schott, J. R. 2016. Matrix Analysis for Statistics. New Jersey: Wiley, 3rd edition.
  23. Active learning for convolutional neural networks: a core-set approach. In ICLR.
  24. Settles, B. 2009. Active learning literature survey. Technical Report Computer Sciences Technical Report 1648, University of Wisconsin-Madison.
  25. Very deep convolutional networks for large-scale image recognition. In ICLR, arXiv:1409.1556.
  26. Variational adversarial active learning. In ICCV, 5972–5981.
  27. Sparse Gaussian processes using pseudo-inputs. In NIPS.
  28. Using the equivalent kernel to understand Gaussian process regression. In NIPS, 1313–1320.
  29. Support vector machine active learning with applications to text classification. JMLR, 2: 45–66.
  30. Bayesian Generative Active Deep Learning. In ICML, 6295–6304.
  31. Visual recognition and detection under bounded computational resources. In CVPR, 562–577.
  32. Boosting active learning via improving test performance. In AAAI, 8566–8574.
  33. FashionMNIST: a novel image dataset for benchmarking machine learning algorithms. In arXiv:1708.07747.
  34. Learning loss for active Learning. In CVPR, 93–102.
  35. Weight decay scheduling and knowledge distillation for active learning. In ECCV, 431–447.
  36. State-relabeling adversarial active learning. In CVPR, 8756–8765.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now