Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Markovian Formalism for Active Querying

Published 13 Jun 2023 in cs.LG and cs.AI | (2306.08001v1)

Abstract: Active learning algorithms have been an integral part of recent advances in artificial intelligence. However, the research in the field is widely varying and lacks an overall organizing leans. We outline a Markovian formalism for the field of active learning and survey the literature to demonstrate the organizing capability of our proposed formalism. Our formalism takes a partially observable Markovian system approach to the active learning process as a whole. We specifically outline how querying, dataset augmentation, reward updates, and other aspects of active learning can be viewed as a transition between meta-states in a Markovian system, and give direction into how other aspects of active learning can fit into our formalism.

Authors (1)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Algorithms for inverse reinforcement learning. In Pat Langley, editor, Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000), Stanford University, Stanford, CA, USA, June 29 - July 2, 2000, pages 663–670. Morgan Kaufmann, 2000.
  2. Aligning robot and human representations, 2023.
  3. Burr Settles. Active Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, 2012.
  4. Provably efficient reinforcement learning with linear function approximation. CoRR, abs/1907.05388, 2019.
  5. A sequential algorithm for training text classifiers, 1994. active learning roots.
  6. Query by committee. In Annual Conference Computational Learning Theory, 1992.
  7. Multiple-instance active learning. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc., 2007.
  8. Burr Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison, 2009.
  9. Jeff M. Phillips. Coresets and sketches. CoRR, abs/1601.00617, 2016.
  10. Asking easy questions: A user-friendly approach to active reward learning. CoRR, abs/1910.04365, 2019.
  11. Dana Angluin. Queries and concept learning. Machine Learning, 2:319–342, 1988.
  12. Heterogeneous uncertainty sampling for supervised learning. In International Conference on Machine Learning, 1994.
  13. Suggestive annotation: A deep active learning framework for biomedical image segmentation. CoRR, abs/1706.04737, 2017.
  14. Entity matching by pool-based active learning, 2022.
  15. Batch active learning at scale. CoRR, abs/2107.14263, 2021.
  16. Active learning for convolutional neural networks: A core-set approach, 2018.
  17. Active learning of driving scenario trajectories. Engineering Applications of Artificial Intelligence, 113:104972, 2022.
  18. Reinforcement learning with feedback from multiple humans with diverse skills. CoRR, abs/2111.08596, 2021.
  19. Designing robot learners that ask good questions. In 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 17–24, 2012.
  20. The power of comparisons for actively learning linear classifiers. CoRR, abs/1907.03816, 2019.
  21. Active classification with comparison queries. CoRR, abs/1704.03564, 2017.
  22. Principled reinforcement learning with human feedback from pairwise or k𝑘kitalic_k-wise comparisons, 2023.
  23. Active learning with label comparisons, 2022.
  24. Learning reward functions by integrating human demonstrations and preferences. CoRR, abs/1906.08928, 2019.
  25. Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences. CoRR, abs/2006.14091, 2020.
  26. Active deep q-learning with demonstration. Machine Learning, 109(9):1699–1725, 2020.
  27. Trajectories and keyframes for kinesthetic teaching: A human-robot interaction perspective. In 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 391–398, 2012.
  28. Active learning by labeling features. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1, EMNLP ’09, page 81–90, USA, 2009. Association for Computational Linguistics.
  29. Hema Raghavan. Active learning with feedback on both features and instances. 2006.
  30. Training language models to follow instructions with human feedback, 2022.
  31. Learning from human feedback: Challenges for real-world reinforcement learning in NLP. CoRR, abs/2011.02511, 2020.
  32. Learning to summarize from human feedback. CoRR, abs/2009.01325, 2020.
  33. Modeling interaction via the principle of maximum causal entropy. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, page 1255–1262, Madison, WI, USA, 2010. Omnipress.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up