Papers
Topics
Authors
Recent
Search
2000 character limit reached

Meta-Learning with Heterogeneous Tasks

Published 24 Oct 2024 in cs.LG | (2410.18894v1)

Abstract: Meta-learning is a general approach to equip machine learning models with the ability to handle few-shot scenarios when dealing with many tasks. Most existing meta-learning methods work based on the assumption that all tasks are of equal importance. However, real-world applications often present heterogeneous tasks characterized by varying difficulty levels, noise in training samples, or being distinctively different from most other tasks. In this paper, we introduce a novel meta-learning method designed to effectively manage such heterogeneous tasks by employing rank-based task-level learning objectives, Heterogeneous Tasks Robust Meta-learning (HeTRoM). HeTRoM is proficient in handling heterogeneous tasks, and it prevents easy tasks from overwhelming the meta-learner. The approach allows for an efficient iterative optimization algorithm based on bi-level optimization, which is then improved by integrating statistical guidance. Our experimental results demonstrate that our method provides flexibility, enabling users to adapt to diverse task settings and enhancing the meta-learner's overall performance.

Authors (4)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Regret bounds for lifelong learning. In Artificial Intelligence and Statistics (AISTATS), 261–269.
  2. How to train your MAML. arXiv preprint arXiv:1810.09502.
  3. Meta-learning with task-adaptive loss function for few-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 9465–9474.
  4. Meta-learning with adaptive hyperparameters. Advances in neural information processing systems, 33: 20755–20765.
  5. Provable Guarantees for Gradient-Based Meta-Learning. In International Conference on Machine Learning (ICML), 424–433.
  6. Meta-learning with differentiable closed-form solvers. In International Conference on Learning Representations (ICLR).
  7. Transductive information maximization for few-shot learning. arXiv. arXiv preprint arXiv:2008.11297.
  8. Mathematical programs with optimization problems in the constraints. Operations research, 21(1): 37–44.
  9. Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile. In International Conference on Machine Learning, 3662–3678. PMLR.
  10. Task-robust model-agnostic meta-learning. Advances in Neural Information Processing Systems, 33: 18860–18871.
  11. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, 1126–1135. PMLR.
  12. Online Meta-Learning. In International Conference on Machine Learning (ICML), 1920–1930.
  13. Recasting gradient-based meta-learning as hierarchical bayes. In International Conference on Learning Representations (ICLR).
  14. On the iteration complexity of hypergradient computation. In International Conference on Machine Learning, 3748–3758. PMLR.
  15. Holland, M. J. 2023. Flexible risk design using bi-directional dispersion. In International Conference on Artificial Intelligence and Statistics, 1586–1623. PMLR.
  16. Learning by minimizing the sum of ranked range. Advances in Neural Information Processing Systems, 33: 21013–21023.
  17. A survey of deep meta-learning. Artificial Intelligence Review, 54(6): 4483–4541.
  18. Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms. arXiv preprint arXiv:2010.07962.
  19. Bilevel optimization: Convergence analysis and enhanced design. In International conference on machine learning, 4882–4892. PMLR.
  20. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In International conference on machine learning, 2304–2313. PMLR.
  21. A Nested Bi-level Optimization Framework for Robust Few Shot Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 7176–7184.
  22. Meta-learning with differentiable convex optimization. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  23. Cleannet: Transfer learning for scalable image classifier training with label noise. In Proceedings of the IEEE conference on computer vision and pattern recognition, 5447–5456.
  24. Few-shot learning with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9089–9098.
  25. Adaptive task sampling for meta-learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16, 752–769. Springer.
  26. Not All Tasks Are Equal: A Parameter-Efficient Task Reweighting Method for Few-Shot Learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 421–437. Springer.
  27. Efficient Meta label correction based on Meta Learning and bi-level optimization. Engineering Applications of Artificial Intelligence, 117: 105517.
  28. Task Weighting in Meta-learning with Trajectory Optimisation. arXiv preprint arXiv:2301.01400.
  29. Rapid learning or feature reuse? towards understanding the effectiveness of MAML. In International Conference on Learning Representations (ICLR).
  30. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676.
  31. Learning to reweight examples for robust deep learning. In International conference on machine learning, 4334–4343. PMLR.
  32. Imagenet large scale visual recognition challenge. International journal of computer vision, 115: 211–252.
  33. Learning to rectify for robust learning with noisy labels. Pattern Recognition, 124: 108467.
  34. Matching networks for one shot learning. Advances in neural information processing systems, 29.
  35. Training noise-robust deep neural networks via meta-learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4524–4533.
  36. Meta-learning with an adaptive task scheduler. Advances in Neural Information Processing Systems, 34: 7497–7509.
  37. Deep meta-learning: Learning to learn in the concept space. arXiv preprint arXiv:1802.03596.
  38. Efficient Meta Learning via Minibatch Proximal Update. In Advances in Neural Information Processing Systems (NeurIPS), 1532–1542.
  39. Robust co-teaching learning with consistency-based noisy label correction for medical image classification. International Journal of Computer Assisted Radiology and Surgery, 18(4): 675–683.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up