2000 character limit reached
M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling (2403.13728v3)
Published 20 Mar 2024 in cs.LG and cs.AI
Abstract: A probabilistic graphical model is proposed, modeling the joint model parameter and multiplier evolution, with a hypervolume based likelihood, promoting multi-objective descent in structural risk minimization. We address multi-objective model parameter optimization via a surrogate single objective penalty loss with time-varying multipliers, equivalent to online scheduling of loss landscape. The multi-objective descent goal is dispatched hierarchically into a series of constraint optimization sub-problems with shrinking bounds according to Pareto dominance. The bound serves as setpoint for the low-level multiplier controller to schedule loss landscapes via output feedback of each loss term. Our method forms closed loop of model parameter dynamic, circumvents excessive memory requirements and extra computational burden of existing multi-objective deep learning methods, and is robust against controller hyperparameter variation, demonstrated on domain generalization tasks with multi-dimensional regularization losses.
- Nonlinear programming: theory and algorithms. John wiley & sons, 2013.
- Dimitri P Bertsekas. Constrained optimization and Lagrange multiplier methods. Academic press, 2014.
- Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2229–2238, 2019.
- Local distance preserving auto-encoders using continuous knn graphs. In Topological, Algebraic and Geometric Learning Workshops 2022, pages 55–66, 2022.
- Thomas G Dietterich et al. The maxq method for hierarchical reinforcement learning. In ICML, volume 98, pages 118–126, 1998.
- Feedback control theory. Courier Corporation, 2013.
- Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096–2030, 2016.
- Roman Garnett. Bayesian optimization. Cambridge University Press, 2023.
- The hypervolume indicator: Computational problems and algorithms. ACM Computing Surveys (CSUR), 54(6):1–42, 2021.
- In search of lost domain generalization. arXiv preprint arXiv:2007.01434, 2020.
- Model-free nonlinear feedback optimization. IEEE Transactions on Automatic Control, 2023.
- Automatic algorithm configuration based on local search. In Aaai, volume 7, pages 1152–1157, 2007.
- Aclib: A benchmark library for algorithm configuration. In Learning and Intelligent Optimization: 8th International Conference, Lion 8, Gainesville, FL, USA, February 16-21, 2014. Revised Selected Papers 8, pages 36–40. Springer, 2014.
- Diva: Domain invariant variational autoencoders. In Medical Imaging with Deep Learning, pages 322–348. PMLR, 2020.
- PID control. Springer, 2005.
- Adam: A method for stochastic optimization. International Conference for Learning Representations, 2015.
- Learning hierarchical priors in VAEs. Advances in Neural Information processing Systems, 32, 2019.
- Probabilistic graphical models: principles and techniques. MIT press, 2009.
- Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. Advances in neural information processing systems, 29, 2016.
- Domain invariant adversarial learning. arXiv preprint arXiv:2104.00322, 2021.
- Sergey Levine. Reinforcement learning and control as probabilistic inference: Tutorial and review. arXiv preprint arXiv:1805.00909, 2018.
- Optimal control. John Wiley & Sons, 2012.
- Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, pages 5542–5550, 2017.
- Learning to generalize: Meta-learning for domain generalization. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
- The irace package: Iterated racing for automatic algorithm configuration. Operations Research Perspectives, 3:43–58, 2016.
- Tristan Milne. Piecewise strong convexity of neural networks. Advances in Neural Information Processing Systems, 32, 2019.
- Closing the loop: Dynamic state estimation and feedback optimization of power grids. Electric Power Systems Research, 189:106753, 2020.
- Tunability: Importance of hyperparameters of machine learning algorithms. The Journal of Machine Learning Research, 20(1):1934–1965, 2019.
- Fishr: Invariant gradient variances for out-of-distribution generalization. In International Conference on Machine Learning, pages 18347–18377. PMLR, 2022.
- On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237, 2019.
- D. J. Rezende and F. Viola. Taming VAEs. CoRR, 2018.
- Controlvae: Controllable variational autoencoder. In International Conference on Machine Learning, pages 8655–8664. PMLR, 2020.
- Domain adversarial neural networks for domain generalization: When it works and how to improve. Machine Learning, pages 1–37, 2023.
- Ladder variational autoencoders. Advances in neural information processing systems, 29, 2016.
- Tutorial and survey on probabilistic graphical model and variational inference in deep reinforcement learning. In 2019 IEEE Symposium Series on Computational Intelligence (SSCI), number 1908.09381, 2019.
- Hierarchical variational auto-encoding for unsupervised domain generalization. ICLR 2021 RobustML, arXiv preprint arXiv:2101.09436, 2021.
- High dimensional restrictive federated model selection with multi-objective bayesian optimization over shifted distributions. arXiv preprint arXiv:1902.08999, 2019a.
- Variational resampling based assessment of deep neural networks under distribution shift. In 2019 IEEE Symposium Series on Computational Intelligence (SSCI), number 1906.02972, 2019b.
- Reinbo: Machine learning pipeline conditional hierarchy search and configuration with bayesian optimization embedded reinforcement learning. In Machine Learning and Knowledge Discovery in Databases: International Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I, pages 68–84. Springer International Publishing, 2020.
- Indicator-based selection in multiobjective search. In International conference on parallel problem solving from nature, pages 832–842. Springer, 2004.
- The hypervolume indicator revisited: On the design of pareto-compliant indicators via weighted integration. In Evolutionary Multi-Criterion Optimization: 4th International Conference, EMO 2007, Matsushima, Japan, March 5-8, 2007. Proceedings 4, pages 862–876. Springer, 2007.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.