Specifying and Solving Robust Empirical Risk Minimization Problems Using CVXPY (2306.05649v3)
Abstract: We consider robust empirical risk minimization (ERM), where model parameters are chosen to minimize the worst-case empirical loss when each data point varies over a given convex uncertainty set. In some simple cases, such problems can be expressed in an analytical form. In general the problem can be made tractable via dualization, which turns a min-max problem into a min-min problem. Dualization requires expertise and is tedious and error-prone. We demonstrate how CVXPY can be used to automate this dualization procedure in a user-friendly manner. Our framework allows practitioners to specify and solve robust ERM problems with a general class of convex losses, capturing many standard regression and classification problems. Users can easily specify any complex uncertainty set that is representable via disciplined convex programming (DCP) constraints.
- Laurent El Ghaoui and Hervé Lebret “Robust solutions to least-squares problems with uncertain data” In SIAM Journal on matrix analysis and applications 18.4 SIAM, 1997, pp. 1035–1064
- “Convex Optimization” Cambridge University Press, 2004
- Pannagadatta K. Shivaswamy, Chiranjib Bhattacharyya and Alexander J. Smola “Second Order Cone Programming Approaches for Handling Missing and Uncertain Data” In Journal of Machine Learning Research 7.47, 2006, pp. 1283–1314
- Huan Xu, Constantine Caramanis and Shie Mannor “Robust Regression and Lasso” In Advances in Neural Information Processing Systems 21 Curran Associates, Inc., 2008
- Aharon Ben-Tal, Laurent El Ghaoui and Arkadi Nemirovski “Robust optimization” Princeton university press, 2009
- Huan Xu, Constantine Caramanis and Shie Mannor “Robustness and Regularization of Support Vector Machines” In Journal of Machine Learning Research 10.51, 2009, pp. 1485–1510
- Dimitris Bertsimas, David Brown and Constantine Caramanis “Theory and applications of robust optimization” In SIAM review 53.3 SIAM, 2011, pp. 464–501
- “CVXPY: A Python-embedded modeling language for convex optimization” In Journal of Machine Learning Research 17.83, 2016, pp. 1–5
- “Determinants of Airbnb prices in European cities: A spatial econometrics approach” In Tourism Management 86, 2021, pp. 104319
- Alireza Aghasi, MohammadJavad Feizollahi and Saeed Ghadimi “RIGID: Robust Linear Regression with Missing Data” In arXiv preprint arXiv:2205.13635, 2022
- Philipp Schiele, Eric Luxenberg and Stephen Boyd “Disciplined Saddle Programming” In arXiv preprint arXiv:2301.13427, 2023