2000 character limit reached
Adjusted Wasserstein Distributionally Robust Estimator in Statistical Learning (2303.15579v3)
Published 27 Mar 2023 in stat.ML and cs.LG
Abstract: We propose an adjusted Wasserstein distributionally robust estimator -- based on a nonlinear transformation of the Wasserstein distributionally robust (WDRO) estimator in statistical learning. The classic WDRO estimator is asymptotically biased, while our adjusted WDRO estimator is asymptotically unbiased, resulting in a smaller asymptotic mean squared error. Further, under certain conditions, our proposed adjustment technique provides a general principle to de-bias asymptotically biased estimators. Specifically, we will investigate how the adjusted WDRO estimator is developed in the generalized linear model, including logistic regression, linear regression, and Poisson regression. Numerical experiments demonstrate the favorable practical performance of the adjusted estimator over the classic one.
- The performance of Wasserstein distributionally robust M-estimators in high dimensions. arXiv preprint arXiv:2206.13269, 2022.
- Quantifying distributional model risk via optimal transport. Mathematics of Operations Research, 44(2):565–600, 2019.
- Statistical analysis of Wasserstein distributionally robust estimators. In Tutorials in Operations Research: Emerging Optimization Methods and Modeling Techniques with Applications, pages 227–254. INFORMS, 2021.
- Distributionally robust mean-variance portfolio selection with Wasserstein distances. Management Science, 68(9):6382–6410, 2022a.
- Confidence regions in Wasserstein distributionally robust estimation. Biometrika, 109(2):295–315, 2022b.
- Optimal transport-based distributionally robust optimization: Structural properties and iterative schemes. Mathematics of Operations Research, 47(2):1500–1529, 2022c.
- Wasserstein distance and the distributionally robust TSP. Operations Research, 66(6):1603–1624, 2018.
- A robust learning approach for regression models based on distributionally robust optimization. Journal of Machine Learning Research, 19(13), 2018.
- Rui Gao. Finite-sample guarantees for Wasserstein distributionally robust optimization: Breaking the curse of dimensionality. Operations Research, 2022.
- Distributionally robust stochastic optimization with Wasserstein distance. Mathematics of Operations Research, 2022.
- Distributional robustness and regularization in statistical learning. arXiv preprint arXiv:1712.06050, 2017.
- Wasserstein distributionally robust optimization: Theory and applications in machine learning. In Operations research & management science in the age of analytics, pages 130–166. Informs, 2019.
- A first-order algorithmic framework for distributionally robust logistic regression. Advances in Neural Information Processing Systems, 32, 2019.
- Decomposition algorithm for distributionally robust optimization using Wasserstein metric with an application to a class of regression models. European Journal of Operational Research, 278(1):20–35, 2019.
- Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. Mathematical Programming, 171(1):115–166, 2018.
- Distributionally robust inverse covariance estimation: The Wasserstein shrinkage estimator. Operations Research, 70(1):490–515, 2022.
- Distributionally robust logistic regression. Advances in Neural Information Processing Systems, 28, 2015.
- Regularization via mass transportation. Journal of Machine Learning Research, 20(103):1–68, 2019.
- Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.
- Certifying some distributional robustness with principled adversarial training. In International Conference on Learning Representations, 2018.
- The optimizer’s curse: Skepticism and postdecision surprise in decision analysis. Management Science, 52(3):311–322, 2006.
- Weak Convergence and Empirical Processes: With Applications to Statistics. Springer Series in Statistics. Springer, 1996. ISBN 9780387946405. URL https://books.google.com/books?id=OCenCW9qmp4C.
- Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
- Risk-based distributionally robust optimal gas-power flow with Wasserstein distance. IEEE Transactions on Power Systems, 34(3):2190–2204, 2018.
- On generalization and regularization via Wasserstein distributionally robust optimization. arXiv preprint arXiv:2212.05716, 2022.
- Insoon Yang. Wasserstein distributionally robust stochastic control: A data-driven approach. IEEE Transactions on Automatic Control, 66(8):3863–3870, 2020.