2000 character limit reached
Outlier-resilient model fitting via percentile losses: Methods for general and convex residuals (2405.09436v1)
Published 15 May 2024 in eess.SP
Abstract: We consider the problem of robustly fitting a model to data that includes outliers by formulating a percentile optimization problem. This problem is non-smooth and non-convex, hence hard to solve. We derive properties that the minimizers of such problems must satisfy. These properties lead to methods that solve the percentile formulation both for general residuals and for convex residuals. The methods fit the model to subsets of the data, and then extract the solution of the percentile formulation from these partial fits. As illustrative simulations show, such methods endure higher outlier percentages, when compared with standard robust estimates. Additionally, the derived properties provide a broader and alternative theoretical validation for existing robust methods, whose validity was previously limited to specific forms of the residuals.
- G. S. Watson, “Linear least squares regression,” The Annals of Mathematical Statistics, pp. 1679–1699, 1967.
- S. Van Huffel and H. Zha, “10 the total least squares problem,” in Computational Statistics, ser. Handbook of Statistics. Elsevier, 1993, vol. 9, pp. 377–408. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0169716105801348
- R. L. Dykstra, “An algorithm for restricted least squares regression,” Journal of the American Statistical Association, vol. 78, no. 384, pp. 837–842, 1983.
- M. L. Johnson and L. M. Faunt, “[1] parameter estimation by least-squares methods,” in Numerical Computer Methods, ser. Methods in Enzymology. Academic Press, 1992, vol. 210, pp. 1–37. [Online]. Available: https://www.sciencedirect.com/science/article/pii/007668799210003V
- H. W. Sorenson, “Least-squares estimation: from gauss to kalman,” IEEE Spectrum, vol. 7, no. 7, pp. 63–68, 1970.
- P. J. Rousseeuw, “Least median of squares regression,” Journal of the American statistical association, vol. 79, no. 388, pp. 871–880, 1984.
- P. J. Rousseeuw and M. Hubert, “Recent developments in progress,” Lecture Notes-Monograph Series, pp. 201–214, 1997.
- T. Bernholt, “Robust estimators are hard to compute,” Technical reports, 2006. [Online]. Available: https://api.semanticscholar.org/CorpusID:14288274
- D. L. Souvaine and J. M. Steele, “Time-and space-efficient algorithms for least median of squares regression,” Journal of the American Statistical Association, vol. 82, no. 399, pp. 794–801, 1987.
- G. Shapira and T. Hassner, “Gpu-based computation of 2d least median of squares with applications to fast and robust line detection,” arXiv preprint arXiv:1510.01041, 2015.
- D. M. Mount, N. S. Netanyahu, K. Romanik, R. Silverman, and A. Y. Wu, “A practical approximation algorithm for the lms line estimator,” Computational statistics & data analysis, vol. 51, no. 5, pp. 2461–2486, 2007.
- H. Edelsbrunner and D. L. Souvaine, “Computing least median of squares regression lines and guided topological sweep,” Journal of the American Statistical Association, vol. 85, no. 409, pp. 115–119, 1990.
- A. J. Stromberg, “Computing the exact value of the least median of squares estimate in multiple linear regression,” University of Minnesota, Tech. Rep., 1991.
- J. Agulló, “Exact algorithms for computing the least median of squares estimate in multiple linear regression,” Lecture Notes-Monograph Series, pp. 133–146, 1997.
- A. Giloni and M. Padberg, “Least trimmed squares regression, least median squares regression, and mathematical programming,” Mathematical and Computer Modelling, vol. 35, no. 9-10, pp. 1043–1060, 2002.
- D. Bertsimas and R. Mazumder, “Least quantile regression via modern optimization,” The Annals of Statistics, vol. 42, no. 6, pp. 2494 – 2525, 2014. [Online]. Available: https://doi.org/10.1214/14-AOS1223
- R. Koenker and K. F. Hallock, “Quantile regression,” Journal of economic perspectives, vol. 15, no. 4, pp. 143–156, 2001.
- R. Koenker and G. Bassett Jr, “Regression quantiles,” Econometrica: journal of the Econometric Society, pp. 33–50, 1978.
- R. T. Rockafellar, S. Uryasev et al., “Optimization of conditional value-at-risk,” Journal of risk, vol. 2, pp. 21–42, 2000.
- J. Domingos and J. Xavier, “Robust target localization in 2d: A value-at-risk approach,” Under review but currently on arXiv:2307.00548, 2023.
- B. Chakraborty and P. Chaudhuri, “On an optimization problem in robust statistics,” Journal of computational and Graphical Statistics, vol. 17, no. 3, pp. 683–702, 2008.
- J. Joss and A. Marazzi, “Probabilistic algorithms for least median of squares regression,” Computational Statistics & Data Analysis, vol. 9, no. 1, pp. 123–133, 1990.
- R. Cook, D. Hawkins, and S. Weisberg, “Exact iterative computation of the robust multivariate minimum volume ellipsoid estimator,” Statistics & probability letters, vol. 16, no. 3, pp. 213–218, 1993.
- S. Van Aelst and P. Rousseeuw, “Minimum volume ellipsoid,” Wiley Interdisciplinary Reviews: Computational Statistics, vol. 1, no. 1, pp. 71–82, 2009.
- A. J. Stromberg, “Computing the exact least median of squares estimate and stability diagnostics in multiple linear regression,” SIAM Journal on Scientific Computing, vol. 14, no. 6, pp. 1289–1299, 1993.
- D. M. Hawkins, “The feasible set algorithm for least median of squares regression,” Computational Statistics & Data Analysis, vol. 16, no. 1, pp. 81–101, 1993.
- G. A. Watson, “On computing the least quantile of squares estimate,” SIAM Journal on Scientific Computing, vol. 19, no. 4, pp. 1125–1138, 1998.
- T. Rivlin, “Overdetermined systems of linear equations,” SIAM Review, vol. 5, no. 1, pp. 52–66, 1963.
- L. Källberg, “Minimum enclosing balls and ellipsoids in general dimensions,” Ph.D. dissertation, Mälardalen University, 2019.
- E. J. Candès and Y. Plan, “Near-ideal model selection by l1 minimization,” The Annals of Statistics, vol. 37, no. 5A, pp. 2145 – 2177, 2009. [Online]. Available: https://doi.org/10.1214/08-AOS653
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.