Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 162 tok/s
Gemini 2.5 Pro 51 tok/s Pro
GPT-5 Medium 31 tok/s Pro
GPT-5 High 28 tok/s Pro
GPT-4o 95 tok/s Pro
Kimi K2 202 tok/s Pro
GPT OSS 120B 425 tok/s Pro
Claude Sonnet 4.5 36 tok/s Pro
2000 character limit reached

Minimax Analysis for Inverse Risk in Nonparametric Planer Invertible Regression (2112.00213v3)

Published 1 Dec 2021 in math.ST, stat.ML, and stat.TH

Abstract: We study a minimax risk of estimating inverse functions on a plane, while keeping an estimator is also invertible. Learning invertibility from data and exploiting an invertible estimator are used in many domains, such as statistics, econometrics, and machine learning. Although the consistency and universality of invertible estimators have been well investigated, analysis of the efficiency of these methods is still under development. In this study, we study a minimax risk for estimating invertible bi-Lipschitz functions on a square in a $2$-dimensional plane. We first introduce two types of $L2$-risks to evaluate an estimator which preserves invertibility. Then, we derive lower and upper rates for minimax values for the risks associated with inverse functions. For the derivation, we exploit a representation of invertible functions using level-sets. Specifically, to obtain the upper rate, we develop an estimator asymptotically almost everywhere invertible, whose risk attains the derived minimax lower rate up to logarithmic factors. The derived minimax rate corresponds to that of the non-invertible bi-Lipschitz function, which shows that the invertibility does not reduce the complexity of the estimation problem in terms of the rate. % the minimax rate, similar to other shape constraints.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Bellec, P. C. (2018). Sharp oracle inequalities for least squares estimators in shape restricted regression. The Annals of Statistics, 46(2):745–780.
  2. Bias robust estimation in finite populations using nonparametric calibration. Journal of the American Statistical Association, 88(421):268–277.
  3. Smooth approximation of bi-lipschitz orientation-preserving homeomorphisms. Annales de l’IHP Analyse non linéaire, 31(3):567–589.
  4. Block Neural Autoregressive Flow. In Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, volume 115 of Proceedings of Machine Learning Research, pages 1263–1273. PMLR.
  5. Devroye, L. (1978). The uniform convergence of nearest neighbor regression function estimators and their application in optimization. IEEE Transactions on Information Theory, 24(2):142–151.
  6. On the strong universal consistency of nearest neighbor regression function estimates. The Annals of Statistics, pages 1371–1385.
  7. NICE: Non-Linear Independent Components Estimation. arXiv preprint arXiv:1410.8516.
  8. Density Estimation Using Real NVP. In 5th International Conference on Learning Representations. OpenReview.net.
  9. Quasiconformal 4-manifolds. Acta Mathematica, 163(none):181 – 252.
  10. Nonparametric Estimation under Shape Constraints, volume 38. Cambridge University Press.
  11. Gruet, M.-A. (1996). A nonparametric calibration analysis. The Annals of Statistics, 24(4):1474–1492.
  12. Global risk bounds and adaptation in univariate convex regression. Probability Theory and Related Fields, 163(1):379–411.
  13. Hatcher, A. (2002). Algebraic Topology. Cambridge University Press, Cambridge.
  14. Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2722–2730. PMLR.
  15. Horowitz, J. L. (2011). Applied nonparametric instrumental variables estimation. Econometrica, 79(2):347–394.
  16. Neural Autoregressive Flows. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2078–2087. PMLR.
  17. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in Neural Information Processing Systems, 29:3765–3773.
  18. Sum-of-Squares Polynomial Flow. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 3009–3018. PMLR.
  19. Kawamura, Y. (1979). Invertibility of Lipschitz Continuous Mappings and Its Application to Electrical Network Equations. SIAM Journal on Mathematical Analysis, 10(2):253–265.
  20. Adaptation in log-concave density estimation. The Annals of Statistics, 46(5):2279–2306.
  21. Improved Variational Inference with Inverse Autoregressive Flow. In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc.
  22. Nonparametric calibration. Technometrics, 26(3):233–241.
  23. Normalizing flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11):3964–3979.
  24. The expressive power of a class of normalizing flow models. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 3599–3609. PMLR.
  25. Krief, J. M. (2017). Direct instrumental nonparametric estimation of inverse regression functions. Journal of Econometrics, 201(1):95–107.
  26. Estimating monotone functions. Statistics & probability letters, 56(4), 361-367.
  27. Instrumental variable estimation of nonparametric models. Econometrica, 71(5):1565–1578.
  28. Osborne, C. (1991). Statistical Calibration: A Review. International Statistical Review/Revue Internationale de Statistique, 59(3):309–336.
  29. Masked Autoregressive Flow for Density Estimation. In Advances in Neural Information Processing Systems, volume 30, pages 2338–2347. Curran Associates, Inc.
  30. Pourciau, B. (1988). Global Invertibility of Nonsmooth Mappings. Journal of Mathematical Analysis and Applications, 131(1):170–179.
  31. Variational Inference with Normalizing Flows. In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 1530–1538. PMLR.
  32. Normalizing Flows on Tori and Spheres. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 8083–8092. PMLR.
  33. A two-stage hybrid procedure for estimating an inverse regression function. The Annals of Statistics, 39(2):956–989.
  34. Two-stage plans for estimating the inverse of a monotone function. Technometrics, 57(3):395–407.
  35. Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators. In Advances in Neural Information Processing Systems, volume 33, pages 3362–3373. Curran Associates, Inc.
  36. Tsybakov, A. B. (2008). Introduction to nonparametric estimation. Springer Science & Business Media.
  37. Frequentist coverage and sup-norm convergence rate in gaussian process regression. arXiv preprint arXiv:1708.04753.
  38. Supremum norm posterior contraction and credible sets for nonparametric multivariate regression. The Annals of Statistics, 44(3):1069–1102.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up