2000 character limit reached
Penalized Generative Variable Selection (2402.16661v1)
Published 26 Feb 2024 in stat.ML, cs.LG, and stat.ME
Abstract: Deep networks are increasingly applied to a wide variety of data, including data with high-dimensional predictors. In such analysis, variable selection can be needed along with estimation/model building. Many of the existing deep network studies that incorporate variable selection have been limited to methodological and numerical developments. In this study, we consider modeling/estimation using the conditional Wasserstein Generative Adversarial networks. Group Lasso penalization is applied for variable selection, which may improve model estimation/prediction, interpretability, stability, etc. Significantly advancing from the existing literature, the analysis of censored survival data is also considered. We establish the convergence rate for variable selection while considering the approximation error, and obtain a more efficient distribution estimation. Simulations and the analysis of real experimental data demonstrate satisfactory practical utility of the proposed analysis.
- On deep learning as a remedy for the curse of dimensionality in nonparametric regression. The Annals of Statistics, 47(4):2261 – 2285.
- Characterizations of łojasiewicz inequalities: subgradient flows, talweg, convexity. Transactions of the American Mathematical Society, 362(6):3319–3363.
- Nonlinear variable selection via deep neural networks. Journal of Computational and Graphical Statistics, 30(2):484–492.
- Optimal and safe estimation for high-dimensional semi-supervised learning. Journal of the American Statistical Association, 0(0):1–12.
- Consistent feature selection for analytic deep neural networks. Advances in Neural Information Processing Systems, 33:2420–2431.
- Deep neural networks for estimation and inference. Econometrica, 89(1):181–213.
- Sparse-input neural networks for high-dimensional nonparametric regression and classification. arXiv preprint arXiv:1711.07592.
- Improved training of wasserstein gans. Advances in neural information processing systems, 30.
- Gradient-induced model-free variable selection with composite quantile regression. Statistica Sinica, 28(3):1521–1538.
- Variable selection in nonparametric additive models. The Annals of Statistics, 38(4):2282–2313.
- Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors. The Annals of Statistics, 51(2):691 – 716.
- Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1–9.
- On the inhibitory effect of albumin on platelet aggregation. Thrombosis research, 17(1):13–18.
- Kallenberg, O. (2002). Foundations of Modern Probability. Springer.
- Lei, J. (2020). Convergence and concentration of empirical measures under Wasserstein distance in unbounded functional spaces. Bernoulli, 26(1):767 – 798.
- Lassonet: A neural network with feature sparsity. The Journal of Machine Learning Research, 22(1):5633–5661.
- Better approximations of high dimensional smooth functions by deep neural networks with rectified power units. Communications in Computational Physics.
- Powernet: Efficient representations of polynomials and smooth functions by deep neural networks with rectified power units. Journal of Mathematical Study, 53(2):159–191.
- Deep feature selection: theory and application to identify enhancers and promoters. Journal of Computational Biology, 23(5):322–336.
- Bayesian neural networks for selection of drug sensitive genes. Journal of the American Statistical Association, 113(523):955–972.
- Spam: Sparse additive models. In Advances in Neural Information Processing Systems, volume 20.
- Wasserstein generative learning of conditional distribution. arXiv, 2112.10039.
- Deeppink: reproducible feature selection in deep neural networks. Advances in neural information processing systems, 31.
- A universal approximation theorem of deep neural networks for expressing probability distributions. volume 33, pages 3094–3105.
- Sparse-input neural network using group concave regularization. arXiv preprint arXiv:2307.00344.
- Quantile regression with relu networks: Estimators and minimax rates. Journal of Machine Learning Research, 23:247:1–247:42.
- Pham, T. S. (2012). An explicit bound for the Łojasiewicz exponent of real polynomials. Kodai Mathematical Journal, 35(2):311 – 319.
- Genotypic predictors of human immunodeficiency virus type 1 drug resistance. Proceedings of the National Academy of Sciences, 103(46):17355–17360.
- Group sparse regularization for deep neural networks. Neurocomputing, 241:81–89.
- Schmidt-Hieber, J. (2020). Nonparametric regression using deep neural networks with relu activation function. Annals of statistics, 48:1875–1897.
- Estimation of non-crossing quantile regression process with deep requ neural networks. arXiv preprint arXiv:2207.10442.
- Robust nonparametric regression with deep neural networks. arXiv preprint arXiv:2107.10343.
- Stute, W. (1996). Distributional convergence under random censorship when covariables are present. Scandinavian Journal of Statistics, 23(4):461–471.
- Consistent sparse deep learning: Theory and computation. Journal of the American Statistical Association, 117(540):1981–1995.
- Albumin, white blood cell count, and body mass index improve discrimination of mortality in hiv-positive individuals. AIDS (London, England), 33(5):903.
- Weak convergence. Springer.
- Villani, C. (2009). Optimal Transport: Old and New. Springer.
- On the capacity of deep generative networks for approximating distributions. Neural Networks, 145:144–154.
- Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(1):49–67.
- Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(2):894 – 942.
- Heterogeneous feature selection with multi-modal deep neural networks and sparse group lasso. IEEE Transactions on Multimedia, 17(11):1936–1948.
- Regularization and variable selection via the elastic net. Journal of the royal statistical society: series B (statistical methodology), 67(2):301–320.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
