Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Additive interaction modelling using I-priors (2007.15766v4)

Published 30 Jul 2020 in math.ST, stat.ME, stat.ML, and stat.TH

Abstract: Additive regression models with interactions are widely studied in the literature, using methods such as splines or Gaussian process regression. However, these methods can pose challenges for estimation and model selection, due to the presence of many smoothing parameters and the lack of suitable criteria. We propose to address these challenges by extending the I-prior methodology (Bergsma, 2020) to multiple covariates, which may be multidimensional. The I-prior methodology has some advantages over other methods, such as Gaussian process regression and Tikhonov regularization, both theoretically and practically. In particular, the I-prior is a proper prior, is based on minimal assumptions, yields an admissible posterior mean, and estimation of the scale (or smoothing) parameters can be done using an EM algorithm with simple E and M steps. Moreover, we introduce a parsimonious specification of models with interactions, which has two benefits: (i) it reduces the number of scale parameters and thus facilitates the estimation of models with interactions, and (ii) it enables straightforward model selection (among models with different interactions) based on the marginal likelihood.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Alpay, D. (1991). Some remarks on reproducing kernel Krein spaces. Rocky Mountain J. Math.
  2. Linear operators in spaces with an indefinite metric. John Wiley & Sons, Incorporated.
  3. Bergsma, W. P. (2004). Testing conditional independence for continuous random variables. Eurandom technical report, 2004-048.
  4. Bergsma, W. P. (2020). Regression with I-priors. Econometrics and Statistics, 14:89–111.
  5. Reproducing kernel Hilbert spaces in probability and statistics. Kluwer Academic.
  6. The conditional permutation test for independence while controlling for confounders. J. Roy. Statist. Soc. B.
  7. Bognár, J. (1974). Indefinite inner product spaces, volume 78. Springer Science & Business Media.
  8. Hybrid regularisation and the (in)admissibility of ridge regression in infinite dimensional Hilbert spaces. Bernoulli, 25(3):1939–1976.
  9. Clyde, M. (2022). BAS: Bayesian Variable Selection and Model Averaging using Bayesian Adaptive Sampling. R package version 1.6.2.
  10. Denwood, M. J. (2016). runjags: An R package providing interface utilities, model templates, parallel computing methods and additional distributions for MCMC models in JAGS. J. Stat. Softw., 71:1–25.
  11. Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw., 33(1):1–22.
  12. Friedman, J. H. (1991). Multivariate adaptive regression splines. Ann. Stat., pages 1–67.
  13. Variable Selection Via Gibbs Sampling. J. Am. Stat. Assoc., 88(423):881–889.
  14. Gheondea, A. (2013). A survey on reproducing kernel Krein spaces. arXiv preprint arXiv:1309.2393.
  15. Gu, C. (2013). Smoothing spline ANOVA models. Springer.
  16. Jamil, H. (2018). Regression modelling using priors depending on Fisher information covariance kernels (I-priors). PhD thesis, The London School of Economics and Political Science (LSE).
  17. Jamil, H. (2019). iprior: Regression Modelling using I-priors. R package version 0.7.3.
  18. iprior: An R package for regression modelling using I-priors. https://arxiv.org/abs/1912.01376.
  19. Bayesian variable selection for linear models using I-priors. In Abdul Karim, S. A., editor, Theoretical, Modelling and Numerical Simulations Toward Industry 4.0, Studies in Systems, Decision and Control, pages 107–132. Springer, Singapore.
  20. Kenward, M. G. (1987). A method for comparing profiles of repeated measurements. Applied Statistics, pages 296–308.
  21. A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Stat., 41(2):495–502.
  22. Neveu, J. (1968). Processus aléatoires gaussiens. Les presses de l’Université de Montréal.
  23. Ntzoufras, I. (2011). Bayesian Modeling Using WinBUGS. Wiley.
  24. Modeling nonstationary longitudinal data. Biometrics, 56(3):699–705.
  25. Learning with non-positive kernels. In ICML 21, pages 639–646.
  26. On modelling mean-covariance structures in longitudinal studies. Biometrika, 90(1):239–244.
  27. Picard, J. (2011). Representation formulae for the fractional Brownian motion. In Séminaire de Probabilités XLIII, pages 3–70. Springer.
  28. Plummer, M. (2003). JAGS: A Program for Analysis of Bayesian Graphical Models Using Gibbs Sampling. In Hornik, K., Leisch, F., and Zeileis, A., editors, Proc. DSC 2003).
  29. Pourahmadi, M. (2000). Maximum likelihood estimation of generalised linear models for multivariate normal covariance matrix. Biometrika, 87(2):425–435.
  30. Gaussian processes for machine learning. MIT press Cambridge, MA.
  31. The hardness of conditional independence testing and the generalised covariance measure. J. Roy. Statist. Soc. B.
  32. Stanley, R. P. (2011). Enumerative Combinatorics, Vol. 1. Cambridge Univ. Press.
  33. Stone, C. J. (1985). Additive regression and other nonparametric models. Ann. Stat., 13(2):689–705.
  34. Wahba, G. (1986). Partial and interaction splines for the semiparametric estimation. Department of Statistics Technical Report No. 784. University of Wisconsin, Madison.
  35. Wahba, G. (1990). Spline models for observational data, volume 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia, PA.
  36. Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with g-prior distributions. Bayesian inference and decision techniques: Essays in Honor of Bruno De Finetti, 6:233–243.
  37. A joint modelling approach for longitudinal studies. J. Roy. Statist. Soc. B.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now