Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 147 tok/s
Gemini 2.5 Pro 52 tok/s Pro
GPT-5 Medium 27 tok/s Pro
GPT-5 High 30 tok/s Pro
GPT-4o 96 tok/s Pro
Kimi K2 188 tok/s Pro
GPT OSS 120B 398 tok/s Pro
Claude Sonnet 4.5 36 tok/s Pro
2000 character limit reached

DRCFS: Doubly Robust Causal Feature Selection (2306.07024v3)

Published 12 Jun 2023 in cs.LG and stat.ME

Abstract: Knowing the features of a complex system that are highly relevant to a particular target variable is of fundamental interest in many areas of science. Existing approaches are often limited to linear settings, sometimes lack guarantees, and in most cases, do not scale to the problem at hand, in particular to images. We propose DRCFS, a doubly robust feature selection method for identifying the causal features even in nonlinear and high dimensional settings. We provide theoretical guarantees, illustrate necessary conditions for our assumptions, and perform extensive experiments across a wide range of simulated and semi-synthetic datasets. DRCFS significantly outperforms existing state-of-the-art methods, selecting robust features even in challenging highly non-linear and high-dimensional problems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Generalized random forests. The Annals of Statistics, 47(2):1148–1178, 2019.
  2. The alarm monitoring system: A case study with two probabilistic inference techniques for belief networks. In AIME 89: Second European Conference on Artificial Intelligence in Medicine, London, August 29th–31st 1989. Proceedings, pages 247–256. Springer, 1989.
  3. Y. Benjamini and D. Yekutieli. The control of the false discovery rate in multiple testing under dependency. Annals of statistics, pages 1165–1188, 2001.
  4. Feature Selection for High-Dimensional Data. Springer, 2015.
  5. Foundations of structural causal models with cycles and latent variables. The Annals of Statistics, 49(5):2885 – 2915, 2021.
  6. B. Butcher and B. J. Smith. Feature engineering and selection: A practical approach for predictive models: by max kuhn and kjell johnson. boca raton, fl: Chapman & hall/crc press, 2019, xv+ 297 pp., 79.95 (h), isbn: 978-1-13-807922-9., 2020.
  7. G. C. Cawley. Causal & non-causal feature selection for ridge regression. In Causation and Prediction Challenge, 2008.
  8. Automatic debiased machine learning via neural nets for generalized linear regression. arXiv:2104.14737, 2021.
  9. RieszNet and ForestRiesz: Automatic debiased machine learning with neural nets and random forests. In Proc. of ICML, pages 3901–3914, 2022.
  10. D. M. Chickering. Optimal structure identification with greedy search. Journal of Machine Learning Research, 3:507–554, 2003.
  11. Large-sample learning of bayesian networks is np-hard. Journal of Machine Learning Research, 5:1287–1330, 2004.
  12. Learning sparse causal models is not np-hard. In Proc. of UAI, page 172–181, 2013.
  13. D. Colombo and M. H. Maathuis. Order-independent constraint-based causal structure learning. Journal of Machine Learning Research, 15(116):3921–3962, 2014.
  14. On-line student modeling for coached problem solving using bayesian networks. In User Modeling: Proceedings of the Sixth International Conference UM97 Chia Laguna, Sardinia, Italy June 2–5 1997, pages 231–242. Springer, 1997.
  15. Least angle regression. The Annals of statistics, 32(2):407–499, 2004.
  16. I. Guyon. Practical feature selection: from correlation to causality. Mining massive data sets for security: advances in data mining, search, social networks and text mining, and their applications to security, pages 27–43, 2008.
  17. I. Guyon and C. Aliferis. Causal feature selection. In Computational methods of feature selection. Chapman and Hall/CRC, 2007.
  18. Causal structure learning. Annual Review of Statistics and Its Application, 5:371–391, 2018.
  19. Nonlinear causal discovery with additive noise models. In Proc. of NIPS, pages 689–696, 2008.
  20. Application of causal inference to genomic analysis: advances in methodology. Frontiers in Genetics, 9:238, 2018.
  21. Feature relevance quantification in explainable ai: A causal problem. In Proc. of AISTATS, pages 2907–2916, 2020.
  22. A. Javanmard et al. De-biasing the lasso: Optimal sample size for gaussian designs. arxiv, 2015.
  23. K. Kira and L. A. Rendell. A practical approach to feature selection. In Machine Learning Proceedings 1992, pages 249–256. Elsevier, 1992.
  24. Exact post-selection inference, with application to the lasso. The Annals of Statistics, 44(3):907–927, 2016.
  25. Feature selection: A data perspective. ACM Computing Surveys, 50(6):1–45, 2017.
  26. Gut microbiome structure and association with host factors in a korean population. Msystems, 6(4):e00179–21, 2021.
  27. H. Liu and H. Motoda. Computational Methods of Feature Selection. CRC press, 2007.
  28. A significance test for forward stepwise model selection. arXiv:1405.3920, 2014.
  29. Causal inference in microbiome medicine: principles and applications. Trends in microbiology, 29(8):736–746, 2021.
  30. O. J. Maclaren and R. Nicholson. What can be estimated? identifiability, estimability, causal inference and ill-posed inverse problems. arXiv:1904.02826, 2019.
  31. Selecting causal brain features with a single conditional independence test per feature. In Proc. of NeurIPS, pages 12532–12543, 2019.
  32. Genedisco: A benchmark for experimental design in drug discovery. arXiv:2110.11875, 2021.
  33. K. P. Murphy. Active learning of causal bayes net structure. Technical report, technical report, UC Berkeley, 2001.
  34. High-dimensional consistency in score-based and hybrid structure learning. Annals of Statistics, 46, 03 2016.
  35. R. Opgen-Rhein and K. Strimmer. From correlation to causation networks: a simple approximate learning algorithm and its application to high-dimensional plant gene expression data. BMC systems biology, 1(1):1–10, 2007.
  36. M. Paul. Feature selection as causal inference: Experiments with text classification. In Proc. of CoNLL, pages 163–172, 2017.
  37. J. Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, 2000.
  38. J. Pearl. Direct and indirect effects. In Proc. of UAI, pages 411–420, 2001.
  39. J. Pearl. Causality: Models, Reasoning and Inference, 2nd Ed. Cambridge University Press, 2009.
  40. Identifiability of causal graphs using functional models. In Proc. of UAI, pages 589–598, 2011.
  41. Causal discovery with continuous additive noise models. Journal of Machine Learning Research, 15(1):2009–2053, 2014.
  42. Combining whole-genome shotgun sequencing and rrna gene amplicon analyses to improve detection of microbe–microbe interaction networks in plant leaves. The ISME Journal, 14(8):2116–2130, 2020.
  43. Characterization of parameters with a mixed bias property. Biometrika, 108(1):231–238, 08 2020.
  44. Detecting and quantifying causal associations in large nonlinear time series datasets. Science Advances, 5(11):eaau4996, 2019.
  45. On causal and anticausal learning. In Proc. of ICML, 2012.
  46. M. Scutari. Learning bayesian networks with the bnlearn R package. Journal of Statistical Software, 35(3):1–22, 2010. doi: 10.18637/jss.v035.i03.
  47. R. D. Shah and J. Peters. The hardness of conditional independence testing and the generalised covariance measure. The Annals of Statistics, 48(3):1514–1538, 2020.
  48. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(Oct):2003–2030, 2006a.
  49. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7:2003–2030, 2006b.
  50. J. Siebert. Applications of statistical causal inference in software engineering. arXiv:2211.11482, 2022.
  51. Causal feature selection via orthogonal search. Transactions on Machine Learning Research, 2022.
  52. Causation, Prediction, and Search, 2nd Ed. MIT Press, 2000.
  53. Exact post-selection inference for sequential regression procedures. Journal of the American Statistical Association, 111(514):600–620, 2016.
  54. The max-min hill-climbing bayesian network structure learning algorithm. Machine Learning, 65:31–78, 10 2006.
  55. Evaluating digital tools for sustainable agriculture using causal inference. arXiv:2211.03195, 2022.
  56. H. R. Varian. Causal inference in economics and marketing. Proceedings of the National Academy of Sciences, 113(27):7310–7315, 2016.
  57. Modeling air pollution, climate, and health data using bayesian networks: A case study of the english regions. Earth and Space Science, 5(4):76–88, 2018.
  58. D. Williams. Probability with Martingales. Cambridge University Press, 1991.
  59. L. Yang and J. Chen. A comprehensive evaluation of microbial differential abundance analysis methods: current status and potential solutions. Microbiome, 10(1):130, 2022.
  60. Causality-based feature selection: Methods and evaluations. ACM Computing Surveys, 53(5):1–36, 2020.
  61. A unified view of causal and non-causal feature selection. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(4):1–46, 2021.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up