Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Structural transfer learning of non-Gaussian DAG (2310.10239v1)

Published 16 Oct 2023 in stat.ML, cs.LG, and stat.ME

Abstract: Directed acyclic graph (DAG) has been widely employed to represent directional relationships among a set of collected nodes. Yet, the available data in one single study is often limited for accurate DAG reconstruction, whereas heterogeneous data may be collected from multiple relevant studies. It remains an open question how to pool the heterogeneous data together for better DAG structure reconstruction in the target study. In this paper, we first introduce a novel set of structural similarity measures for DAG and then present a transfer DAG learning framework by effectively leveraging information from auxiliary DAGs of different levels of similarities. Our theoretical analysis shows substantial improvement in terms of DAG reconstruction in the target study, even when no auxiliary DAG is overall similar to the target DAG, which is in sharp contrast to most existing transfer learning methods. The advantage of the proposed transfer DAG learning is also supported by extensive numerical experiments on both synthetic data and multi-site brain functional connectivity network data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Agoalikum, E., Klugah-Brown, B., Hongzhou, W., Hu, P., Jing, J., and Biswal, B. B. (2023), “Structural differences among children, adolescents, and adults with attention deficit hyperactivity disorder and abnormal granger causality of the right pallidum and whole-brain,” Frontiers in Human Neuroscience, 17, 40.
  2. Bellec, P., Chu, C., Chouinard-Decorte, F., Benhajali, Y., Margulies, D. S., and Craddock, R. C. (2017), “The neuro bureau ADHD-200 preprocessed repository,” Neuroimage, 144, 275–286.
  3. Bonnet, L., Comte, A., Tatu, L., Millot, J.-L., Moulin, T., and Medeiros de Bustos, E. (2015), “The role of the amygdala in the perception of positive emotions: an “intensity detector”,” Frontiers in Behavioral Neuroscience, 9, 178.
  4. Cai, T. T. and Wei, H. (2021), “Transfer learning for nonparametric classification: Minimax rate and adaptive classifier,” The Annals of Statistics, 49, 100–128.
  5. Chen, W., Drton, M., and Wang, Y. S. (2019), “On causal discovery with an equal-variance assumption,” Biometrika, 106, 973–980.
  6. Chen, X., Sun, H., Ellington, C., Xing, E., and Song, L. (2021), “Multi-task learning of order-consistent causal graphs,” Advances in Neural Information Processing Systems, 34, 11083–11095.
  7. Danks, D., Glymour, C., and Tillman, R. (2008), “Integrating locally learned causal structures with overlapping variables,” Advances in Neural Information Processing Systems, 21.
  8. Ghoshal, A. and Honorio, J. (2018), “Learning linear structural equation models in polynomial time and sample complexity,” in International Conference on Artificial Intelligence and Statistics, PMLR.
  9. Giedd, J. N., Blumenthal, J., Molloy, E., and Castellanos, F. X. (2001), “Brain imaging of attention deficit/hyperactivity disorder,” Annals of the New York Academy of Sciences, 931, 33–49.
  10. Huang, B., Zhang, K., Gong, M., and Glymour, C. (2020), “Causal discovery from multiple data sets with non-identical variable sets,” in Proceedings of the AAAI conference on artificial intelligence, volume 34.
  11. Hyvärinen, A. and Smith, S. M. (2013), “Pairwise likelihood ratios for estimation of non-Gaussian structural equation models,” The Journal of Machine Learning Research, 14, 111–152.
  12. Kalisch, M. and Bühlman, P. (2007), “Estimating high-dimensional directed acyclic graphs with the PC-algorithm.” Journal of Machine Learning Research, 8.
  13. Lam, C. and Fan, J. (2009), “Sparsistency and rates of convergence in large covariance matrix estimation,” The Annals of Statistics, 37, 4254–4278.
  14. Li, S., Cai, T., and Duan, R. (2023b), “Targeting underrepresented populations in precision medicine: A federated transfer learning approach,” The Annals of Applied Statistics, 1–25.
  15. Li, S., Cai, T. T., and Li, H. (2022a), “Transfer learning for high-dimensional linear regression: Prediction, estimation, and minimax optimality,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 1–26.
  16. — (2022b), “Transfer learning in large-scale gaussian graphical models with false discovery rate control,” Journal of the American Statistical Association, 1–13.
  17. Li, S., Zhang, L., Cai, T. T., and Li, H. (2023a), “Estimation and inference for high-dimensional generalized linear models with knowledge transfer,” Journal of the American Statistical Association, 1–12.
  18. Liu, J., Sun, W., and Liu, Y. (2019), “Joint skeleton estimation of multiple directed acyclic graphs for heterogeneous population,” Biometrics, 75, 36–47.
  19. Liu, W. and Luo, X. (2015), “Fast and adaptive sparse precision matrix estimation in high dimensions,” Journal of Multivariate Analysis, 135, 153–162.
  20. Mooij, J. M., Magliacane, S., and Claassen, T. (2020), “Joint causal inference from multiple contexts,” The Journal of Machine Learning Research, 21, 3919–4026.
  21. Park, G. (2020), “Identifiability of additive noise models using conditional variances,” The Journal of Machine Learning Research, 21, 2896–2929.
  22. Peters, J. and Bühlmann, P. (2014), “Identifiability of Gaussian structural equation models with equal error variances,” Biometrika, 101, 219–228.
  23. Ravikumar, P., Wainwright, M. J., Raskutti, G., and Yu, B. (2011), “High-dimensional covariance estimation by minimizing ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-penalized log-determinant divergence,” Electronic Journal of Statistics, 5, 935–980.
  24. Reeve, H. W., Cannings, T. I., and Samworth, R. J. (2021), “Adaptive transfer learning,” The Annals of Statistics, 49, 3618–3649.
  25. Shimizu, S. (2012), “Joint estimation of linear non-Gaussian acyclic models,” Neurocomputing, 81, 104–107.
  26. Shimizu, S., Hoyer, P. O., Hyvärinen, A., Kerminen, A., and Jordan, M. (2006), “A linear non-Gaussian acyclic model for causal discovery.” Journal of Machine Learning Research, 7.
  27. Székely, G. J., Rizzo, M. L., and Bakirov, N. K. (2007), “Measuring and testing dependence by correlation of distances,” The Annals of Statistics, 35, 2769–2794.
  28. Tian, Y. and Feng, Y. (2022), “Transfer learning under high-dimensional generalized linear models,” Journal of the American Statistical Association, 1–30.
  29. Tomasi, D. and Volkow, N. D. (2012), “Abnormal functional connectivity in children with attention-deficit/hyperactivity disorder,” Biological Psychiatry, 71, 443–450.
  30. Triantafillou, S. and Tsamardinos, I. (2015), “Constraint-based causal discovery from multiple interventions over overlapping variable sets,” The Journal of Machine Learning Research, 16, 2147–2205.
  31. Tsamardinos, I., Brown, L. E., and Aliferis, C. F. (2006), “The max-min hill-climbing Bayesian network structure learning algorithm,” Machine learning, 65, 31–78.
  32. Wang, Y., Segarra, S., and Uhler, C. (2020), “High-dimensional joint estimation of multiple directed Gaussian graphical models,” Electronic Journal of Statistics, 14, 2439 – 2483.
  33. Wang, Y. S. and Drton, M. (2020), “High-dimensional causal discovery under non-Gaussianity,” Biometrika, 107, 41–59.
  34. Yuan, Y., Shen, X., Pan, W., and Wang, Z. (2019), “Constrained likelihood for reconstructing a directed acyclic Gaussian graph,” Biometrika, 106, 109–125.
  35. Zhang, H., Zhao, Y., Cao, W., Cui, D., Jiao, Q., Lu, W., Li, H., and Qiu, J. (2020), “Aberrant functional connectivity in resting state networks of ADHD patients revealed by independent component analysis,” BMC neuroscience, 21, 1–11.
  36. Zhao, R., He, X., and Wang, J. (2022), “Learning linear non-Gaussian directed acyclic graph with diverging number of nodes,” Journal of Machine Learning Research, 23, 1–34.
  37. Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., and He, Q. (2020), “A comprehensive survey on transfer learning,” Proceedings of the IEEE, 109, 43–76.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now