Federated Causal Discovery from Heterogeneous Data (2402.13241v2)
Abstract: Conventional causal discovery methods rely on centralized data, which is inconsistent with the decentralized nature of data in many real-world situations. This discrepancy has motivated the development of federated causal discovery (FCD) approaches. However, existing FCD methods may be limited by their potentially restrictive assumptions of identifiable functional causal models or homogeneous data distributions, narrowing their applicability in diverse scenarios. In this paper, we propose a novel FCD method attempting to accommodate arbitrary causal models and heterogeneous data. We first utilize a surrogate variable corresponding to the client index to account for the data heterogeneity across different clients. We then develop a federated conditional independence test (FCIT) for causal skeleton discovery and establish a federated independent change principle (FICP) to determine causal directions. These approaches involve constructing summary statistics as a proxy of the raw data to protect data privacy. Owing to the nonparametric properties, FCIT and FICP make no assumption about particular functional forms, thereby facilitating the handling of arbitrary causal models. We conduct extensive experiments on synthetic and real datasets to show the efficacy of our method. The code is available at https://github.com/lokali/FedCDH.git.
- Fed-cd: Federated causal discovery from interventional and observational data. arXiv preprint arXiv:2211.03846, 2022.
- A survey on homomorphic encryption schemes: Theory and implementation. ACM Computing Surveys (Csur), 51(4):1–35, 2018.
- Distributed simulated annealing. Distributed Constraint Problem Solving and Reasoning in Multi-Agent Systems, 112, 2004.
- Marcus Bendtsen. Regime aware learning. In Conference on Probabilistic Graphical Models, pp. 1–12. PMLR, 2016.
- Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine learning, 3(1):1–122, 2011.
- The chronnectome: time-varying connectivity networks as the next frontier in fmri data discovery. Neuron, 84(2):262–274, 2014.
- Chandler Squires. causaldag: creation, manipulation, and learning of causal models. https://github.com/uhlerlab/causaldag, 2018.
- Learning bayesian network structure from distributed data. In Proceedings of the 2003 SIAM International Conference on Data Mining, pp. 284–288. SIAM, 2003.
- David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learning research, 3(Nov):507–554, 2002.
- Secure multiparty computation. Cambridge University Press, 2015.
- JJ Daudin. Partial association measures and an application to qualitative regression. Biometrika, 67(3):581–590, 1980.
- A simple algorithm to construct a consistent extension of a partially oriented graph. Technicial Report R-185, Cognitive Systems Laboratory, UCLA, 1992.
- On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci, 5(1):17–60, 1960.
- Kernel measures of conditional dependence. Advances in neural information processing systems, 20, 2007.
- Feddag: Federated dag structure learning. Transactions on Machine Learning Research, 2022.
- Slawomir Goryczka and Li Xiong. A comprehensive comparison of multiparty secure additions with differential privacy. IEEE transactions on dependable and secure computing, 14(5):463–477, 2015.
- Learning bayesian network structure from distributed homogeneous data. In Eighth acis international conference on software engineering, artificial intelligence, networking, and parallel/distributed computing (snpd 2007), volume 3, pp. 250–254. IEEE, 2007.
- A kernel statistical test of independence. Advances in neural information processing systems, 20, 2007.
- Nonlinear causal discovery with additive noise models. Advances in neural information processing systems, 21, 2008.
- Identification of time-dependent causal model: A gaussian process treatment. In Twenty-Fourth international joint conference on artificial intelligence, 2015.
- Causal discovery from heterogeneous/nonstationary data. The Journal of Machine Learning Research, 21(1):3482–3534, 2020.
- Towards privacy-aware causal structure learning in federated setting. arXiv preprint arXiv:2211.06919, 2022.
- Federated learning for sparse bayesian models with applications to electronic health records and genomics. In PACIFIC SYMPOSIUM ON BIOCOMPUTING 2023: Kohala Coast, Hawaii, USA, 3–7 January 2023, pp. 484–495. World Scientific, 2022.
- Efficient neural causal discovery without acyclicity constraints. International Conference on Learning Representations, 2022.
- Amortized inference for causal structure learning. Advances in Neural Information Processing Systems, 35:13104–13118, 2022.
- Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp. 1273–1282. PMLR, 2017.
- Nothing but regrets—privacy-preserving federated causal discovery. In International Conference on Artificial Intelligence and Statistics, pp. 8263–8278. PMLR, 2023.
- Distributed bayesian network structure learning. In 2010 IEEE International Symposium on Industrial Electronics, pp. 1607–1611. IEEE, 2010.
- Towards federated bayesian network structure learning with continuous optimization. In International Conference on Artificial Intelligence and Statistics, pp. 8095–8111. PMLR, 2022.
- On the convergence of continuous constrained optimization for structure learning. In International Conference on Artificial Intelligence and Statistics, 2022.
- Structure learning with continuous optimization: A sober look and beyond. arXiv preprint arXiv:2304.02146, 2023.
- Causal discovery in machine learning: Theories and applications. Journal of Dynamics & Games, 8(3):203, 2021.
- Identifiability of gaussian structural equation models with equal error variances. Biometrika, 101(1):219–228, 2014.
- Long-term neural and physiological phenotyping of a single human. Nature communications, 6(1):8885, 2015.
- Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007.
- Beware of the simulated DAG! causal discovery benchmarks may be easy to game. In Advances in Neural Information Processing Systems, 2021.
- Causal structure discovery from distributions arising from mixtures of dags. In International Conference on Machine Learning, pp. 8336–8345. PMLR, 2020.
- Privacy-preserving bayesian network for horizontally partitioned data. In 2009 International Conference on Computational Science and Engineering, volume 3, pp. 9–16. IEEE, 2009.
- Challenges and opportunities with causal discovery algorithms: application to alzheimer’s pathophysiology. Scientific reports, 10(1):1–12, 2020.
- A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
- Peter Spirtes. An anytime algorithm for causal inference. In International Workshop on Artificial Intelligence and Statistics, pp. 278–285. PMLR, 2001.
- Causal discovery and inference: concepts and recent methodological advances. In Applied informatics, volume 3, pp. 1–28. SpringerOpen, 2016.
- Causation, prediction, and search. MIT press, 2000.
- Approximate kernel-based conditional independence tests for fast non-parametric causal discovery. Journal of Causal Inference, 7(1), 2019.
- On the error of random fourier features. arXiv preprint arXiv:1506.02785, 2015.
- Neuropathic pain diagnosis simulator for causal discovery algorithm evaluation. Advances in Neural Information Processing Systems, 32, 2019.
- Causal discovery in manufacturing: A structured literature review. Journal of Manufacturing and Materials Processing, 6(1):10, 2022.
- Towards practical federated causal structure learning. arXiv preprint arXiv:2306.09433, 2023.
- DAGs with no fears: A closer look at continuous optimization for learning Bayesian networks. In Advances in Neural Information Processing Systems, 2020.
- Robert F Woolson. Wilcoxon signed-rank test. Wiley encyclopedia of clinical trials, pp. 1–3, 2007.
- A state-space mixed membership blockmodel for dynamic network tomography. The Annals of Applied Statistics, 4(2):535–566, 2010.
- Federated learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 13(3):1–207, 2019.
- Distributed learning of generalized linear causal networks. arXiv preprint arXiv:2201.09194, 2022.
- Dag-gnn: Dag structure learning with graph neural networks. In International Conference on Machine Learning, pp. 7154–7163. PMLR, 2019.
- Dags with no curl: An efficient dag structure learning approach. In International Conference on Machine Learning, pp. 12156–12166. PMLR, 2021.
- Extensions of ica for causality discovery in the hong kong stock market. In International Conference on Neural Information Processing, pp. 400–409. Springer, 2006.
- On the identifiability of the post-nonlinear causal model. arXiv preprint arXiv:1205.2599, 2012.
- Kernel-based conditional independence test and application in causal discovery. arXiv preprint arXiv:1202.3775, 2012.
- Discovery and visualization of nonstationary causal models. arXiv preprint arXiv:1509.08056, 2015.
- Dags with no tears: Continuous optimization for structure learning. Advances in Neural Information Processing Systems, 31, 2018.
- Causal discovery with heterogeneous observational data. In Uncertainty in Artificial Intelligence, pp. 2383–2393. PMLR, 2022.