Data collaboration for causal inference from limited medical testing and medication data (2501.06511v2)

Abstract: Observational studies enable causal inferences when randomized controlled trials (RCTs) are not feasible. However, integrating sensitive medical data across multiple institutions introduces significant privacy challenges. The data collaboration quasi-experiment (DC-QE) framework addresses these concerns by sharing "intermediate representations" -- dimensionality-reduced data derived from raw data -- instead of the raw data. While the DC-QE can estimate treatment effects, its application to medical data remains unexplored. This study applied the DC-QE framework to medical data from a single institution to simulate distributed data environments under independent and identically distributed (IID) and non-IID conditions. We propose a novel method for generating intermediate representations within the DC-QE framework. Experimental results demonstrated that DC-QE consistently outperformed individual analyses across various accuracy metrics, closely approximating the performance of centralized analysis. The proposed method further improved performance, particularly under non-IID conditions. These outcomes highlight the potential of the DC-QE framework as a robust approach for privacy-preserving causal inferences in healthcare. Broader adoption of this framework and increased use of intermediate representations could grant researchers access to larger, more diverse datasets while safeguarding patient confidentiality. This approach may ultimately aid in identifying previously unrecognized causal relationships, support drug repurposing efforts, and enhance therapeutic interventions for rare diseases.