Metadata-guided Feature Disentanglement for Functional Genomics (2405.19057v1)
Abstract: With the development of high-throughput technologies, genomics datasets rapidly grow in size, including functional genomics data. This has allowed the training of large Deep Learning (DL) models to predict epigenetic readouts, such as protein binding or histone modifications, from genome sequences. However, large dataset sizes come at a price of data consistency, often aggregating results from a large number of studies, conducted under varying experimental conditions. While data from large-scale consortia are useful as they allow studying the effects of different biological conditions, they can also contain unwanted biases from confounding experimental factors. Here, we introduce Metadata-guided Feature Disentanglement (MFD) - an approach that allows disentangling biologically relevant features from potential technical biases. MFD incorporates target metadata into model training, by conditioning weights of the model output layer on different experimental factors. It then separates the factors into disjoint groups and enforces independence of the corresponding feature subspaces with an adversarially learned penalty. We show that the metadata-driven disentanglement approach allows for better model introspection, by connecting latent features to experimental factors, without compromising, or even improving performance in downstream tasks, such as enhancer prediction, or genetic variant discovery. The code for our implemementation is available at https://github.com/HealthML/MFD
- Y. Luo et al. New developments on the Encyclopedia of DNA Elements (ENCODE) data portal. Nucleic Acids Research, 48(D1):D882–D889, 11 2019.
- J. Zhou and O. Troyanskaya. Predicting effects of noncoding variants with deep learning–based sequence model. Nature Methods, 12(10):931–934, 2015.
- D. R. Kelley et al. Basset: learning the regulatory code of the accessible genome with deep convolutional neural networks. Genome Research, 26(7):990–999, Jul 2016. Epub 2016 May 3.
- Ž. Avsec et al. Base-resolution models of transcription-factor binding reveal soft motif syntax. Nature Genetics, 53(3):354–366, 2021.
- G. Novakovsky et al. Obtaining genetics insights from deep learning via explainable artificial intelligence. Nature Reviews Genetics, 24(2):125–137, 2023.
- S. Hooker et al. A benchmark for interpretability methods in deep neural networks, 2019. In NeurIPS 2019.
- A. Majdandzic et al. Correcting gradient-based interpretations of deep neural networks for genomics. Genome Biology, 24:109, 2023.
- Capturing heterogeneity in gene expression studies by surrogate variable analysis. PLoS genetics, 3(9):e161, 2007.
- J. Wang et al. Correcting nucleotide-specific biases in high-throughput sequencing data. BMC Bioinformatics, 18:357, 2017.
- M. Ghanbari and U. Ohler. Deep neural networks for interpreting rna-binding protein target preferences. Genome research, 30(2):214–226, 2020.
- Y. Bengio et al. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828, 2013.
- F. Locatello et al. Weakly-supervised disentanglement without compromises. In International Conference on Machine Learning, pp. 6348–6359. PMLR, 2020.
- I. Khemakhem et al. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pp. 2207–2217. PMLR, 2020.
- Y. Ganin et al. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096–2030, 2016.
- Q. Zhao et al. Training confounder-free deep learning models for medical applications. Nature communications, 11(1):1–9, 2020.
- E. Adeli et al. Representation learning with statistical independence to mitigate bias. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2513–2523, 2021.
- D. He and L. Xie. Code-ae: a coherent de-confounding autoencoder for predicting patient-specific drug response from cell line transcriptomics. arXiv preprint arXiv:2102.00538, 2021.
- P. Chormai et al. Disentangled explanations of neural network predictions by finding relevant subspaces. ArXiv, abs/2212.14855, 2022.
- J. Schreiber et al. Avocado: a multi-scale deep tensor factorization method learns a latent representation of the human epigenome. Genome biology, 21(1):1–18, 2020.
- S. Yang et al. Deepnoise: Signal and noise disentanglement based on classifying fluorescent microscopy images via deep learning. Genomics, Proteomics & Bioinformatics, 20(5):989–1001, 2022. AI in Omics.
- M. Lotfollahi et al. Biologically informed deep learning to query gene programs in single-cell atlases. Nature Cell Biology, 25:337–350, 2023.
- Ž. Avsec et al. Effective gene expression prediction from sequence by integrating long-range interactions. Nature methods, 18(10):1196–1203, 2021.
- D. Ha et al. Hypernetworks. arXiv preprint arXiv:1609.09106, 2016.
- M. Sundararajan et al. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319–3328. PMLR, 2017.
- N. Kokhlikyan et al. Captum: A unified and generic model interpretability library for pytorch, 2020.
- M. Bentsen et al. Atac-seq footprinting unravels kinetics of transcription factor binding during zygotic genome activation. Nature Communications, 11(1):4267, 2020.
- M. Dalby et al. Fantom5 transcribed enhancers in hg38, April 2017.
- A. Visel et al. Vista enhancer browser—a database of tissue-specific human enhancers. Nucleic acids research, 35(suppl_1):D88–D92, 2007.
- J. Lonsdale et al. The genotype-tissue expression (gtex) project. Nature genetics, 45(6):580–585, 2013.
- G. Benegas et al. Gpn-msa: an alignment-based dna language model for genome-wide variant effect prediction. bioRxiv, 2023.
- D. R. Kelley et al. Sequential regulatory activity prediction across chromosomes with convolutional neural networks. Genome research, 28(5):739–750, 2018.
- S. J. Reddi et al. On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237, 2019.
- D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- A. Paszke et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
- W. Falcon and The PyTorch Lightning team. PyTorch Lightning, March 2019.
- M. I. Belghazi et al. Mine: mutual information neural estimation. arXiv preprint arXiv:1801.04062, 2018.
- H. M. Amemiya et al. The encode blacklist: Identification of problematic regions of the genome. Scientific reports, 9(1):9354, 2019. 27 Jun. 2019.
- A. Visel et al. Vista enhancer browser–a database of tissue-specific human enhancers. Nucleic acids research, 35(Database issue):D88–D92, 2007.
- F. Pedregosa et al. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
- A. S. Hinrichs et al. The ucsc genome browser database: update 2006. Nucleic acids research, 34(suppl_1):D590–D598, 2006.
- N. Kerimov et al. eqtl catalogue 2023: New datasets, x chromosome qtls, and improved detection and visualisation of transcript-level qtls. PLoS Genetics, 19(9):e1010932, 2023.
- S. Chen et al. A genomic mutational constraint map using variation in 76,156 human genomes. Nature, pp. 1–11, 2023.
- J. E. Moore et al. Expanded encyclopaedias of dna elements in the human and mouse genomes. Nature, 583(7818):699–710, 2020.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.