Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data (1612.08544v2)

Abstract: Data science models, although successful in a number of commercial domains, have had limited applicability in scientific problems involving complex physical phenomena. Theory-guided data science (TGDS) is an emerging paradigm that aims to leverage the wealth of scientific knowledge for improving the effectiveness of data science models in enabling scientific discovery. The overarching vision of TGDS is to introduce scientific consistency as an essential component for learning generalizable models. Further, by producing scientifically interpretable models, TGDS aims to advance our scientific understanding by discovering novel domain insights. Indeed, the paradigm of TGDS has started to gain prominence in a number of scientific disciplines such as turbulence modeling, material discovery, quantum chemistry, bio-medical science, bio-marker discovery, climate science, and hydrology. In this paper, we formally conceptualize the paradigm of TGDS and present a taxonomy of research themes in TGDS. We describe several approaches for integrating domain knowledge in different research themes using illustrative examples from different disciplines. We also highlight some of the promising avenues of novel research for realizing the full potential of theory-guided data science.

• The paper introduces TGDS, integrating scientific theories as constraints in data models to enhance interpretability and generalizability.
• It presents a detailed taxonomy covering theory-guided design, learning, refinement, hybrid modeling, and augmentation techniques.
• TGDS shows practical benefits in fields like climate science and hydrology by addressing challenges of non-stationarity and limited data samples.

Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data

The paper "Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data" by Karpatne et al. presents an emerging paradigm called Theory-guided Data Science (TGDS) that aims to integrate scientific knowledge with data science models to advance scientific discovery. The authors acknowledge the limitations of traditional data science models in scientific applications, characterized by complex physical phenomena and limited training samples, and propose TGDS as a solution to these challenges.

Core Concepts and Motivation

The authors argue that while data science models have revolutionized many commercial domains, their application to scientific problems has been less effective due to under-constrained problems and the non-stationary nature of physical variables. They highlight that scientific discovery often requires not just actionable models but also scientifically interpretable results. Traditional "black-box" data science methods may achieve high accuracy but often lack the interpretability required for scientific advancements.

TGDS proposes to bridge this gap by incorporating scientific theories as constraints, priors, or regularizers in data science models. This integration ensures that the models not only learn from the data but also adhere to known scientific principles, leading to better generalizability and interpretability.

Taxonomy and Research Themes

The paper provides a comprehensive taxonomy of TGDS methods, categorized into five main themes:

1. Theory-guided Design of Data Science Models: This involves designing model architectures and specifying response functions based on scientific theories. For instance, biologically plausible neural network architectures can better capture domain-specific phenomena, enhancing both accuracy and interpretability.
2. Theory-guided Learning of Data Science Models: Several approaches are discussed here, including:
   • Theory-guided Initialization: Using physically meaningful parameters for initializing models.
   • Theory-guided Probabilistic Models: Incorporating priors and constraints based on scientific knowledge into probabilistic models.
   • Theory-guided Constrained Optimization: Enforcing domain-specific constraints during the optimization process.
   • Theory-guided Regularization: Utilizing domain-specific regularization techniques to prevent overfitting and ensure physical consistency.
3. Theory-guided Refinement of Data Science Outputs: Model outputs are refined using explicit or implicit scientific knowledge to ensure physical consistency. This can involve post-processing steps that utilize scientific principles to correct model outputs.
4. Learning Hybrid Models of Theory and Data Science: This involves creating models that combine theory-based components with data science methods, enabling the capture of complex dependencies and improving model performance and interpretability.
5. Augmenting Theory-based Models using Data Science: Data science methods are employed to enhance theory-based models, either by assimilating data to improve model states or by calibrating model parameters to better reflect physical systems.

Practical Implications and Future Directions

The implications of TGDS are manifold. Practical applications span multiple scientific disciplines including climate science, hydrology, material discovery, and biomedical sciences. For example, in hydrological modeling, theory-guided data science can significantly improve the prediction of subsurface flow by integrating domain-specific equations with data-driven models.

The theoretical implications are equally profound. TGDS challenges the traditional dichotomy of theory-based and data-driven approaches, promoting a more integrated framework for scientific discovery. This paradigm encourages the development of models that not only perform well on known data but also generalize to unseen scenarios while providing insights into underlying physical processes.

Future developments in TGDS are likely to explore novel ways of integrating scientific knowledge at various stages of the model-building process, enhancing both the robustness and interpretability of scientific models. For instance, advancements in machine learning techniques such as deep learning and probabilistic graphical models can be leveraged to develop more complex and accurate TGDS frameworks.

Conclusion

In conclusion, the paper successfully conceptualizes TGDS as a novel approach to scientific discovery, emphasizing the integration of data science with domain-specific scientific knowledge. This paradigm shifts the focus from purely data-driven models to those that are also scientifically consistent and interpretable. By providing a robust taxonomy and illustrating various research themes with concrete examples, the paper lays a solid foundation for future research in this promising field. The integration of TGDS into scientific disciplines holds the potential to significantly advance our understanding and application of complex scientific phenomena.