Glucose-ML: A collection of longitudinal diabetes datasets for development of robust AI solutions

Abstract: AI algorithms are a critical part of state-of-the-art digital health technology for diabetes management. Yet, access to large high-quality datasets is creating barriers that impede development of robust AI solutions. To accelerate development of transparent, reproducible, and robust AI solutions, we present Glucose-ML, a collection of 10 publicly available diabetes datasets, released within the last 7 years (i.e., 2018 - 2025). The Glucose-ML collection comprises over 300,000 days of continuous glucose monitor (CGM) data with a total of 38 million glucose samples collected from 2500+ people across 4 countries. Participants include persons living with type 1 diabetes, type 2 diabetes, prediabetes, and no diabetes. To support researchers and innovators with using this rich collection of diabetes datasets, we present a comparative analysis to guide algorithm developers with data selection. Additionally, we conduct a case study for the task of blood glucose prediction - one of the most common AI tasks within the field. Through this case study, we provide a benchmark for short-term blood glucose prediction across all 10 publicly available diabetes datasets within the Glucose-ML collection. We show that the same algorithm can have significantly different prediction results when developed/evaluated with different datasets. Findings from this study are then used to inform recommendations for developing robust AI solutions within the diabetes or broader health domain. We provide direct links to each longitudinal diabetes dataset in the Glucose-ML collection and openly provide our code.

• The paper introduces a novel aggregation of 10 longitudinal diabetes datasets, merging over 300K days of CGM data from 2500+ participants.
• It provides a comprehensive comparative analysis highlighting variations in data quality and population diversity to inform AI model selection.
• A case study demonstrates that baseline predictors for short-term blood glucose forecasting vary significantly with dataset choice, underscoring dataset impact.

Glucose-ML: A Collection of Longitudinal Diabetes Datasets for Development of Robust AI Solutions

Introduction

The paper presents Glucose-ML, a comprehensive collection of longitudinal datasets aimed at facilitating the development of robust AI solutions for diabetes management. Glucose-ML comprises 10 publicly available datasets, aggregating over 300,000 days of continuous glucose monitor (CGM) data and 38 million glucose samples from 2500+ participants across four countries. The emphasis is placed on supporting AI development by providing access to high-quality, diverse data that reflects various diabetes conditions including type 1 diabetes (T1D), type 2 diabetes (T2D), prediabetes, and no diabetes conditions.

Figure 1: Overview of data types included within individual datasets in the Glucose-ML collection.

Dataset Composition and Accessibility

The datasets in Glucose-ML are sourced from varied demographics, comprising individuals with different diabetes statuses. These datasets are publicly accessible via platforms like Vivli, Physionet, and others, with controlled access in some cases to maintain privacy standards. Importantly, the datasets adhere to FAIR principles ensuring they are findable, accessible, interoperable, and reusable, thereby promoting research reproducibility and transparency.

Comparative Analysis of Datasets

A detailed comparative analysis of these datasets highlights the strengths and limitations inherent in each dataset and helps guide AI practitioners in dataset selection. Key parameters like sample size, population diversity, glucose dynamics, and data quality sufficiency are critically evaluated. The analysis reveals significant intra- and inter-dataset variations, emphasizing the importance of selecting appropriate datasets for specific research goals.

Figure 2: Comparative analysis of sample size and population (A), longitudinal glucose duration (B), data quality and sufficiency (C), and glucose dynamics (D) across 10 public diabetes datasets.

Case Study: Blood Glucose Prediction

The paper provides a case study focusing on the task of short-term blood glucose prediction — one of the most prevalent AI applications in diabetes management. Using na\"{\i}ve baseline algorithms, such as the zero-order hold predictor and simple linear regression predictor, a benchmark for predicting blood glucose levels 30 minutes ahead is established. It is demonstrated that the choice of dataset considerably impacts prediction performance, thus reinforcing the necessity for careful dataset selection in AI-driven diabetes research.

Figure 3: Performance overview for two na\"{\i}ve baseline algorithms, predicting blood glucose 30 minutes ahead using 10 publicly available diabetes datasets.

Discussion

Glucose-ML serves as a significant resource for data-centric AI research in diabetes, addressing the barriers of data accessibility and quality. The research findings underscore the profound impact of data selection on algorithmic performance. While not exhaustive, the dataset collection is comprehensive, enhancing the field's capability to develop and validate robust AI models.

Future directions might include expanding the collection to incorporate more diverse real-world data and improving the representation of underrepresented demographics such as pediatric populations and specific ethnic groups susceptible to diabetes.

Conclusion

Glucose-ML fundamentally enhances the landscape for AI development in digital health, particularly diabetes management, by offering an extensive array of high-quality datasets. The paper emphasizes the critical nature of dataset characteristics in influencing the outcome of AI applications like blood glucose prediction, thus supplying researchers with the tools necessary for informed algorithmic development and evaluation.