Global disease monitoring and forecasting with Wikipedia (1405.3612v2)

Abstract: Infectious disease is a leading threat to public health, economic stability, and other key social structures. Efforts to mitigate these impacts depend on accurate and timely monitoring to measure the risk and progress of disease. Traditional, biologically-focused monitoring techniques are accurate but costly and slow; in response, new techniques based on social internet data such as social media and search queries are emerging. These efforts are promising, but important challenges in the areas of scientific peer review, breadth of diseases and countries, and forecasting hamper their operational usefulness. We examine a freely available, open data source for this use: access logs from the online encyclopedia Wikipedia. Using linear models, language as a proxy for location, and a systematic yet simple article selection procedure, we tested 14 location-disease combinations and demonstrate that these data feasibly support an approach that overcomes these challenges. Specifically, our proof-of-concept yields models with $r^2$ up to 0.92, forecasting value up to the 28 days tested, and several pairs of models similar enough to suggest that transferring models from one location to another without re-training is feasible. Based on these preliminary results, we close with a research agenda designed to overcome these challenges and produce a disease monitoring and forecasting system that is significantly more effective, robust, and globally comprehensive than the current state of the art.

• The paper introduces a method using Wikipedia access logs with linear models to monitor and forecast infectious diseases, showing promising predictive capabilities with r ^2 values up to 0.92 for influenza.
• The approach addresses key challenges of internet-based surveillance by demonstrating openness, breadth across 14 disease-location contexts, transferability potential, and forecasting utility up to 28 days.
• This research suggests Wikipedia can supplement traditional surveillance, particularly where data is scarce or delayed, and proposes further work to refine methods and explore non-linear models and data influences.

Summary of "Global Disease Monitoring and Forecasting with Wikipedia"

The paper authored by Nicholas Generous and colleagues introduces an innovative approach to infectious disease monitoring and forecasting using statistical models based on access logs from Wikipedia. Traditional disease surveillance methodologies, which are inherently accurate, suffer from significant cost and time delays. In contrast, the internet provides large-scale, real-time social data streams that can potentially supplement traditional methods. Among these, Wikipedia’s freely available access logs offer a unique combination of openness and broad applicability.

The research investigates 14 combinations of diseases and countries employing a linear modeling approach to explore Wikipedia access logs as estimators of disease incidence. Results exhibit promising predictive capabilities with $r^2$ values up to 0.92 for influenza in various contexts, demonstrating that the models could provide effective nowcasting and short-term forecasting (up to 28 days). Importantly, this paper displays favorable conditions for transferring models across different disease-location contexts without necessitating retraining, leveraging the inter-language links in Wikipedia as a means for translation.

Generous et al. address four distinct challenges of internet-based disease monitoring:

1. Openness: Wikipedia access logs are open and accessible, supporting third-party review, replication, and improvement.
2. Breadth: The approach has demonstrated success across 14 diverse disease-location contexts, indicating widespread applicability.
3. Transferability: Models exhibit translatability potential between different geographic and linguistic contexts.
4. Forecasting: Demonstrated utility in predicting future incidence within the tested 28-day time horizon.

The research outlines multiple areas for further exploration. These include refining article selection processes, improving geolocation granularity, employing sophisticated non-linear models, understanding the influences of media, and addressing issues involving Wikipedia’s data stability and biases. The current findings imply that Wikipedia can serve as several real-world indicators, specifically when official health data are lacking or delayed.

Practically, this research can significantly enhance global health surveillance operations by providing a rapid assessment of disease incidence, particularly in regions with limited traditional surveillance. Theoretically, the paper expands the landscape of utilizing non-traditional data sources in epidemiology, offering a robust agenda for continued development of open and complementary disease monitoring systems.

In conclusion, Generous et al.'s research substantiates Wikipedia’s potential as an effective open data source for disease surveillance, presenting a valuable alternative to existing methodologies hindered by availability and access issues. This work importantly sketches an agenda that, if advanced, might establish a more comprehensive and robust disease monitoring system on a global scale. The findings warrant engagement from the scientific and public health communities to corroborate and extend these promising preliminary results.