Feasibility of machine learning-based rice yield prediction in India at the district level using climate reanalysis data (2403.07967v1)

Abstract: Yield forecasting, the science of predicting agricultural productivity before the crop harvest occurs, helps a wide range of stakeholders make better decisions around agricultural planning. This study aims to investigate whether machine learning-based yield prediction models can capably predict Kharif season rice yields at the district level in India several months before the rice harvest takes place. The methodology involved training 19 machine learning models such as CatBoost, LightGBM, Orthogonal Matching Pursuit, and Extremely Randomized Trees on 20 years of climate, satellite, and rice yield data across 247 of Indian rice-producing districts. In addition to model-building, a dynamic dashboard was built understand how the reliability of rice yield predictions varies across districts. The results of the proof-of-concept machine learning pipeline demonstrated that rice yields can be predicted with a reasonable degree of accuracy, with out-of-sample R2, MAE, and MAPE performance of up to 0.82, 0.29, and 0.16 respectively. These results outperformed test set performance reported in related literature on rice yield modeling in other contexts and countries. In addition, SHAP value analysis was conducted to infer both the importance and directional impact of the climate and remote sensing variables included in the model. Important features driving rice yields included temperature, soil water volume, and leaf area index. In particular, higher temperatures in August correlate with increased rice yields, particularly when the leaf area index in August is also high. Building on the results, a proof-of-concept dashboard was developed to allow users to easily explore which districts may experience a rise or fall in yield relative to the previous year.

• The paper presents a robust ML framework combining 19 models with climate and satellite data to achieve up to 0.82 R² in rice yield predictions.
• It employs a meticulous data integration process including fuzzy matching of 20-year climate and yield data from 247 districts to enhance accuracy.
• An interactive dashboard visualizes the yield forecasts, enabling actionable insights for improved irrigation, fertilizer, and harvest planning.

Feasibility of Machine Learning-Based Rice Yield Prediction in India at the District Level

The paper "Feasibility of machine learning-based rice yield prediction in India at the district level using climate reanalysis data" by Djavan De Clercq and Adam Mahdi represents a significant contribution to the field of agricultural analytics, with a specific focus on rice yield prediction. This paper evaluates the potential of machine learning models to forecast district-level rice yields several months before harvest, leveraging a combination of climate reanalysis and satellite data.

Methodology and Data Integration

The authors employed a comprehensive methodology involving 19 machine learning models, including CatBoost, LightGBM, Orthogonal Matching Pursuit, and Extremely Randomized Trees. These models were trained on a diverse dataset spanning 20 years, covering climate, satellite, and rice yield data from 247 rice-producing districts in India. The data sources include ERA5 from ECMWF for climate variables and MODIS from NASA for vegetation indices.

This paper is noteworthy for its spatially matched dataset, achieved through fuzzy matching algorithms to align district-level yield data with geographic boundaries accurately. This meticulous preprocessing step ensures the reliability of the training data, which is crucial for effective model performance.

Model Performance and Results

The models exhibited robust performance with out-of-sample R², MAE, and MAPE values reaching up to 0.82, 0.29, and 0.16, respectively. These results surpass those reported in related literature on rice yield prediction in other regions and using other methods. For instance, previous studies on rice yield models in China and Vietnam reported significantly lower R² and higher error metrics, highlighting the effectiveness of the approach used in this paper.

SHAP value analysis provided insights into the importance and directional impact of various features, identifying temperature, soil water content, and leaf area index as key drivers of rice yields. The paper found that higher temperatures in August, especially when coupled with a high leaf area index, correlate with increased yields, a finding consistent with agronomic literature.

Practical Implications and Interactive Dashboard

The paper goes beyond theoretical model development by creating an interactive dashboard to visualize yield predictions and regional performance variations. This tool allows stakeholders to explore and interpret model outputs, making the research highly actionable. For example, the dashboard highlighted variation in model performance across districts, with absolute percentage errors ranging from 7.1% in Uttarakhand to 14.7% in Uttar Pradesh.

Implications and Future Directions

The implications of this research are multifaceted. Practically, the model's ability to provide timely yield forecasts can significantly enhance decision-making in irrigation scheduling, fertilizer application, and harvest planning, thus optimizing resource use and crop output. Theoretically, the paper sets a new benchmark for district-level yield prediction models in India, offering a reference point for future research.

Future research could explore incorporating more agronomically relevant climatological variables, leveraging LLMs to communicate yield prediction results effectively, and modeling the dissemination of these predictions within the agricultural sector. Additionally, integrating optimization approaches to aid allocation decisions could further enhance the practical utility of these models.

Conclusion

In summary, the paper presents a robust, scalable approach to rice yield prediction at the district level in India using machine learning, supported by a comprehensive dataset and validated through rigorous model evaluation. The development of an interactive dashboard underscores the practical applicability of the research, providing a valuable tool for agricultural stakeholders. This work not only advances the field of agricultural yield prediction but also lays the groundwork for future innovations aimed at enhancing food security and agricultural resilience.