ClimaEmpact: Domain-Aligned Small Language Models and Datasets for Extreme Weather Analytics

Abstract: Accurate assessments of extreme weather events are vital for research and policy, yet localized and granular data remain scarce in many parts of the world. This data gap limits our ability to analyze potential outcomes and implications of extreme weather events, hindering effective decision-making. LLMs can process vast amounts of unstructured text data, extract meaningful insights, and generate detailed assessments by synthesizing information from multiple sources. Furthermore, LLMs can seamlessly transfer their general language understanding to smaller models, enabling these models to retain key knowledge while being fine-tuned for specific tasks. In this paper, we propose Extreme Weather Reasoning-Aware Alignment (EWRA), a method that enhances small LLMs (SLMs) by incorporating structured reasoning paths derived from LLMs, and ExtremeWeatherNews, a large dataset of extreme weather event-related news articles. EWRA and ExtremeWeatherNews together form the overall framework, ClimaEmpact, that focuses on addressing three critical extreme-weather tasks: categorization of tangible vulnerabilities/impacts, topic labeling, and emotion analysis. By aligning SLMs with advanced reasoning strategies on ExtremeWeatherNews (and its derived dataset ExtremeAlign used specifically for SLM alignment), EWRA improves the SLMs' ability to generate well-grounded and domain-specific responses for extreme weather analytics. Our results show that the approach proposed guides SLMs to output domain-aligned responses, surpassing the performance of task-specific models and offering enhanced real-world applicability for extreme weather analytics.

ClimaEmpact: Domain-Aligned Small Language Models and Datasets for Extreme Weather Analytics

The increasing prevalence of extreme weather events poses significant risks to communities and environments, necessitating precise assessment and analysis to aid decision-making and policy formulation. Traditional global datasets often do not provide the localized or nuanced data needed for effective management of these events. This paper introduces the ClimaEmpact framework, which leverages Large Language Models (LLMs) and Small Language Models (SLMs) to enhance understanding and analysis of extreme weather events through advanced language processing and domain-specific alignment techniques.

Key Contributions

1. Extreme Weather Reasoning-Aware Alignment (EWRA): This novel method is designed to improve the performance of SLMs by incorporating the structured reasoning pathways derived from LLMs. The approach enables SLMs to synthesize complex information into coherent and domain-specific responses tailored specifically for extreme weather analytics.

2. ExtremeWeatherNews and ExtremeAlign Datasets: The paper presents ExtremeWeatherNews, a comprehensive dataset compiling news articles about extreme weather events, serving as a foundational resource for trained models. ExtremeAlign is a derivative dataset aimed at aligning SLMs for improved reasoning structures applied to extreme-weather-related tasks.

3. ClimaEmpact Framework: Comprising the EWRA methodology and the aforementioned datasets, the ClimaEmpact framework tackles three critical tasks in extreme weather analytics: categorization of tangible vulnerabilities and impacts, topic labeling, and emotion analysis. This alignment significantly enhances the SLM's ability to interpret and predict outcomes, outperforming existing task-specific models with higher accuracy and applicability.

Methodology

The study employs the advanced reasoning capabilities of LLMs to generate structured reasoning paths for three key tasks. By doing so, it fills the critical need for domain-specific reasoning in small models, which traditionally struggle with understanding fine-grained contextual information. The EWRA method uses a two-step fine-tuning process:
- First, SLMs internalize reasoning logic without relying solely on patterns in prompt inputs.
- Second, the SLMs utilize detailed task definitions for explicit fine-tuning, ensuring both coherent task-specific explanation and general language comprehension.

Results and Implications

Model evaluation shows that SLMs using the EWRA methodology achieve significant improvements:
- On the Vulnerability/Impact/Emergency assessment task, the EWRA approach resulted in a 5.2% improvement in Spearman Rank Correlation over ReasonExplicit-SFT when tested on the Qwen2.5-3B-Instruct model. This demonstrates a superior alignment with human-annotated reasoning patterns.
- Comparable or superior results are obtained in emotion analysis and topic/subtopic labeling tasks, underscoring the applicability of the method in various analytical contexts associated with extreme weather.

The implications of this research extend both practically and theoretically. Practically, it offers a more accurate and nuanced real-time analysis tool for extreme weather events, potentially improving emergency response strategies and risk assessment frameworks. Theoretically, it advances the development of domain-specific language models by showcasing how reasoning capabilities can be distilled and transferred to smaller models.

Future Directions

The future of AI-driven weather modeling lies in enhancing the computational efficacy and environmental accessibility of such models. The ClimaEmpact framework suggests that further integration of domain-specific data and reasoning can strengthen the utility of AI in climate sciences. Future work might focus on extending these models to other areas of climate analysis, digital humanities, or interdisciplinary applications, deepening their integration with global environmental monitoring systems.

In conclusion, this paper successfully demonstrates how advances in natural language processing can be specifically tailored to enhance extreme weather analytics, providing a critical tool in the arsenal against climate-related disasters.