Real-time Monitoring of Economic Shocks using Company Websites
The paper introduces an innovative methodology for real-time economic monitoring in the form of the Web-Based Affectedness Indicator (WAI), which leverages data from company websites to analyze the impacts of economic shocks on firms. This tool operates on a global scale and offers insights into firm-level responses to disruptions such as those caused by the COVID-19 pandemic. The authors propose WAI as a transformative instrument for policymakers, businesses, and researchers, enhancing the understanding of economic resilience and supporting adaptive policy-making.
Methodology and Implementation
WAI represents a significant improvement over traditional forms of data collection like surveys and administrative data. These conventional methods often suffer from time lags, high costs, and limited coverage. To overcome these limitations, the authors analyze over five million company websites using a combination of information extracted from CommonCrawl and classification by LLMs. Specifically focusing on the COVID-19 pandemic as a validation scenario, they demonstrate that WAI correlates strongly with pandemic containment measures and effectively predicts firm performance metrics.
The processing within WAI involves extracting relevant textual data from websites for mentions of COVID-19, which are subsequently evaluated for affectedness levels using LLMs. These affectedness scores are categorized into three severity levels: mildly, moderately, and severely affected. Validation against the Oxford COVID-19 Government Response Tracker (OxCGRT) stringency index reveals high correlation coefficients (typically above 0.8) between WAI aggregates and government policy measures at both state and country levels.
Validation and Results Analysis
The paper validates WAI by running it alongside alternative COVID-19 impact measures, examining its correlation with financial performance indicators. Specifically, the authors conduct panel regressions correlating WAI scores with quarterly sales growth and stock returns, using data from approximately 30,000 publicly listed firms. Results indicate that WAI scores are significantly predictive of economic performance, with moderate and severely affected firms experiencing pronounced sales growth reductions of around 12% to 15% during the analysis period.
The cross-country analysis suggests some variations in WAI's effectiveness, with higher efficacy observed in regions with robust internet infrastructure and significant English-speaking populations, such as the US, UK, and Canada. This reflects an underlying bias in LLM training data, emphasizing ongoing opportunities for multilingual improvements in this domain.
Implications and Future Directions
The development and successful implementation of WAI illustrate a novel approach to economic monitoring that provides granular, firm-level insights in real time. This tool is poised to aid policymakers and researchers by providing actionable data during various types of economic crises, not limited to public health but extending to supply chain disruptions, climate-related events, and financial crises.
The paper acknowledges the potential biases introduced by the LLMs' reliance on predominantly English-language data. However, as internet adoption continues to expand globally and as LLMs enhance their multilingual capabilities, the efficacy of WAI will likely improve. The framework set by WAI indicates promising developments for the field of economic monitoring, paving the way for more comprehensive and adaptable policy-making strategies in response to global disruptions.
This paper contributes significantly to the methodologies available for understanding firm-level impacts of worldwide disruptions, offering a scalable and adaptable tool for economic resilience and adaptive policy formulation.