Magnitude of external validity threat due to fintech context

Ascertain the magnitude of the external validity threat to generalizing the measured effectiveness of anti-phishing training from a U.S.-based fintech company to other industry sectors, specifically due to potential differences in baseline security awareness between financial services employees and employees in other industries.

Background

The paper was conducted within a U.S.-based fintech organization, and the authors note that financial services employees may have different baseline levels of security awareness compared to other sectors. This raises concerns about how well the findings—particularly the measured effectiveness (or lack thereof) of anti-phishing training—would transfer to different organizational contexts.

While the measured effect sizes in this paper were minimal, the authors explicitly state uncertainty about how large the generalizability threat is. Quantifying this external validity issue is important for determining whether conclusions about training effectiveness can be reliably applied across industries beyond financial services.