Phishing in Organizations: Findings from a Large-Scale and Long-Term Study (2112.07498v1)

Abstract: In this paper, we present findings from a large-scale and long-term phishing experiment that we conducted in collaboration with a partner company. Our experiment ran for 15 months during which time more than 14,000 study participants (employees of the company) received different simulated phishing emails in their normal working context. We also deployed a reporting button to the company's email client which allowed the participants to report suspicious emails they received. We measured click rates for phishing emails, dangerous actions such as submitting credentials, and reported suspicious emails. The results of our experiment provide three types of contributions. First, some of our findings support previous literature with improved ecological validity. One example of such results is good effectiveness of warnings on emails. Second, some of our results contradict prior literature and common industry practices. Surprisingly, we find that embedded training during simulated phishing exercises, as commonly deployed in the industry today, does not make employees more resilient to phishing, but instead it can have unexpected side effects that can make employees even more susceptible to phishing. And third, we report new findings. In particular, we are the first to demonstrate that using the employees as a collective phishing detection mechanism is practical in large organizations. Our results show that such crowd-sourcing allows fast detection of new phishing campaigns, the operational load for the organization is acceptable, and the employees remain active over long periods of time.

• The paper confirms that age and computer skills significantly predict phishing vulnerability, with repeated warnings reducing malicious clicks.
• The paper reveals that embedded phishing training may inadvertently increase risk by fostering over-reliance on defenses.
• The paper introduces employee crowd-sourced detection as an effective, sustainable strategy for rapid phishing identification.

A Large-Scale and Long-Term Evaluation of Phishing Susceptibility and Countermeasures in an Organizational Context

In the field of cybersecurity, phishing remains a significant threat to organizations worldwide. The paper "Phishing in Organizations: Findings from a Large-Scale and Long-Term Study" presents a comprehensive investigation into the dynamics of phishing susceptibility and the efficacy of prevalent countermeasures within an organizational setting. Conducted over 15 months with more than 14,000 employees from a partner company, the paper provides critical insights into human factors associated with phishing vulnerability and evaluates the real-world effectiveness of phishing detection and prevention strategies.

Key Contributions and Findings

The paper's contributions are multifaceted, spanning confirmations of existing knowledge, contradiction of previously accepted norms, and novel insights into phishing mitigation strategies.

1. Confirmation with Enhanced Ecological Validity:
   • The paper reaffirms previous observations that age and computer skills are significant predictors of phishing susceptibility, with younger and older employees exhibiting higher vulnerability. Unlike prior studies with smaller or less diverse samples, this large-scale evaluation lends greater ecological validity to these findings.
   • Another confirmed aspect is that repeated warnings appended to potentially malicious emails are effective in reducing phishing link clicks and harmful interactions, supporting prior research outcomes.
2. Contradictory Findings:
   • Perhaps the most contentious finding is the ineffectiveness of embedded phishing training—a widely adopted industry practice intended to bolster employee resistance to phishing. Remarkably, the paper observed that such training, in its current form, potentially increases susceptibility due to unintended side effects, such as fostering a false sense of security or over-reliance on institutional defenses.
   • This finding challenges existing pedagogical approaches and emphasizes the need for a reevaluation of training methodologies utilized in enterprise environments.
3. New Insights:
   • The concept of utilizing employees as a collective phishing detection mechanism is empirically validated, demonstrating that crowd-sourced phishing detection is both efficient and sustainable over a prolonged period. Employees effectively reported phishing attempts with minimal operational overhead, providing rapid detection of new phishing campaigns.
   • The paper also identifies an intriguing aspect of phishing susceptibility concerning the type of computer usage: employees engaged in repetitive, specialized computer tasks were more prone to phishing than those with less frequent engagement.

Implications for Practice and Future Research

The implications of this paper are significant for both practice and future research within the sphere of cybersecurity. Practically, the insights necessitate a reassessment of prevalent training models in phishing defense strategies. Organizations might need to explore more engaging and perhaps mandatory forms of training that may yield better retention and practical application of anti-phishing techniques.

The paper also opens several avenues for further research. The results raise questions about the psychological impacts of phishing simulations and embedded training within realistic work contexts. Understanding these impacts and their influence on actual phishing susceptibility can guide the development of more effective countermeasures. Future work could explore alternative modes of training delivery and assess their efficacy in various organizational settings.

Moreover, the promising validation of a crowd-sourcing approach to phishing detection warrants further exploration. Investigations into optimizing feedback loops, report processing automation, and incentivization frameworks could potentially enhance the effectiveness and feasibility of this approach across different organizational scales.

Overall, this paper significantly contributes to the body of knowledge on organizational phishing prevention and provides a robust foundation for developing more effective countermeasures to tackle the persistent threat posed by phishing attacks. Through its empirical examination of diverse influencing factors and intervention strategies, it advances the discourse while challenging conventional practices in cybersecurity training and awareness programs.