- The paper presents personalized guidelines for the design, implementation, and evaluation of anti-phishing interventions to address limitations of one-size-fits-all approaches.
- A multi-vocal literature review identified 22 dominant factors influencing effectiveness and synthesized 41 specific guidelines from academic and grey literature.
- The guidelines are personalized for different practitioners, intervention types, and stages, offering practical recommendations like displaying underlying URLs or using visual examples.
The paper addresses the limitations of current anti-phishing interventions, which often employ a one-size-fits-all approach, leading to usability issues and poor implementation. The authors present personalized guidelines for developers and practitioners involved in the design, implementation, and evaluation of anti-phishing interventions.
The paper identifies 22 dominant factors affecting the effectiveness of anti-phishing interventions at the individual, technical, and organizational levels. Based on these factors, the authors propose 41 guidelines to improve the outcomes of these interventions, synthesized from an analysis of 53 academic studies and 16 items of grey literature.
The authors conduct a Multi-vocal Literature Review (MLR), adhering to the protocols outlined by Kitchenham and Charters, and Garousi et al. For academic studies, the Scopus database was used and the search included terms like "aware*", "interven*", "nudge*", "warn*", "protect*", "security indicators", or "alert*" along with "phish*". Studies with a CORE rank of A*, A, and B were included, with exceptions for CORE B papers published before 2012. Google was used to collect grey literature, using the search terms "education", "training", and "awareness" with "phish*". The quality of grey literature studies was assessed based on the publication's authority, methodology, references, publication date, novelty, and outlet type.
Thematic analysis, following Braun and Clarke's methodology and using the tool Nvivo, was employed to identify challenges and recommendations. The challenges were classified into design, implementation, and evaluation phases of phishing education, training, and awareness interventions.
To provide personalized guidelines, the authors categorized practitioners into four major groups based on their functional roles: designer/developer, information security team, cyber security experts, and C-suite employees. The responsibilities of each group were investigated based on recommendations from the MLR.
The interventions were classified into three types: education, training, or awareness, based on their goal, presentation, and delivery method. The dominant factors were identified from the challenges and recommendations related to anti-phishing interventions.
The paper identifies 22 dominant factors impacting anti-phishing interventions:
- Individual Human Factors: These include age, complacency, confusion, curiosity, distraction, educational qualification, knowledge decay, ignorance, lack of communication, lack of motivation, lack of trust, optimism bias, perceived vulnerability and severity, pressure, and fatigue.
- Technical Factors: The technical factors are device type, gamer type, lack of knowledge, and lack of resources.
- Organizational Factors: The organizational factors are organizational position, social influence, and norms.
The paper presents 41 guidelines for the design, implementation, and evaluation of anti-phishing interventions. These guidelines are personalized to different practitioner groups, intervention stages, intervention types, challenges, and dominant factors. Examples include removing deceptive user interface elements, displaying the underlying URL of suspicious links, using visual examples, minimizing intervention frequency, designing distinct phishing warnings, and providing clear choices for users.
The paper acknowledges potential threats to validity, such as the comprehensiveness of the guidelines and the generalizability of the findings. To mitigate researcher bias, a well-defined research protocol was followed, and activities were conducted in accordance with established guidelines.