Research from Gartner

The Impact of Generative AI on Security Behavior and Culture Programs

GenAI and LLM have captured the interest of organizations aiming to enhance their security training programs. SRM leaders should understand how this technology can be used to improve security training programs, and more importantly, the organization’s overall security culture.

Overview

Impacts

• Threat actors exploit generative AI (GenAI), enabling them to rapidly and continuously adapt their attack tactics.
• GenAI also makes it far easier for employees across the organization to undertake technology work. This is a double-edged sword — it can enhance operational outcomes but also introduce new vectors for realizing operational cybersecurity risks.
• Traditional (or legacy) cybersecurity skills training falls short in addressing modern GenAI risks.

Recommendations

SRM leaders exploring GenAI to improve cybersecurity training and overall corporate security culture should:

• Focus on protecting the organization against evolving attack methods by incorporating AI capabilities.
• Expand their focus beyond static computer-based training and phishing campaigns by embracing GenAI to enable behavioral changes.
• Augment their existing security behavior and culture program with GenAI tools to improve the scale and understandability of security guidance and expectations.

Strategic Planning Assumption

By 2026, enterprises combining GenAI with an integrated platforms-based architecture in security behavior and culture programs will experience 40% fewer employee-driven cybersecurity incidents.

Introduction

The integration of AI into security behavior and culture program capabilities is not a recent cybersecurity trend. For the past decade, SRM leaders and vendors have incorporated AI elements, such as threat and anomaly detection, into their programs and platforms, which eventually feed into personalized learning. What is new, however, is the rapid and pervasive emergence of GenAI.

Gartner defines GenAI as technologies that “can generate new, derived versions of content, strategies, designs and methods by learning from large repositories of original source content.”

GenAI and its capabilities have enhanced existing tactics and processes that aim to promote secure behavior in employees. Examples of potential enhancements include AI assist chatbots, interactive simulations and adaptive, hyperpersonalized social engineering threat simulation capabilities. The current maturity of these enhancements vary from nascent to established. Ultimately, these advancements address the need for more adaptive and dynamic security engagement with counterparts in IT and the business.

Moreover, GenAI empowers SRM leaders to take greater ownership of their security training programs, reducing reliance on vendors to build campaigns and training. Vendors will still have an important role to play. Organizations developing internal GenAI tools do so to improve productivity, which is crucial in the cybersecurity function where resources are often limited. Those responsible for security training programs should expect a significant increase in impact and scale. This increase comes with potential challenges and ethical concerns, such as data privacy.

Many leaders are still in the experimentation phase, exploring the best ways to leverage GenAI within their security training programs. This research aims to delve into the impact of GenAI on these capabilities, specifically addressing tools and content.

Impacts and Recommendations

Threat Actors Exploit GenAI, Enabling Them to Rapidly and Continuously Adapt Their Attack Tactics

The rapid evolution of GenAI has provided threat actors with sophisticated tools to enhance and diversify their attack methodologies. This paradigm shift in cyberthreat landscapes necessitates a comprehensive understanding of the capabilities and implications of GenAI in cybersecurity.

Additionally, it is important to note that threat actors are using GenAI for other malicious actions. While not an exhaustive list, Figure 1 depicts these other actions.

Figure 1. Be Prepared for How Threat Actors Are Using Generative AI

We explore two main examples of social engineering.

Enhanced Phishing Attacks

One of the primary areas where GenAI has made a significant impact is phishing. Traditional phishing attacks often relied on rudimentary techniques that could be easily detected by vigilant users and advanced email filters. However, with the advent of GenAI, threat actors can now craft highly convincing and personalized phishing emails. GenAI, including LLMs that generate synthetic text, visual deepfakes of faces, and audio deepfakes of speech, enables adversaries to scale targeted phishing campaigns. LLMs can interact with users via text conversations and be programmed with a meta prompt to phish for sensitive information.¹ According to a report, global phishing attacks increased 60% in 2023, with a notable rise in the sophistication of these attacks attributed to AI-generated content.²

Deepfake Technology

The proliferation of deepfake technology, powered by GenAI, poses a significant threat to cybersecurity. Deepfakes can be used to create realistic but fraudulent images, videos and audio recordings. This technology has been employed in social engineering attacks to impersonate executives and other high-profile individuals, thereby facilitating enhanced victim targeting and other forms of fraud. The FBI, Cybersecurity and Infrastructure Security Agency (CISA) and National Security Agency (NSA) in the U.S. have issued warnings about the increasing use of deepfakes in cybercrime, emphasizing the need for advanced detection mechanisms.³

Technological Advancements From GenAI Improve the Existing Capabilities of Traditional Training Programs

GenAI is not something that is only used by attackers. It can be used to improve the security knowledge of employees in your organization. Traditional security awareness programs are often ad hoc (e.g., part-time effort, informal reporting and few, if any, metrics), with lean staffing and moderate spending.⁴ The prevalence of ad hoc training programs reflects the low staffing and funding levels training programs receive. While severely lean staffing will persist for the foreseeable future, SRM leaders can expect to see GenAI improve the following existing capabilities in security training programs:

Advanced Social Engineering Campaigns

Organizations can create targeted training programs for each employee, based on their individual weaknesses and areas of improvement identified through data analysis. Data to consider includes:

• Social engineering campaign results: Analyzing employee performance during simulated phishing attacks can highlight specific vulnerabilities and areas needing improvement.
• User behavior analytics: Monitoring how employees interact with emails and links can reveal patterns that indicate susceptibility to campaign attempts.
• Incident reports: Reviewing past incidents of successful attacks within the organization can help identify common traits among affected employees.
• Surveys and feedback: Collecting feedback from employees about their experiences and knowledge can provide insights into their security behavior and culture aptitude.
• Campaign history: Evaluating previous training sessions and their effectiveness can guide the development of more targeted content.
• Demographic data: Understanding the roles and responsibilities of employees can help tailor campaigns to specific job functions that may be more vulnerable.

This approach ensures that employees receive the most relevant and effective training to enhance their resistance against phishing attacks.

Advanced social engineering campaigns utilize GenAI algorithms to automatically deliver microtrainings to employees in real-time, exactly when they are most susceptible to falling for a phishing attempt. These microtrainings are designed to engage employees and promote safe behaviors, such as recognizing phishing emails, avoiding suspicious links and reporting potential threats.

The use of GenAI in advanced social engineering campaigns not only improves the effectiveness of training programs but also saves time and resources for organizations. Instead of conducting generic and repetitive training sessions, GenAI-driven campaigns deliver personalized and dynamic content that adapts to the evolving threat landscape.

Furthermore, GenAI can continuously monitor and evaluate employee performance, providing ongoing feedback and recommendations for improvement. This feedback loop ensures that employees remain vigilant and proactive in their defense against phishing attacks.

Recommendation:

• Use GenAI to guide each employee on a unique path to improve their security behaviors. Advanced social engineering campaigns automatically deliver in-the-moment microtrainings to drive engagement and safe behaviors.

Hyperpersonalized Learning

Engagement levels are a key determinant of SBCP effectiveness. The more employees are motivated to engage with SBCP initiatives (training, communications etc.), the more likely they will be prepared to alter their behaviors and work practices — thereby helping to reduce employee-driven cybersecurity incidents. GenAI will enable SRM leaders to create hyperpersonalized learning material that speaks to each employee’s unique requirements. By analyzing individual learner’s strengths, weaknesses and learning styles, GenAI can create personalized training programs that cater to each employee’s unique needs. This ensures that learners receive the most relevant and effective training materials, increasing their engagement and retention of security training behaviors.

One innovative approach GenAI employs to achieve personalized learning is through the enhancement of role-based training, leveraging user behavior analytics data sourced from existing security tools such as security information and event management (SIEM) systems and web application firewalls (WAF). By integrating this data with a commercial or internally developed LLM, GenAI can generate hyperpersonalized training material tailored to address an individual’s specific deficiencies. This methodology ensures that employees receive cybersecurity knowledge and skills that are directly pertinent to their job functions.

By analyzing the specific challenges and risks associated with different roles within the organization, GenAI tools can craft training content that speaks directly to the recipient’s unique context. This targeted approach not only increases engagement but also enhances the likelihood of behavioral adaptation, as employees perceive the training to be more relevant and applicable to their daily responsibilities.

For instance, consider a scenario in which a financial analyst within an organization frequently accesses sensitive financial data. User behavior analytics might reveal that this individual has a pattern of neglecting multifactor authentication (MFA) protocols. By integrating this behavioral data with an LLM, GenAI can generate training modules that emphasize the importance of MFA, provide real-world examples of breaches resulting from MFA lapses, and offer step-by-step guidance on implementing MFA effectively. This personalized training material would be far more impactful than generic cybersecurity training, as it directly addresses the analyst’s specific vulnerabilities and the critical nature of their role.

Additionally, GenAI incorporates segmented training to account for the varying levels of cybersecurity knowledge and expertise among employees. GenAI can assess the proficiency of individuals and provide training materials appropriate for their skill level. This ensures that employees receive training that is neither too basic nor too advanced, optimizing their learning experience.

Whether it is for the entire organization or individuals who have previously undergone training, GenAI’s hyperfocused personalized learning approach ensures has two main benefits:

• Employees receive training that is tailored to their specific needs and maximizes their learning potential.
• Allows the user to interact with an unbiased curriculum, reducing the risk of employee backlash to training, HR friction, and less resource intensive material planning.

Recommendation:

• Continuously assess and update the training materials based on individual learner’s progress and feedback. By regularly evaluating the effectiveness of the training programs and incorporating learner feedback, GenAI can ensure that the personalized training remains relevant and impactful.

Metrics Collection and Reporting

Historically, training programs struggle to measure success. Measurable employee behavior change is the primary objective of the vast majority (84%) of training programs; yet, less than half (43%) of programs consistently measure employee behavior.4 GenAI can improve security training metrics and reporting in several ways:

Enhanced data analysis: GenAI can analyze vast amounts of security-related data, such as logs, incident reports and user behavior patterns to identify trends, anomalies, and potential risks. By processing this data more efficiently than traditional methods, GenAI can provide deeper insights into security incidents and help identify areas that require attention.

Natural language processing: GenAI can leverage natural language processing to analyze and understand security-related conversations, emails, or other textual data. This capability enables it to identify potential security risks, detect phishing attempts and provide real-time guidance to users, thereby improving overall security training and reducing the likelihood of security incidents.

GenAI’s impact of specific cybersecurity training and cultures metrics to track include:

• Enhancing phishing detection to reduce susceptibility.
• Prioritizing and categorizing security service edge (SSE) and security information and event management (SIEM) alerts for more accurate insights.
• Analyzing web browsing behavior to decrease unsafe activity.
• Identifying patterns of deliberate security control avoidance.
• Monitoring and detecting unauthorized external storage device usage.
• Analyzing policies and configurations to reduce violations and misconfigurations.
• Assisting developers in identifying and remediating nonsecure code.
• Proactively identifying critical vulnerabilities in in-house applications.
• Providing recommendations to reduce security misconfigurations and rollback incidents.

Recommendation:

• Use cybersecurity outcome driven metrics to help inform and guide future engagements with targeted employees. Additionally, consult with the privacy office, legal and HR to conduct the necessary assessments to identify risks to employee privacy and implement appropriate mitigating measures.

GenAI Introduces New Possibilities for Generating Security Training Content

The integration of GenAI into traditional training programs introduces a range of new capabilities. These include processing large amounts of data quickly and efficiently, adapting and learning from new information, generating realistic content, and enhancing interactivity and engagement. As a result, GenAI significantly boosts the effectiveness and efficiency of traditional training programs, enabling them to better understand and respond to complex situations in real time. SRM leaders can expect to see GenAI create the following new capabilities:

GenAI Assist Chatbot for Cybersecurity

The primary use case for the GenAI Assist Chatbot is to enable cybersecurity teams to provide scalable guidance to application developers, business technologists and other internal or external stakeholders. This empowers them to make informed decisions regarding cybersecurity.

To effectively use the GenAI Assist Chatbot, it is important to have a well-defined security control environment and clear behavior expectations. This ensures that the guidance provided by the chatbot aligns with the organization’s security policies and requirements.

One of the key benefits of using AI in cybersecurity is the speed and scalability it offers. The GenAI Assist Chatbot can help lower the barrier to entry for cybersecurity training and implementation, making it more accessible to a wider audience within the organization.

Overall, the GenAI Assist Chatbot has the potential to improve the adoption and implementation of cybersecurity training. By targeting the internal cybersecurity team and providing scalable guidance, it can empower employees to make more informed decisions regarding cybersecurity.

Recommendation:

• GenAI Assist Chatbot should be directed toward the internal cybersecurity team, specifically Business Information Security Officers (BISOs), analysts, and governance, risk, and compliance (GRC) leads. These individuals are experts in the field and can effectively interpret documents within the security ecosystem.

Interactive Attack Simulations

GenAI opens up opportunities to create more immersive attack simulation platform capabilities for security operation teams, executive management teams and other senior leaders. These GenAI-enhanced platforms enable organizations to improve cyber readiness, test incident response processes and playbooks, measure and track capabilities, and cultivate a skilled, aligned and confident security team. By leveraging realistic simulations of various security threats and attacks, GenAI-enabled attack simulation capabilities empower learners to actively participate and experience the real-time consequences of their actions, fostering a hands-on learning approach that reinforces the importance of security behaviors.

One of the key requirements for optimizing the use of this capability is having well-defined incident response and crisis management playbooks based on foreseeable risk scenarios. This ensures that organizations have a structured framework in place to effectively respond to security incidents and align their simulation exercises accordingly.

GenAI-enabled immersive attack simulation capabilities should test incident response playbooks, allowing security leaders to continuously validate and refine their incident response strategies through realistic scenarios.

Recommendation:

• Leverage realistic simulations of various security threats and attacks. GenAI empowers learners to actively participate and experience the consequences of their actions, fostering a hands-on learning approach that reinforces the importance of security behaviors.

Figure 2 summarizes the key impacts of GenAI and the top recommendations for security and risk management leaders.

Figure 2. Impacts and Recommendations for SRM Leaders

Source: Gartner Research Note G00816736, Alex Michaels, Will Candrick, Richard Addiscott, Andrew Walls, Victoria Cason, 11 November 2024