What CISOs need to know about AI-augmented attacks
AI-augmented attacks remain one of the most pervasive cybersecurity threats. While AI is not creating entirely new forms of attack, it’s lowering the barrier to entry and enabling threat actors to scale familiar tactics with greater speed, reach and sophistication. “The broad availability of large language models (LLMs) primarily helps unsophisticated attackers scale their operations,” says Jeremy D’Hoinne, Distinguished Vice President Analyst at Gartner.
The most frequent of these attacks are social engineering and deepfake identity impersonation. According to a Gartner survey, 41% of organizations have dealt with an attack involving a deepfake and social engineering on an audio call to an employee. Yet despite the prevalence of these kinds of threats, many executives remain focused on sensational scenarios involving autonomous attacks and highly advanced AI-powered malware. Instead, CISOs must help their organization navigate the hype and focus on changes in the initial breach vectors and increased automation of postbreach activities.
You might also like this webinar: Defending Against Rogue AI: What the OpenAI/Hugging Face Incident Means for Cybersecurity Leaders
While social engineering and deepfake attacks remain the most visible manifestations of AI-related risk, the broader impact of AI on cyberthreats is still unfolding. Organizations should not only address the threats already occurring at scale, but also evaluate how AI may reshape attacker capabilities, defensive requirements and cybersecurity readiness in the years ahead.
Much of the concern around AI-driven cyberattacks centers on the prospect of fully automated attack chains. While that risk is evolving, successful large-scale attacks still depend on several factors, including finding an exploitable entry point, automating activities at scale and avoiding detection. AI can support parts of this process, but today’s tools still have a learning curve, requiring attackers to gain new skills and often generate activity that can be detected by security teams. For now, fully autonomous attacks remain more difficult to execute than headlines often suggest. Still, this forces organizations to rethink how they will handle incident responses against LLM-orchestrated postbreach activities.
AI-augmented attacks now go beyond impersonating employees to include biometric fraud and disinformation campaigns, giving organizations more ways to be targeted. While deepfake detection tools can help, the best defense is often stronger verification processes and additional checks for high-risk requests and transactions. Attackers are also using generative AI to help them work faster and more effectively, much like businesses use it to improve productivity.
Despite growing interest in AI-powered malware, most observed use cases remain experimental. Threat actors are using AI coding assistants to accelerate malware creation, convert existing malware into new programming languages and test ways to reduce detection.
Analysis of discovered malware has revealed attempts to interact with commercial AI models through stolen credentials, as well as efforts to exploit AI tools installed on endpoints. However, many of these examples show limited sophistication, and there is little evidence that AI has enabled entirely new malware capabilities. Even so, early signs of investment in these techniques suggest cybersecurity leaders should monitor the space closely as AI-enabled malware continues to evolve.
Recent advances in LLM-driven vulnerability discovery and automated penetration testing are creating new opportunities for attackers. The bigger challenge is not necessarily speed, as threat actors already exploit vulnerabilities quickly, but scale — more unsophisticated attackers are gaining access to automation that was previously the exclusive domain of advanced threat actors. As AI helps identify more potential entry points, organizations may face a growing volume of attack attempts against public-facing systems. Early examples of automated attacks have emerged, but so far their impact has remained relatively limited.
The most immediate effect of AI-driven automation may be in postbreach activities rather than initial compromise. New offensive security tools are making it easier to automate tasks that previously required significant manual effort, allowing attackers to operate more efficiently once inside an environment. As these capabilities mature, organizations should expect continued experimentation with increasingly automated attack workflows. This makes incident detection, response and visibility across systems critical to limiting the potential impact of an attack.
AI-augmented attacks are cyberattacks that use AI tools to make existing tactics more effective, scalable and convincing. Rather than creating entirely new attack methods, attackers use AI to enhance phishing, social engineering, impersonation and malware development.
Social engineering and deepfake identity impersonation remain the most common AI-augmented attacks. Attackers frequently use AI-generated text, voice and video to deceive employees and gain access to sensitive information or systems.
AI-powered malware is an emerging concern, but most examples remain experimental. Threat actors are using AI to assist with malware development and automation, but there is little evidence that AI has created entirely new malware capabilities.
While attack automation is advancing, successful large-scale attacks still require exploitable vulnerabilities, automation and the ability to evade detection. For most organizations, this redefines what “good-enough” defense is, forcing everyone to aim at shorter patching SLAs and putting pressure on incident response at scale.
Attend a Conference
Experience IT Security and Risk Management conferences
With exclusive insights from Gartner analysts on the latest trends, sessions curated for your role and unmatched peer networking, Gartner conferences help you accelerate your priorities.
Gartner Identity & Access Management Summit
Las Vegas, NV
Drive stronger performance on your mission-critical priorities.