Predicts 2026: Secure AI Agents to Avoid Ungoverned Sprawl and Abuses
17 December 2025 - ID G00841644 - 15 min read
By Jeremy D'Hoinne, Andrew Walls, and 3 more
Gartner observed that the industry has entered a new period of AI turbulence driven by ungoverned AI agent sprawl. Cybersecurity leaders must learn from past disappointments and difficulties to successfully manage and secure the next wave of generative AI integrations and developments.
Overview
Key Findings
Organizational focus has shifted from initial GenAI chatbots to the deployment of AI agents embedded in enterprise software and development of custom-built stand-alone AI agents and even multiagent systems (MAS).
Employee experimentation with internal GenAI and AI agents highlights ineffective access control of unstructured data distributed across a range of internal and cloud-based storage locations.
Executives rushing to unlock AI value push to deploy customer-facing applications before cybersecurity teams can enforce strong security practices.
Teams procuring AI applications, especially in highly regulated sectors, increasingly favor solutions that explicitly meet explainability mandates.
Recommendations
Inventory high-risk AI agents based on data access and agency via cross-team collaboration. Mandate a comprehensive multichannel discovery process but acknowledge that full visibility might be out of reach, especially for third-party AI agents used by employees.
Seek appropriate funding and explore reallocation of existing resources to support business objectives for AI deployments.
Ensure early involvement in custom-built AI application projects to ensure sufficient time exists, resources are planned, and expectations are managed for adequate security controls.
Plan for the ongoing expansion of regulatory requirements related to AI explainability. Build agile compliance and cybersecurity programs that can adapt to evolving mandates and minimize operational disruption.
Analysis
What You Need to Know
Securing AI applications developed internally and employees’ usage of third-party AI applications has become a significant part of the cybersecurity programs. Last year, Gartner predicted that the quieter period witnessed at the time would precede a new wave of turbulence (see Predicts 2025: Navigating Imminent AI Turbulence for Cybersecurity).
In 2026, a new wave of turbulence will take the form of AI agent sprawl, and internal development projects aiming at transforming custom-built AI chatbots into AI agents. When surveying cybersecurity leaders, only 45% have evidence of unsanctioned use of prohibited public generative AI tools,1 though the majority of organizations suspect it. Discovery of various AI usage within the company remains a challenge in 2026, as the available discovery tools are not yet fully ready for AI agent discovery.
The first challenge for cybersecurity leaders in AI security is discovery; however, their primary challenge is managing the scale of the response to what has been discovered. This is true for public AI applications, embedded AI features in enterprise software, and even homegrown applications.
Cybersecurity leaders report that their primary challenges in managing embedded AI features are limited resources for assessment (80%),1 which creates AI cybersecurity debt. Cybersecurity debt refers to the subset of technical debt that is within the control of the cybersecurity teams. It is typically associated with building cybersecurity best practices, enforcing cybersecurity policies and implementing technical controls (see 4 Types of Cybersecurity Technical Debt CISOs Can’t Ignore and Build an Adaptive Roadmap to Secure and Enable the Use of AI). While introducing the concept might be an unnecessary layer of abstraction for some teams, many cybersecurity leaders already express regrets in how they handled the first wave of generative AI adoption and do not want to reproduce the same errors for the next one.
Nearly half of custom-built GenAI applications never reach production or fail to deliver expected value. This creates pacing challenges for cybersecurity teams that have invested early effort into enforcing multiple controls and processes, only to see many projects abandoned. Early involvement is important for collaboration, but it’s also valuable to agree on when application and cybersecurity resources will work on specific measures for each project. This planning helps teams focus their effort when they will have the most impact.
Prioritizing AI security investments isn’t enough. To minimize risks, CISOs must pace cybersecurity spending to match each AI project’s stage and its potential to deliver business results.
This document is part of a series, as securing AI applications and usage involves many activities (see Predicts 2026: 4 Forces Reshaping Application and Data Security)
Strategic Planning Assumptions
Strategic planning assumption: Through 2027, the costs to enterprises from task-driven AI agent abuses will be at least 4x higher than those from multiagent systems.
Analysis by: Jeremy D’Hoinne
Key findings:
Organizations are moving from GenAI chatbots to embedded and custom-built AI agents, with an eye toward multiagent systems (MAS).
While MAS are hyped for their promise of full automation, they are complex to design and fragile due to their reliance on nondeterministic models, emerging protocols and platforms. This leads to abandoned projects or technical bottlenecks that slow down the time to production.
Technology providers, seeking profitability for their substantial investments in AI, are rapidly converting (or simply renaming) existing AI assistants into AI agents. Adding basic automations through the integration of existing internal APIs is the fastest path to production, but it results in ungoverned proliferation of AI capabilities across the enterprise.
The continuing AI adoption frenzy overwhelms cybersecurity teams, preventing them from comprehensively inventorying agents and their data access and autonomy. This pushes them back to a default “no” position, creating friction with business teams who feel already pressured to deliver value and productivity improvements.
Market implications:
Attempts to perform full discovery of AI agents, especially those embedded in existing enterprise software, deployed locally or accessible as SaaS, will create an inventory bottleneck for cybersecurity teams.
Securing embedded AI agents primarily depends on the application security features offered by the software provider. These controls often lag behind the release of the AI agent itself, creating inevitable cybersecurity technical debt.
Multiagent systems will rely on low-maturity protocols for tool invocation (e.g., MCP) or agent-to-agent (A2A), but cybersecurity teams will have better visibility on these projects and more time to mandate foundational attack surface minimization practices.
Recommendations:
Inventory high-risk AI agents based on data access and agency via cross-team collaboration.
Mandate a clear, multichannel discovery process. Use SIEM, web proxies or endpoint controls. Collaborate across teams and consult AI agent registries, if implemented. Acknowledge that full visibility might be out of reach, especially for AI agents used by employees.
Prioritize identity and access management because it is the most foundational control, applying to all types of AI agents. Enforce an identity delegation policy, with a single, traceable identity per agent, and as granular access control needed for resource access and tool instrumentation (see How to Securely Delegate Access From Humans to AI Agents). Minimize agency early in the AI agent life cycle, starting with a preference for narrow-scope agents over generalists. Select third-party AI agent offering tool containment features and implement least privilege agency principles for custom-built AI agents.
Favor the “test and prevent” approach for nondeterministic models, such as LLMs to optimize the complementary runtime controls based on the results of comprehensive security testing, including adversarial prompting.
Communicate outside of the cybersecurity team about the inevitable residual risks, especially when it comes to prompt injections.
Strategic planning assumption: Through 2030, 1/3 of the work in IT will be spent remediating AI data debt to secure AI.
Analysis by: Andrew Walls
Key findings:
Existing data is not AI-ready: Over 75% of data management leaders indicate that AI-ready data is one of their top five investments for the next two to three years (2024 Gartner Evolution of Data Management as a Dedicated Function Survey).3
Lack of AI-ready data inhibits AI implementation: Data availability or quality is rated as one of the top three barriers to AI implementation among 30% of executive and AI leaders (2024 Gartner AI Mandates for the Enterprise Survey).2
Poorly secured, unstructured data is everywhere: Employee experimentation with internal GenAI and AI agents highlight ineffective access control of unstructured data distributed across a range of internal and cloud-based storage locations
Market implications:
Data guardians and IT leaders will expand their investments in products and services that automate the discovery of stored data, review access controls, and modify access privileges.
Cybersecurity leaders will expand data loss prevention (DLP) mechanisms to monitor and restrict data flows triggered by GenAI and agentic AI data access events and requests.
Chief data and analytics officers (CDAOs), AI, and cybersecurity leaders will task employees with the review and remediation of data quality and security across all data repositories.
Recommendations:
Cybersecurity leaders should collaborate with the CDAOs and AI leaders to define a structured program of data discovery, assessment, and data access control remediation.
Based on the defined program, estimate the time and resources required to support business objectives for AI deployments, seek appropriate funding, and explore reallocation of existing resources.
Cybersecurity leaders should work with data and system owners to prioritize data for review, improvement, and revision of access privileges to support both humans and AI agents.
Strategic planning assumption: By 2028, organizations skipping preproduction offensive testing on GenAI deployments will face twice as many cybersecurity incidents as proactive users.
Analysis by: Bart Willemsen and Jeremy D’Hoinne
Key findings:
Pressure to increase AI technology use for the realization of expected value pushes executives to launch customer-facing AI applications.
This pressure leads to bypassing security best practices. Combined with the often experimental nature of AI application technology, the resulting attack surface considerably increases.
Cybersecurity professionals are consistently involved way too late in the process; at best right before AI applications go to production. This eliminates any chance of successful preparation.
The added AI-specific security risks can be caught with consistent adherence to emerging best practices, but the majority of organizations have yet to even start building an AI red teaming program.
AI red teaming programs suffer from a relatively low technology maturity, with difficulties increasing mainly due to nondeterministic components, such as foundation models. No tool can provide perfect protection against direct and indirect prompt injections today.
Market implications:
Accelerated AI deployments without proper security controls weaken operations, leaving public-facing applications vulnerable, lacking vulnerable component detection (including across the supply chain), infrastructure hardening to minimize exposure from misconfigurations, secure (custom) code development practices, and more.
These weaknesses make AI applications especially appealing to attackers, since they often have back-end access to valuable data. attractive targets for attackers. It works as an invitational trigger to prioritize these systems as targets, especially since the applications often have back-end access to valuable data.
In operational practice, the problems resulting from the overly hasty and insufficiently controlled AI applications extend well beyond the realm of cybersecurity incidents. Security validation, offensive testing and similar risk-reducing activities also help prevent devastating events in the areas of privacy (e.g., data breaches), trustworthiness (e.g., discrimination or other content/bias controls) and more.
Believing that AI security can be quickly solved with off-the-shelf tools is misguided and leads vendors to rely on jargon-heavy messages and overly polished demos.
While offensive AI security testing technology still needs to mature, it can already serve today as a cybersecurity benchmark for custom-built AI applications, highlighting gaps in controls and serving as support for communication on residual risks.
Recommendations:
Cybersecurity leaders must:
Proactively involve themselves in AI application projects at the earliest possible development stages, to ensure sufficient time exists, resources are planned, and expectations are managed for adequate security controls.
In the absence of a red team, use public, consumer-facing AI projects to showcase security validation and red teaming programs benefits. Once these applications are live, they have the greatest impact on brand reputation and customer confidence. Leverage the success story to extend red teaming approaches into other systems.
Encourage existing red teams to build literacy by experimenting with open-source tools, such as PyRIT or Promptfoo.
Include consistent requirements in vendor questionnaires, both for existing vendor relations and new ones, to gauge to what extent their application security testing activities include features like adversarial prompt generation.
Enhance effectiveness by integrating AI security tools with runtime controls to ensure adequate treatment of the discovered vulnerabilities.
Strategic planning assumption: By 2029, vendor noncompliance with regulatory requirements for AI explainability audits will delay 50% of critical cybersecurity deployments.
Analysis by: Niyati Daftary
Key findings:
Most vendors lack mature processes to secure and validate AI decision making explainability, leaving them exposed to regulatory noncompliance, integrity and reputational risks.
Global Jurisdictions, including the EU (AI Act), the U.S. (FTC guidance), and the U.K., are introducing explainability requirements that outpace vendors’ readiness. Any technology, product or consulting engagement leveraged in cybersecurity projects, failing these audits can trigger compliance delays, extending timelines and impacting strategic roadmaps.
Procurement teams in highly regulated sectors, therefore, increasingly prioritize AI solutions that comply with emerging explainability mandates, ensuring that AI-driven decision-making tools, including those in cybersecurity, are transparent, auditable and aligned with regulatory standards.
Vendors failing to meet these standards will face remediation timelines, fines, or contract termination, which will indirectly impact cybersecurity projects that depend on them, delaying schedules and forcing roadmap revisions, thereby jeopardizing strategic objectives.
Organizations relying heavily on AI-driven cybersecurity tools will be most vulnerable to these delays.
Market implications:
Divergent AI regulations (such as the EU AI Act, emerging U.S. state-level laws, etc.) will force vendors to maintain region-specific AI architectures, increasing complexity, attack surface and slowing global expansion.
CISOs may not own explainability governance but will need to coordinate with AI and compliance teams to avoid delays in cybersecurity initiatives.
AI-driven cybersecurity solutions (e.g., threat detection, anomaly response, AI-assisted vulnerability management) will likely experience longer release cycles due to the introduction of mandatory AI explainability audit checkpoints.
Cybersecurity leaders failing to invest in secure, explainable AI capabilities today will face significant operational and strategic setbacks like delayed execution of critical cybersecurity initiatives, inability to meet regulatory mandates, costly migration plans and increased risk exposure.
Regulatory scrutiny will expand gradually, but its existence will influence procurement decisions well before strict enforcement begins.
Recommendations:
Embed AI regulatory requirements into cybersecurity risk management by aligning cybersecurity policies, vendor assessments and technology roadmaps to ensure compliance and resilience against integrity risks.
Plan for the ongoing expansion of regulatory requirements related to AI explainability. Build agile compliance and cybersecurity programs that can adapt to evolving mandates and minimize operational disruption.
Develop mitigation strategies for critical cybersecurity projects dependent on AI solutions. This includes identifying alternative vendors, maintaining manual fallback processes and allocating buffer time in project timelines for potential compliance audits.
Collaborate closely with procurement to integrate AI explainability and cybersecurity compliance requirements into vendor evaluation criteria, safeguarding resilience and reducing deployment delays.
A Look Back
In response to your requests, we are taking a look back at some key predictions from previous years. We have intentionally selected predictions from opposite ends of the scale — one where we were wholly or largely on target, as well as one we missed.
This report is too new to have on-target or missed predictions.
1 2025 Gartner Cybersecurity Innovations in AI Risk Management and Use Survey. This survey was conducted to understand how organizations manage the cybersecurity risks associated with generative AI (GenAI) and AI techniques that support it. The research was conducted online from 21 March through 9 May 2025 among 302 cybersecurity leaders in North America (n = 181), EMEA (n = 71) and Asia/Pacific (n = 50) regions. Qualifying organizations reported enterprisewide revenue of at least $250 million or equivalent for fiscal 2024 and were senior cybersecurity management involved in activities related to AI cybersecurity risk management within their organization. Disclaimer: The results of this survey do not represent global findings or the market as a whole, but reflect the sentiments of the respondents and companies surveyed.
2 2024 Gartner AI Mandates for the Enterprise Survey. This study was conducted to understand how AI and generative AI (GenAI) are being adopted by enterprises, focusing on areas such as AI strategy, data, governance, literacy, engineering, organization, portfolio and value, to assist clients in keeping pace with AI’s rapid evolution. The research was conducted online from October through December 2024 among 432 respondents from the U.S. (n = 181), the U.K. (n = 70), France (n = 50), Germany (n = 50), India (n = 51) and Japan (n = 30). Quotas were established for company sizes and for industries to ensure a good representation across the sample. Organizations were required to have deployed at least one AI use case in production. Respondents were screened for C-level executives (e.g., chief AI officer, chief data officer, chief data scientist, chief digital officer, chief information officer, chief operating officer, chief technology officer or equivalent) or roles above vice presidents. All respondents were required to have high involvement in at least one AI initiative. Disclaimer: The results of this survey do not represent global findings or the market as a whole but reflect the sentiments of the respondents and companies surveyed.
3 2024 Gartner Evolution of Data Management as a Dedicated Function Survey. This survey was conducted to establish the characteristics of a successful data management function and understand the future operating model, architecture and investment areas of data management teams. It also sought to identify what makes data management leaders successful in delivering data to business domains, meeting their SLAs and being able to defend their position by showcasing value. The research was conducted online from August through September 2024 among 248 respondents from across the world. Respondents were required to have involvement in, knowledge of and responsibility for implementing the data management side of the D&A strategy at their organizations. Disclaimer: The results of this survey do not represent global findings or the market as a whole but reflect the sentiments of the respondents and companies surveyed.