For those who have trialed an AI pentest solution: what was the single biggest gap you encountered?

Applications & PlatformsEngineering
2.3k viewscircle icon1 Upvotecircle icon2 Comments
Sort by:
Chief Information Technology Officer in IT Services4 months ago

From my experience, the biggest gap with AI-driven pentest tools is depth of context. They’re good at scanning broad attack surfaces quickly, but they often miss nuanced issues—like business logic flaws, privilege escalation paths, or chained exploits across systems. Without human creativity to connect the dots, results risk being shallow. The question is how to best blend AI speed with human expertise for meaningful coverage.

CISO in Software4 months ago

Sometimes, they need more environmental context to target the right APIs and instances.

Content you might like

Do you leverage managed threat hunting at your organization?

Yes78%

No22%

How are large professional services organisation deploying GPT4 (or similar LLMs) within their own organisation for internal use, particularly when using sensitive/confidential/PII data?

For example we're a UK based accountancy firm so have created our own internal app based on Microsoft's Open AI GPT4 model deployed to our private Azure hosting. We have built a custom app designed to "preprogramme" personas, i.e. pre-created prompts, for different areas of the profession, so internal users can have GPT4 act in different ways without prior knowledge of prompt engineering.

I would be interested to see how others are building or buying solutions, particularly those based in EU/UK who can't use OpenAI/ChatGPT for with customer/confidential information.

Read More Comments

I lead a department that builds APIs that power the client experience. I am looking to rebrand and establish a new department name that more accurately reflects what we do.  Our current name is API and Microservices however, we are not the only department that builds APIs. I am looking for suggestions for a new department name.

Read More Comments

If you've been piloting Microsoft Copilot, what advice would you give to other CIOs considering implementing Copilot?

Read More Comments

Which collaboration tools are you most concerned about in terms of security? (select all that applies)

Slack18%

Microsoft Teams36%

Office Cloud Apps (Excel, Word)27%

Microsoft Outlook39%

Microsoft SharePoint/OneDrive23%

Google Chat19%

Google Docs and Sheets17%

Google Gmail19%

Google Drive21%

Dropbox19%

Box8%

Zoom22%

DocuSign5%

View Results
For those who have trialed an AI pentest solution: what was the single biggest gap you encountered? | Gartner Peer Community