Which vulnerabilities of LLMs are the most imminent & critical to guard against?

Machine Learning (ML)Artificial Intelligence (AI)

Prompt injection15%

Insecure plug-in design49%

Insecure output handling25%

Sensitive information disclosure12%

Model Denial of Service

113 PARTICIPANTS
843 viewscircle icon1 Comment
Sort by:
CISO in Software2 years ago

On this topic, recommend checking out Caleb Sima recent talk: https://www.youtube.com/watch?v=q_gDtOu1_7E&t=1s&ab_channel=CloudSecurityAlliance

Content you might like

What is the one business pain point / challenge that you regularly face, that you wish AI had solved it? The problem could be a time consuming, routine task that is frustrating to handle. It may happen daily / weekly / monthly, and you may lose many hours on it which results in delays / added cost.

Read More Comments

What are key ways in which your organization should hire affordable AI and data science talent? Check all that apply.

Recruit talent from diverse or non-traditional backgrounds (e.g. different degrees, institutions, or work experience)33%

Recruit less experienced AI talent with a high aptitude to learn 44%

Communicate the intrinsic benefits of the role (e.g., mission, culture, resources, opportunity for impact) 31%

Build talent pipelines through partnerships with academia and professional societies41%

Hire and upskill internal talent52%

Use specialized AI recruitment agencies10%

Other (please share details in comments)1%

View Results

What’s the biggest hype vs. reality gap in AI pentesting today?

Coverage—AI claims full scan, but misses deep flaws48%

Speed—AI is fast but error-prone65%

Creativity—AI scripts can’t improvise10%

Integration—vendor tools don’t plug into DevSecOps28%

View Results

What are the metrics on exposure management do you think are most important for the business? I started my career as Vulnerability Management practitioner and have seen the technology and market evolve from VA to VM to RBVM to CTEM now. Yet, VM platforms still haven't solved how to report VM to the senior management effectively, most of them reporting compliance only. The value prop of VM is changing from compliance to exposure management, and exposure reporting is still not easy.  I was speaking to a fellow CISO who measured the exposure by the number of chatter on Dark Web and showed the progress in 1 year, which decreased drastically after a year due to their efforts. What do you think?

Read More Comments

‘AI’ Business Model – With many components flowing into the AI domain (cost, data, E&C, people, value, strategy, duplication of everything, etc.), I’ve started to think about splitting out ‘AI’ from the operating model and putting it into a separate legal entity. This way, I could manage a) risk and compliance, b) cost, c) resource allocation, d) governance, e) IP, f) revenue generation, etc.

Of course, this isn’t new in general, but I’m especially interested in how this approach could help with the ongoing challenge of ensuring compliance with data privacy and regulations related to LLMs and data access/usage over time.

My question: Is anyone else thinking about this, or has anyone already done it? I know there are examples in the literature, but I wanted to float this here for general comments and discussion.

Read More Comments
Which vulnerabilities of LLMs are the most imminent & critical to guard against? | Gartner Peer Community