When evaluating AI/ML offerings, how do you decide which is best for you and your organization?

Machine Learning (ML)Artificial Intelligence (AI)

1.6k views1 Upvote3 Comments

Sort by:

Head of Information and Data Analytics in Software5 years ago

It's not just about deploying the tool, but focusing on the value part. This is generally missing in general IT, and it’s even worse when it comes to security. It's not just about the speed to insight, but focusing on the insight to action, how do you connect value, and is that going to be right for your stack? That's where the focus should be.

1 Reply

no title5 years ago

Something that we use to our advantage a lot of the time is… we find the data that we know is good, or the traffic that we know is normal, and we put it aside. And then we focus on what's left. We look at that and we try to figure out what is good and what's not. And we throw that out and we focus on what's left. Eventually we work ourselves down to a data set that's manageable, and then we can figure out what's really going on by looking at the big picture and saying, okay, forget all the noise, right. The noise is good. We've checked it. Let's figure out what we don't know. And that's where we find most of our issues. That's hard to do, but that's really how we do it with bigger data sets. You can't look at it all. You got to start saying, this is good. This is good. This is good…. Oh, what's this? And then work your way from there.

Sr. Director, Security Engineering & Threat Management5 years ago

Probably one of the more difficult things, I find, is trying to keep up with all the tech, but also be pragmatic in the process. Because it's easy to go out and want to buy all the new cool stuff. But is it really realistic to try and implement and deploy all that, manage all that? That's the microservices conundrum. Someone just posted something recently... “I thought that moving to microservices was gonna make my job easier for everything. And now every time we have an outage, it's like murder mystery.” So it's interesting to kind of see the give and take that comes with that. A lot of offerings are all going to have things that are going to be more or less localized to what you're looking for.

Content you might like

What is the one business pain point / challenge that you regularly face, that you wish AI had solved it? The problem could be a time consuming, routine task that is frustrating to handle. It may happen daily / weekly / monthly, and you may lose many hours on it which results in delays / added cost.

What are key ways in which your organization should hire affordable AI and data science talent? Check all that apply.

Recruit talent from diverse or non-traditional backgrounds (e.g. different degrees, institutions, or work experience)33%

Recruit less experienced AI talent with a high aptitude to learn 45%

Communicate the intrinsic benefits of the role (e.g., mission, culture, resources, opportunity for impact) 30%

Build talent pipelines through partnerships with academia and professional societies41%

Hire and upskill internal talent51%

Use specialized AI recruitment agencies10%

Other (please share details in comments)1%

View Results

We are in the process of implementing a chatbot for our local government site with intentions to deploy for internal use only for testing. We are having problems quantifying is the cost around token consumption. I know that cost varies depending on the model used, but does anyone have any useful estimators. Consumption calculators I have found don't seem particularly useful. What methods for estimating cost did you use? Did you use number of searches against the site?

What’s the biggest hype vs. reality gap in AI pentesting today?

Coverage—AI claims full scan, but misses deep flaws49%

Speed—AI is fast but error-prone63%

Creativity—AI scripts can’t improvise10%

Integration—vendor tools don’t plug into DevSecOps27%