How have you calculated ROI of AI solutions (including agents) that you've rolled out at your firm? Are there specific KPI's that you've focused on and how have you measured (and validated) the value?

Yes. Our firm has calculated the hard savings (>$3.5B, over three years) for HR, IT Support and other AI Assistants. Savings are measured in terms of staff reductions for functions addressed by the AI Assistants. KPI metrics are not limited to, but include issue resolution success rates (measured with in-line surveys) and MTTR for requests for support.

At current state of tech, we look at Time Saved as the most obtainable metric. translates clearly into cost of personnel if you can avoid additional hires (either for growth or to backfill folks leaving). We find other measures to be super subjective...

We're encouraging clients to estimate ROI in both hard and soft savings compared to cost of deployment and licenses. "Hard" savings being revenue-impacting and direct cost savings (including through reduced headcount), which is what most CFOs care about. "Soft" savings may still have merit if they address particular pain points across the org that affect a large number of users, but we don't want every employee to work on the things that annoy them about their jobs without bringing real dollar value to the company.

In terms of calculation, it's much easier with the hard savings vs. soft savings, but if you're saving folks time in internal processes, it may reduce user downtime or improve customer experience, which can still have some dollar amount attached.

We calculate ROI using a combination of quantitative and qualitative factors, which vary depending on the client. Quantitative factors include utilization, time saved, cost savings, and new revenue opportunities. Qualitative factors include client satisfaction, deliverable quality, retention, and engagement. This two-pronged framework provides a validated view of business value, benchmarked over time to guide strategic deployment.

The most effective approach we’ve found—with partners and customers—is to evaluate ROI at the use‑case level, tied to the P&L. We segment by value driver and validation method:
 • Efficiency plays (e.g., back-office): hard ROI via FTE-hour reduction, cycle-time compression, error-rate cuts, and throughput;
 • Growth plays (e.g., sales force automation at scale): blended ROI via incremental revenue, conversion lift, pipeline velocity, and adoption/usage. Example: we enabled 500+ reps to access actionable insights via voice in WhatsApp and reported improved time-to-insight and call prep, translating into higher contact rates and win rates.

Principles:
 • Start with the unit of value (per case, per ticket, per rep) and roll up.
 • Isolate impact (control groups, before/after) and de-bias with counterfactuals.
 • Track both cash and non-cash benefits (risk, compliance, CX), but prioritize cash conversion within 12 months.
 • Define owner, baseline, target, and validation plan before scaling.
In short: ROI is case-by-case, but governed by a consistent framework that ties operational metrics to financial outcomes and uses disciplined measurement to validate value.