How are you tackling cost estimation for your AI projects? Are there any specific frameworks/methods that have worked well for you?

We are taking an agile approach with AI projects wherein we invest a small amount in an MVP or pilot and prove the ROI out as we go. For instance, we are piloting Microsoft 365 Copilot currently on a small scale. Users who have access are given an expectation that they need to provide feedback through a structured mechanism wherein they quantify the time savings they are getting from the tool. We're dynamically using that information in conjunction with usage details to calculate the total benefit. We're also continually evaluating and quantifying the total cost of ownership as we factor in support cost, change management costs, and licensing costs. Along the way, we've been sharing updates with senior leadership on progress towards a true ROI picture. 

Beyond licensing fees (if you have any) there will be large compute (cloud) costs.  If you use an on prem or cloud datacentre for your compute you have to to understand how costs are calculated in that environment.  You also need to define what the AI system will do (e.g., chatbot, recommendation system, predictive analytics) as they all use different amounts of compute resources.  From there you can start a POC to understand how with x amount of data it will cost you what and then extrapolate from there.  

You must also consider costs for 
 - the Development Team (Salaries for data scientists, AI/ML engineers, testers, and software developers)
 - the dev, training and testing Infrastructure and the deployed infrastructure  (Cloud computing (e.g., AWS, Azure, Google Cloud) or GPU/TPU resources.), Compute hours and infrastructure setup
 - Software Tools and Libraries, Open-source frameworks (TensorFlow, PyTorch) or paid tools (like AutoML).

We recently completed a major Generative AI project for a large retail client. Here are some frameworks we used to estimate costs:

Cloud-Native Approach: Most AI projects, especially Generative AI, require significant cloud infrastructure. We adopted a cloud-native strategy, utilizing the client’s existing cloud environment for deployment. This approach had two main benefits: (i) it removed cloud costs from our cost estimate, and (ii) it addressed data privacy and security concerns by keeping all data within the client’s environment.

AI Readiness: Data is at the core of AI projects. It’s crucial to assess whether the data is ready for AI before jumping into cost estimation. Many times, data requires preparation, which adds to the project scope. This data readiness phase should be factored in as a potential cost discovery stage.

LLM/Prompt Optimization for Generative AI: When dealing with Generative AI, ensuring Large Language Model (LLM) and prompt optimization is key. In our case, we could have improved cost management in this area. As we fine-tuned the LLMs for specific business use cases, we encountered fluctuating accuracy, leading to multiple rounds of LLM tuning. Accounting for this in the estimation is critical for avoiding unexpected costs.