Can you share an example of an AI opportunity that initially seemed promising but didn't deliver the expected value? How is that informing your current approach?

I think most initiatives beyond the basics have presented challenges. One example we’ve been working on is trying to normalize data from very similar documents within our portfolio. On a monthly basis, the boards of directors of our portfolio companies meet and discuss very similar information. However, we find that each company tells its own story, so the data is always slightly different in a variety of ways.
We were hoping to use AI to cut through that and really normalize the information, but that has turned out to be a very difficult proposition. Even when focusing on just one individual value and trying to write a prompt that consistently pulls that value across a wide variety of documents, it has proved to be quite challenging.

We did have a couple of these situations where an opportunity sounded very promising and easy, but once we got into the thick of it, we realized it required a lot more investment, time, and effort before it would start paying dividends. For example, we were trying to train a model to predict profitability based on historic information, using 10, 20, or even 30 parameters as input. The idea was to predict and forecast profitability, demand, and a few other things.
As we started developing the model, we put it out for our forecasting team to review. However, it wasn’t giving the confidence we had hoped for, and we had to tweak it through eight or nine iterations. Eventually, the business decided to just live with it, but I’m not sure they are using it in the way they initially thought they would. It turned out to be much harder to train a model with that many parameters and to get meaningful, confident results, especially since macroeconomics and the political landscape kept changing—factors that weren’t accounted for in the model. Ultimately, the business gave up on that initiative.
The lesson learned was that building and maintaining such a model requires a sustained effort. It’s not a one-time training exercise after which you immediately start reaping benefits. It needs constant feeding and care if you want to continue down that path, and I don’t think our business was ready to invest further because they were not seeing the ROI.