Insights / Finance / Article

Financial Forecasters Should Beware 3 Machine Learning Myths

April 02, 2019

Contributor: Rob van der Meulen

Many financial planning and analysis (FP&A) teams see machine learning as key to finance transformation, but for every success story, there are many failed implementations.

Financial planning and analysis teams need to better understand the limits and advantages of machine learning (ML) to drive finance transformation through improved forecast accuracy and efficiency.

“We often hear FP&A leaders — and corporate controllers — worry that other companies or even other corporate functions will outflank them by turning information into valuable insight before they do,” says Randeep Rathindran, VP, Gartner. “Given that context, we could argue that of the half of finance leaders who plan to deploy predictive analytics technology by 2020, at least some are panic buying.”

“ For now, Gartner advice for the finance function is to experiment with machine learning without incurring a lot of cost or risk”

On the one hand, finance leaders must recognize the potential for effective finance transformation using advanced analytics, ML and artificial intelligence (AI). But on the other hand, it’s important to keep in mind a sober truth: The path to ML and AI nirvana is littered with the corpses of failed implementations.

Download eBook:10 Digital Finance Imperatives

“Finance leaders must cut through the hype around this topic and make sure they understand the true advantages and limits of machine learning as it relates to forecasting,” says Rathindran. “This is not always the case. We regularly hear of troublesome projects that are based on several common misconceptions.”

ML produces more accurate forecasts than traditional methods

In reality, ML is not the right tool for every forecasting job. Most companies forecast with fairly simple methods such as trending or exponential smoothing. Their business units create forecasts from local data sources, such as a sales pipeline tool. If these forecasts already perform with a high level of accuracy, it’s unlikely applying ML will fix what isn’t broken.

Three key characteristics help identify the right problem for ML. First, understand the prediction performance you hope to beat, giving a clear benchmark for success. Secondly, select a problem where the dataset has high integrity and contains the kind of phenomena you wish to forecast. Thirdly, don’t apply ML to situations where codified rules — however complicated — already do or could work.

Learn more: Transform Your Finance Team

Implementing ML-enabled forecasting requires a data scientist and a comprehensive business case for funding

ML software, even in cutting-edge use cases, tends to be free or open source. In that respect the barriers to entry are low, and therefore talent is the obvious impediment to FP&A experimenting with ML. It may be worth hiring an experienced data scientist, but it’s not a necessity.

Given the established skill set of an FP&A team, the chances are high that everything needed to start experimenting with ML is already in place. A creative, critical thinker with deep knowledge of the business and some basic coding skills can make an effective citizen data scientist.

Some companies even find that analysts with non-IT backgrounds make better citizen data scientists than certified programmers because they are more open-minded and willing to reframe their thinking.

ML-enabled forecasts will replace traditional forecasting methods and won’t need human intervention

Gartner experts often encounter the claim that either data or modeling is the key to getting ML right. This dichotomy is unhelpful. Forecasters must understand there’s no “easy part.” Analysts experimenting with ML typically begin with a handful of ML models, called an ensemble.

The analysts then systematically train, measure and compare the prediction performance of these models, individually and as an ensemble. A big part of this effort is tuning the models’ hyperparameters — those values not determined by the problem at hand or the data — which alter how the model learns from the data.

“ Aim to evaluate how the technology can best augment the FP&A team’s analysis and challenge business-generated forecasts”

Even where some parts of the process are automated, the time and attention of humans who know what to look for and how to make adjustments to improve prediction accuracy is still required. Forecasters must also help the business own the forecast, rather than prejudge it as unrealistic or unattainable simply because it is machine-generated. These types of work exemplify the skills most finance teams need to develop or acquire to succeed with ML.

Even when the best-trained models are chosen, they will require regular monitoring and maintenance to retrain them on fresh data. Moreover, basic principles of due diligence and internal control still apply to ML.

For now, Gartner advice for the finance function is to experiment with ML without incurring a lot of cost or risk. Aim to evaluate how the technology can best augment the FP&A team’s analysis and challenge business-generated forecasts, rather than trying to replace them altogether.

Gartner CFO & Finance Executive Conference

Connect with our experienced practitioners and unbiased Gartner experts to discover innovative approaches to smarter business decisions and more.

Recommended resources for Gartner clients*:

Some Gartner clients can read more in 3 Machine Learning Myths for Forecasters by Randeep Rathindran.

*Note that some documents may not be available to all Gartner clients.