How to Prevent Big Data Analytics Failures

Big data analytics projects don’t fail for a single reason, nor due to technology alone.

A combination of factors usually derails big data implementations. Problems and failures occur due to factors including strategy, people, culture, capacities, inattention to analytics details or the nuances of implemented tools, all exacerbated by the rapid advancement of the digital economy.

Ahead of the Gartner Business Intelligence, Analytics and Information Management Summit in Sydney in February, Gartner research director Svetlana Sicular said that faced with overwhelming choices and new challenges, technical professionals often become distracted from the fundamental priorities — finding the right problems for big data solutions, questioning the data and understanding the nuances of the analytical models that are applied to big data.

“To succeed, you must develop a viable strategy to deliver business value from a big data initiative,” said Ms. Sicular. “Then map out and acquire or develop the missing and specialized skills that are needed. Once strategy and skill priorities are addressed, then you can move on to big data analytics.”

How to Tell a Story With Data and Analytics
Deliver insights you need with data visualization and narrative techniques.
Attend Webinar

Learning from the pitfalls faced by others

We often hear of big data success stories, but some of the most revealing information that can help planners of big data analytics is information about failures. One of the most common failures involves setting overly optimistic expectations when a skilled team is not in place to deliver.

The CEO of a retail chain recognized that to stay competitive, his company needed a recommendation engine, like Amazon’s famous “Customers who bought this item also bought …” feature. The retailer had never undertaken any big data projects before, but executives promised the CEO that an engine would be operational within six months.

Learn More: Drive business alignment and profits with the right strategy.

IT worked hard to implement a collaborative-filtering algorithm that usually powers a recommendation engine, but struggled with the data sparsity and scale of huge datasets of purchased items and customer browsing history, as well as the available inventory that was constantly changing, which required further learning and additional skills.

Once strategy and skill priorities are addressed, then you can move on to big data analytics.

To meet the deadline, the team created a fake recommendation engine with bed sheets as the recommended product, regardless of what was being purchased. While there weren’t any actual analytics behind it, it produced a significant sales lift. It then took two years from the CEO directive before they actually developed an effective engine. In that time, they had added behavioral experts, specialists in operations research and engineers into the team to stabilize the company’s big data infrastructure.

This demonstrates that a key success factor for implementing big data analytics is the organisation’s ability to build, grow and sustain a multi-disciplinary team with the expertise needed to address identified business problems.

Organisations should also allow ample time to deliver meaningful results. Alternatively, they may consider buying a commercial recommendation engine. In this case, the strategy should include a decision on build versus buy.

This is just one of many real-life examples organizations can learn from. Gartner clients can read more in the report: Big Data Analytics Failures and How to Prevent Them by Svetlana Sicular.

Get Smarter

Gartner Data & Analytics Summits

Get the tools and insights you need to build on the fundamentals of data and analytics.

Explore Gartner Events

100 Data and Analytics Predictions Through 2021

Over the next few years, data and analytics programs will become even more mission-critical throughout the business and across industries....

Read Free Research

How to Tell a Story With Data and Analytics

The ways in which organizations deliver business intelligence and analytics insights are evolving, notably in the rising use of what...

Start Watching