Ahead of the Gartner Data & Analytics Summit in Mumbai in June, we asked Ehtisham Zaidi, principal research analyst at Gartner, about the issues facing Indian organizations with a traditional enterprise data warehouse (EDW).
Q. What are the top issues faced by organizations with a traditional enterprise data warehouse (EDW) in India?
A. Our clients in India with a traditional enterprise data warehouse (EDW) have informed Gartner that their top issues include: long time-to-delivery of integrated data ready for business intelligence (BI)/analytics, mainly due to inflexibility in integrating data from big data stores (from Hadoop data stores, for example), and the performance challenges of existing enterprise data warehouses leading to long BI/analytics implementation cycles.
Q. Why are traditional EDWs losing their relevance?
A. An increasingly broad diversity of service-level expectations — including data quality, data governance, diverse processing languages and demands for more flexible queries — all combine to reduce the effectiveness of traditional EDWs, making them rigid and costly to implement and maintain, and forcing organizations to look at alternate logical data warehouse (LDW) architectures.
Q. What are logical data warehouse (LDW) architectures.
A. The logical data warehouse (LDW) is a growing data management architecture for analytics that combines the strengths of traditional enterprise data warehouses (EDWs) with alternative data management and access strategy — specifically data virtualization and distributed processing.
This modern data management architecture allows organizations to use their existing investments in EDWs to expand their scope of performing analytics on traditional data types to also incorporating modern data types and data sources, such as big data, and Internet of Things (IoT) data, with agility and flexibility.
Q. What are the drivers for adoption of LDW architectures?
A. We see the insatiable demand to access data in real-time from a myriad of data sources, such as cloud, mobile, IoT sources, big data stores (for example, Hadoop, and NoSQL) and a host of newer data types such as JSON, XML, Avro, and Parquet . With this proliferation of data types and data sources, organizations simply cannot rely on a repository centrioc, slow and non-real-time strategy of EDWs. They need the flexibility and agility of LDW architectures to cater to this data diversity dynamic or risk being rendered irrelevant from a competitive differentiation standpoint.
Q. What are the market implications of LDW for Indian data and analytics leaders?
A. Indian data and analytics leaders must prepare and respond to the changing dynamics in data management for analytics. They must start looking at ways to incorporate and integrate different data types from several data sources in near-real time without physically collecting data into a repository style architecture leading to a data silo.
They must look at alternative data integration techniques like data virtualization to connect to data in place and perform analytics on this in real-time, thereby saving a lot of cost in having to move this data and transform this data into a silo (EDW). They must also look at distributed processing techniques like typically provided by big data stores like NoSQL and Hadoop through data lakes which allow them to store data in its native format without having to pre-process it which leads to more cost savings.
Overall, the data and analytics leaders in India need to stand up and take notice of the alternate data management architectures (like the LDW) which are here to augment the traditional EDW strategy to offer the much needed flexibility for faster and more complete analytics.
Q. What are your recommendations for Indian data and analytics leaders
A. India's data and analytics leaders should:
- Identify and evaluate use cases (especially BI/analytics use cases) and modern data sources and applications that can benefit from the flexibility in data access, and the faster integration and time-to-solution of an LDW.
- Evaluate your existing EDW platform to determine if new technology or software is to support data virtualization and distributed process management. Do not forget to include your current vendor's roadmap. Many data warehouse vendors have already anticipated at least portions of the LDW.
- Assess the maturity of your enterprise information governance program as a prerequisite to investing in logical data warehouse design principles. Governance helps information management succeed anywhere, and in analytics and the LDW (especially with its distributed capabilities), it is paramount. Do not delay in making sure an information governance program is operational, with enterprise visibility and mission, and rooted with strong leadership.
Mr. Zaidi will present more detailed analysis at the Gartner Data & Analytics Summit on June, 6-7, 2017 in Mumbai