The imperative of building an agile data-centric architecture that augments every aspect of the business and responds to constant change (Gartner calls this the ContinuousNext) has never been more critical to business survival.
The size, complexity, the distributed nature of data, the speed of action and the continuous intelligence required by digital business means that rigid and centralized architectures and tools break down. There’s now too much data, it’s already too distributed, it’s too diverse, it’s too complex.
The path to success requires making the right choices in the face of unprecedented business demands and rapidly changing and complex technology options. We know that a lot of organizations working on data and analytics struggle with balancing between investments that drive innovation and renovating their technology core.
The very challenges created by digital disruption — too much data — have also created an unprecedented opportunity. Leveraging these new vast amounts of data when coupled with increasingly powerful processing capabilities enabled by the cloud (for both data management and data science) makes it now possible to train and execute the algorithms at the very large scale necessary to finally realize the full potential of AI.
We expect to see a growth in the use and application of data science, machine learning and AI across all industries, particularly as AI matures and the democratization of AI accelerates. AI will no longer be for the privileged few. AI-enabled analytics and data management tools will empower the many to process the vast amounts of data needed for advanced analytics at scale. This will make it possible for a broader range of skills, such as citizen data scientist and developer data scientists to create AI-driven insights and embed them in applications used by everyone across the organization. AI as the new UI will feature prominently in this transformation where natural language and conversational interfaces will also open up insights to more people in the organization. Augmented and mixed reality, still in its infancy in the enterprise, will also start to play a role in the enterprise, beyond gaming and entertainment, including in data and analytics. With so much data and insights from diverse sources, data storytelling — part art, part technology-enabled — will be a critical personal and organizational competency necessary to drive optimized actions across the enterprise.
Moreover, the broad distribution of analytics capabilities to everyone in the enterprise is giving less technical individuals across the business the ability to generate value from data assets and analytics. To enable this growth we see considerable adoption of capabilities such as agile data preparation, data cataloging and metadata management technologies, and new adaptive data governance-related processes that are enabling content authors using any tool anywhere in the business to do more in creating and using trusted, high-value data. Having the ability to capture all of the knowledge about what data you have, where it resides, how it’s all related, who uses it, why, when and how — and then using that insight to provide more personalized, automated and properly governed solutions to the business. New roles such as the data engineer and processes supporting dataops and user communities are key enablers of this more distributed, but governed approach.
Capabilities such as digital twins are needed create actionable insight from vast amounts of the Internet of Things (IoT) data while blockchain is maturing to cope with demands for trust in a distributed environment in the face of exponentially increasing data volumes, complexity and pace, and yes, bad actors.