What Is Data and Analytics?

The role of data and analytics is to equip businesses, their employees and leaders to make better decisions and  improve decision outcomes. This applies to all types of decisions, including macro, micro, real-time, cyclical, strategic, tactical and operational. At the same time, D&A can unearth new questions, as well as innovative solutions and opportunities that business leaders had not yet considered.

Progressive organizations use data in many ways and must often rely on data from outside their boundary of control for making smarter business decisions.

Data and analytics is also a catalyst for digital transformation as it enables faster, more accurate and more relevant decisions in complex and fast-changing business contexts.

Both individuals and organizational teams make decisions, for example, when a person considers whether to buy a product or service, or when a business function determines how best to serve a client or citizen.

Data-driven decision making means using data to work out how to improve decision making processes. This leads to the idea of a decision model, which can include prescriptive analytical techniques that generate outputs that specify which actions to take. Other analytical models are descriptive, diagnostic or predictive (also see “What are core analytics techniques?”). Each can help with specific kinds of decisions. 

Notably, decisions drive action but may equally determine when not to act. 

Progressive organizations are infusing data and analytics into business strategy and digital transformation by creating a vision of a data-driven enterprise, quantifying and communicating business outcomes, and fostering data-fueled business changes. (Also see “How do you create a data and analytics strategy?”)

Scaling digital business complicates decision making and requires a mix of data science and more advanced techniques. The combination of predictive and prescriptive capabilities enables organizations to respond rapidly to changing requirements and constraints.

The nature and complexity of the problem determines the choice of whether and how to use prediction, forecasting or simulation for the predictive analysis component. (Also see “What is advanced analytics?” and “What are core analytics techniques?”)

The following use case examples combine the predictive capabilities of forecasting and simulation with prescriptive capabilities:

• Forecasting the risk of infection during a surgical procedure combined with defined rules to drive actions that mitigate the risk

• Forecasting incoming orders for products combined with optimization to proactively respond to changing demand across the supply chain, without relying on historical data that might be incomplete or “dirty”

• Simulating the division of customers into microsegments based on risk combined with optimization to quickly assess multiple scenarios and determine the optimal response strategy for each

Organizations also use data and analytics in different ways for different types of decisions. Making more effective business decisions requires executive leaders to know when and why to complement the best of human decision making with the power of data and analytics and AI.

It’s important for each organization to ask, what is data and analytics for us and what initiatives (projects) and budgets are necessary to capture the opportunities. 

The key steps in planning data and analytics strategy are to:

• Start with the mission and goals of the organization

• Determine the strategic impact of data & analytics on those goals

• Prioritize action steps to realize business goals using data & analytics objectives

• Build a data & analytics strategic roadmap

• Implement that roadmap (i.e., projects, programs and products) with a consistent and modern operating model

• Communicate data and analytics strategy and its impact and results to win support for execution

The enterprise operating model for data and analytics must also work to overcome gaps in the data ecosystem, data architectures, organizational delivery approaches and skills — including data analyst, data scientist and data engineering skills — needed to execute the D&A strategy.

Gartner defines data literacy as the ability to read, write and communicate data in context. It requires an understanding of data sources and constructs, analytical methods and techniques, and the ability to describe the use-case application and resulting value. This might sound like an argument for training every employee as a data scientist or data analyst, but that’s not the case. From a business perspective, you might simply summarize data literacy as a program to help business leaders learn how to ask smarter questions of the data they have available.

Building data literacy within an organization is a culture and change management challenge. D&A is ever-more pervasive in all aspects of all business, in communities and even in our personal lives. The ability to communicate in the associated language — to be data literate — is increasingly important to organizations’ success. However, this kind of lasting, meaningful change requires people to learn new skills and behavior.

Therefore best practices include putting more emphasis, energy and effort into the change management piece of D&A strategy, leveraging leadership and change agents and addressing both data literacy skills, or aptitude,  and culture, or attitude. Data literacy starts with a leader taking a stance. For example, the CIO or chief data officer, along with the finance (usually business intelligence [BI] leaders) and HR organizations (development and training), can introduce data literacy programs to provide their peers with the tools to adapt and adopt D&A in their respective departments.

As part of an overall data literacy program, data storytelling can create positive and impactful stakeholder engagement by applying techniques to frame data and insights into data-driven stories. These make it easy for stakeholders to interpret, understand and act on the data being shared.

Data and analytics governance — also called “information governance” — specifies decision rights and accountability to ensure appropriate behavior as organizations seek to value, create, store, access, analyze, consume, retain and dispose of their information assets. It’s critical to link data and analytics governance to the overall business strategy and anchor it to the data analytics assets that organizational stakeholders consider critical.

Data and analytics governance encompasses the people (such as executive policymakers, decision makers and business D&A stewards), processes (such as the D&A architecture and engineering process and decision-making processes) and technologies (such as master data management hubs) that provision trusted and reliable mission- critical data throughout an enterprise.

Notably, while governance originally focused only on regulatory compliance, it is now evolving and expanding to govern the least amount of data for the largest business impact — in other words, D&A governance has grown to accommodate offensive capabilities that add business value, as well as defense capabilities to protect the organization.

Effective data and analytics governance must also balance enterprisewide and business-area governance with a standardized enterprise approach. D&A governance does not exist in a vacuum; it must take its cues from the D&A strategy.

In the past, the group responsible for data was managed independently from the   analytics and insights team.  Data technologies were likewise distinct from analytics technologies. That is changing in many ways. For example, data management platforms increasingly incorporate analytics, especially machine learning (ML).

Analytics and BI platforms are developing data science capabilities and new platforms are emerging to deliver specific functionality for key activities, such as data visualization or D&A governance. Cloud service providers are creating yet another form of complexity as they increasingly dominate the infrastructure platforms on which these services are used.

Traditional platforms across the data, analytics and AI markets struggle to accommodate the growing number of data and analytics use cases.  As a result, organizations must balance the high total cost of ownership of existing, on-premises solutions against the need for increased resources and emerging capabilities. Examples include natural language query, text mining, and analysis of semistructured and unstructured data.

The future of data and analytics therefore requires organizations to invest in composable, augmented data management and analytics architectures to support advanced analytics. Modern D&A systems and technologies are likely to include the following.

Data management solutions

• Master data management (MDM) is a technology-enabled business discipline in which business functions and IT work together to ensure the uniformity, accuracy, stewardship, governance, semantic consistency and accountability of the enterprise’s official shared master data assets.
• Data hubs are focused on enabling data sharing and governance. Producers and consumers of data connect with one another through the data hub, enabled by governance controls and common models that define effective data sharing. MDM is an example of a data hub that is focused only on master data. Data catalogs are increasingly moving into the governance space. They too are starting to become data (and analytics) hubs.
• Data centers physically house servers (as opposed to warehouses, which are data structures housed on servers or in the cloud). Their future depends on the degree to which workloads can be moved to the cloud. Those migration decisions must be based on the business benefits of doing so.
• Data warehouses provide an endpoint for collecting transactional, detailed (and sometimes other types of) data. They support predictable analyses for data whose value is well established — that is, well-known, predefined and repeatable analytics that are scalable across many users in the enterprise.
• Data lakes collect unrefined data (in its native form, with limited transformation and quality assurance and intrinsic governance) and allow users to explore and analyze it in a highly interactive way. Data lakes don’t replace data warehouses or other systems of record; rather, they complement them by storing unrefined data that may hold value. The sweet spot for data lakes is the world of pure discovery, data science and iterative innovation.

Data fabric

Data fabric is an emerging data management design that enables augmented data integration and sharing across heterogeneous data sources. Data fabrics have emerged as an increasingly popular design choice to simplify an organization’s data integration infrastructure and create a scalable data architecture.

If widely implemented, data fabrics could significantly eliminate manual data integration tasks and augment (and, in some cases, completely automate) data integration design and delivery. However, data fabrics are still an emergent design concept.  No single vendor currently delivers, in an integrated manner, all the mature components  needed to stitch together the data fabric. Ultimately, organizations must decide whether to develop their own data fabric using modernized capabilities spanning the above technologies and more, such as active metadata management.

Data fabric also consists of a mix of mature and less mature technology components, so organizations must carefully mix and match composable technology components as their use cases evolve.

D&A in the cloud

Traditional D&A platforms are tasked with handling increasingly complicated analytics. This complexity, coupled with the increased resources needed to  maintain  the environment, is driving growth in the total cost of ownership of on-premises solutions.

In contrast, cloud data and analytics offers more value and capabilities through new services, simplicity and agility to handle data modernization. It can also serve demand for new types of analytics, such as streaming analytics, specialized data stores and more self-service-friendly tools to support end-to-end deployment.

Cloud deployment — whether hybrid, multicloud or intercloud — must account for many D&A components, including data ingestion, data integration, data modeling, data optimization, data security, data quality, a data governance program, management reporting, data science and ML.

Advanced analytics uses sophisticated quantitative methods to produce insights unlikely to be discovered through traditional approaches to business intelligence (BI). It spans predictive, prescriptive and artificial intelligence techniques, such as ML. In short:

• Analytics and BI represent the foundational or traditional way to develop insights, reports and dashboards

• Advanced analytics represents the use of data science and machine learning technologies to support predictive and prescriptive models.

While both are valuable to every organization for different reasons, the market as a whole is changing. Instead of focusing on traditional and separately advanced analytics, the technologies are becoming composable and organizing around roles and personas — from business roles who want self-service capabilities to advanced analytics roles looking to program and engineer.

Advanced analytics is not the same as augmented analytics, which refers to the use of ML/AI techniques to transform how users develop, consume and share insights from analytics.. Augmented analytics includes natural language processing and conversational interfaces, which allow users without advanced skills to interact with data and insights.

Advanced analytics enables executive leaders to ask and answer more complex and challenging questions in a timely and innovative way. This creates a foundation for better decisions by leveraging sophisticated and clever mechanisms to interpret events, support and automate decisions, and take actions.

Advanced analytics can leverage different types and sources of data inputs than traditional analytics does. In some cases, it allows organizations to create net new data, requiring a rigorous data governance strategy and a plan for required infrastructure and technologies. For example, data lakes can be used to manage unstructured data in its raw form. (Also see “What is the future of data and analytics technologies?”)

Advanced analytics provides a growing opportunity for data and analytics leaders to accelerate the maturation and use of data and analytics to drive smarter business decisions and improved outcomes in their organizations. Gauging the current and desired future state of the D&A strategy and operating models is critical to capturing the opportunity.

Data is widely used in every organization. While not all data is used for analytics, analytics cannot be performed without data. The technologies needed across data, all its use cases, and the analysis of that data exist across a wide range, and this helps explain the varied use — by organizations and vendors — of the term “data and analytics” (or “data analytics”).

References to “data” imply or should imply operational uses of that data in, say, business applications and systems, such as core banking, enterprise resource planning and customer service. “Analytics” (or what some call “data analytics”) refers to the analytical use cases of data that often take place downstream, as in after the transaction has occurred.

Analytics, as described, comprises four techniques:

Descriptive analytics

This uses business intelligence (BI) tools, data visualization and dashboards to answer two questions: What happened? or, What is happening? Procurement, for example, can answer questions like: What did we spend on commodity X in the last quarter? and, Who are our biggest suppliers for commodity Y?

Diagnostic analytics

This requires more drilled-down and data mining abilities to answer, why did X happen? For example, sales leaders can use diagnostics to identify the behaviors of sellers who are on track to meet their quotas.

Predictive analytics 

Predictive analytics typically deals with probabilities and can be used to predict a series of outcomes over time (that is, forecasting) or to highlight uncertainties related to multiple possible outcomes (that is, simulation). It tells us what to expect, addressing the question, what is likely to happen? It does not, however, answer other questions, such as what should be done about it?

Predictive analytics relies on techniques, such as predictive modeling, regression analysis, forecasting, multivariate statistics, pattern matching and machine learning (ML).

Prescriptive analytics 

Prescriptive analytics intends to calculate the best way to achieve or influence the outcome — it aims to drive action. When combined with predictive analytics, prescriptive analytics naturally draws on and extends predictive insights, addressing the questions of: What should be done? or, What can we do to make a given outcome happen? 

Prescriptive analytics includes both rule-based approaches (incorporating known knowledge in a structured manner) and optimization techniques (traditionally used by operations research groups) that look for optimal outcomes within constraints to generate executable plans of action. Prescriptive analytics relies on techniques, such as graph analysis, simulation, complex-event processing and recommendation engines.

Combining predictive and prescriptive capabilities is often a key first step in solving business problems and driving smarter decisions. Understanding the potential use cases for different types of analytics is critical to identifying the roles and competencies, infrastructure and technologies that your organization will need to be truly data-driven, especially as the four core types of analytics converge with artificial intelligence (AI) augmentation. (Also see “What is advanced analytics?”)

The term “big data” has been used for decades to describe data characterized by high volume, high velocity and high variety, and other extreme conditions. However, the big data era is epitomized for businesses by its associated opportunities and risks.  On the side of opportunities,  the explosion in data traffic driven by internet use and computing power offers a rich source of insights to improve decisions. On the side of challenges, the same explosion in data creates challenges for organizations related to how they store, manage and analyze big data.

Most organizations have found ways to derive business intelligence from big data analytics, but many struggle to manage and analyze a diverse and broad set of content (including audio, video and image assets) at scale. This struggle has grown as the universe of data sources grows and changes and the need for insights is increasingly enabled by advanced analytics. 

Progressive organizations no longer distinguish between efforts to manage, govern and derive insight from non-big and big data. Today, it's all just data. Instead, they are aggressively looking to leverage new kinds of data and analysis — and to find relationships in combinations of diverse data to improve their business decisions, processes and outcomes. 

Synthetic data, for example, is exploited by generating a sampling technique to real-world data or by creating simulation scenarios where models and processes interact to create completely new data not directly taken from the real world. This is most helpful with ML built on data sets that do not include exceptional conditions that business users know are possible, even if remotely. Such data is still needed to help train these ML models.

The global pandemic and other business disruptions have also accelerated the need to use more types of data across a broad range of use cases (especially as historical big data has proved less relevant as a basis for future decisions). Concerns over data sourcing, data quality, bias and privacy protection have also affected big data gathering and, as a result, new approaches known as “small data” and “wide data” are emerging. 

The wide data approach enables the data analytics and synergy of a variety of small and large data sources — both highly organized largely quantitative (structured) data and qualitative (unstructured) data. The small-data approach uses a range of analytical techniques to generate useful insights, but it does so with less data.

At Gartner, we now use the term X-analytics to collectively describe small, wide and big data — in fact, all kinds of data. We expect that by 2025, 70% of organizations will be compelled to shift their focus from big data to small and wide data to leverage available data more effectively, either by reducing the required volume or by extracting more value from unstructured, diverse data sources. (Also see “What is advanced analytics?”)

This and other predictions for the evolution of data analytics offer important strategic planning assumptions to enhance D&A vision and delivery.