Consider how billions of connected devices generating oceans of events, are going to put pressure on an organization’s ability to ingest, store , process and analyze data. The Internet of Things (IoT) will do just that by producing unprecedented volume, velocity and variety of data. This will force organizations to re-architect their data and analytics capabilities, adopt new data management technologies and platforms, and create new data governance policies and practices to act upon all of this data.
During his session at the Gartner Data & Analytics Summit this week in Grapevine, Texas, Ted Friedman, vice president and distinguished analyst at Gartner, explains that through 2019, one-third of IoT solutions will be abandoned before deployment due to lack of data management and analytics capabilities adapted for IoT.
“Many existing data and analytics capabilities can be applied to IoT initiatives and IoT data, but organizations also need to modernize in several key areas,” says Friedman.
Leverage existing data management and analytics capabilities in support of IoT deployments
The new and unique characteristics of IoT solutions put pressure on various aspects of traditional data management infrastructure and traditional analytics and BI techniques, and leaders of IoT initiatives need to be proactive in identifying gaps and weaknesses early in their initiatives.
Many of the same data management infrastructure tools and technologies applied to more-traditional use cases can be leveraged in some fashion to support IoT
While more than one-third of companies in a recent Gartner survey said they are using or are planning to use new, separate data management capabilities to support IoT, 61% of those surveyed expect to leverage and expand an existing data management infrastructure. This is likely because many of the same data management infrastructure tools and technologies applied to more-traditional use cases can be leveraged in some fashion to support IoT.
“However, data and analytics leaders should evaluate the suitability of existing capabilities for dealing with the scale and distribution requirements of the specific IoT deployment and the unique governance issues of IoT data by assessing whether those technologies can deliver to the level needed,” says Friedman.
Access complimentary research and webinars: Visit the Gartner Data & Analytic Hub
Modernize database infrastructure
IoT solutions and the data they generate represent a significant shift in the requirements for storing and managing data. IoT is driving a significant shift toward Hadoop and nonrelational forms of data persistence that enable high-speed and high-volume data and event stream ingestion and storage, with greater flexibility and cost efficiency.
Despite the new requirements created by IoT, relational database management system (DBMS) technology still has a role to play as its functionality evolves, and there is overlap of capabilities with nonrelational technology. Depending on solution requirements, some organizations are leveraging existing DBMS investments to support IoT.
“Data and analytics leaders should assess the variety of data used in their IoT architecture to refine data persistence requirements and identify specific DBMS utilization and modernization plans,” says Friedman.
“It’s also important to evaluate technologies with strength in capturing streaming, time-series and unstructured data, as well as those supported via the elastic scalability of cloud, because these requirements will be common for future IoT use cases.”
IoT raises security risks
Data governance is a priority for more organizations as data becomes central to business models in all industries. The complexity and distribution of IoT solution architecture present a larger and more attractive “attack surface,” which increases security risks. In addition, IoT solutions are generating, collecting, analyzing and applying increasingly voluminous, detailed and highly valuable data that presents risk for the enterprise if not handled appropriately from a privacy, retention and quality perspective.
Gartner’s survey showed that security is the most significant data governance challenge for those organizations planning and implementing IoT solutions.
“The event-oriented nature of data in many IoT deployments — constant streams of the same sensor readings, for example — means that typical policies for data retention (generally keeping everything) are less effective, and a different philosophy is required on what data to keep and what data to throw away,” says Friedman.