Research from Gartner

Top 10 Strategic Technology Trends for 2017: Digital Technology Platforms

Platforms are evolving to include customer experience and digital business ecosystems built on expanded IoT and intelligence services. Enterprise architecture and technology innovation leaders should guide platform extension to support foundational and vanguard architectural needs.

Key Findings

• A digital business is supported by technology platforms that support distinct constituents' employees, partners, customers and things, as well as the intelligence to serve them all.
• Containers, serverless computing, event processing and API management are critical foundation services to support delivery of a digital business ecosystem platform.
• Internet of Things (IoT) platforms are evolving rapidly, but no single IoT platform today can meet the needs of advanced IoT business solutions. Enterprises will assemble their own IoT platforms.
• Conversational artificial intelligence (AI)-rich platforms will emerge over the next three years as the primary battleground between technology providers to deliver AI-enhanced customer experiences.

Recommendations

Enterprise architecture (EA) and technology innovation leaders using EA to master emerging and strategic trends must:

• Examine the use of both containers and functional platform-as-a-service (PaaS) approaches as part of a flexible mesh app and service architecture design. Use containers in production if you are prepared for the disruption containers bring, and use fPaaS for tightly scoped, short-lived processes.
• Assemble an IoT platform that balances the needs to minimize complexity, maximize utility and ensure appropriate security. Create reference models that recognize the need to integrate the IoT platform with other back-end systems, data and analytics, and provide guidance on how to do so.
• Use conversational, AI-based capabilities to make sense of algorithms and APIs, and to enhance marketing and ecosystem development. Then, establish separate user experience and AI platforms as these areas mature.

Analysis

Why Digital Technology Platforms Is a Top 10 Trend

Digital business is the creation of new business designs and models by blurring the boundaries between the digital and physical worlds. It involves the interaction of people, business and things. Supporting intelligent digital business and creating digital business ecosystems require a new set of platform services that will evolve over the next five years. These digital business platform services will extend beyond existing IT systems with additional platform services that:

• Embrace the IoT
• Deliver advanced customer experiences
• Support the delivery of business ecosystems
• Establish new machine learning and AI services to deliver intelligent apps and things

CIOs and CTOs, supported by enterprise architects and technology innovation leaders, must establish a technology foundation to support digital business and digital business ecosystems (see Note 1). This digital business technology platform has five key aspects (see Figure 1).

Figure 1. The New Digital Platform

Source: Gartner (March 2017)

About Platforms

In this research, "platform" means that the area is built on services-based principles and architecture. The goal is to create an interoperable set of services that can be brought together to produce applications, apps and workflows. This generates a symbiotic collection of technology capabilities and components that form a platform. A service-first versus application-first mindset is one of the main attributes of a loosely coupled, interoperable platform – think of building blocks (services) that can be easily rearranged to meet any need. The openness and composite nature of a platform is ideally suited to the external-facing capabilities required by new digital business processes, moments and models. The platforms described in this research are not typically purchased from a technology vendor as a single unit.

Additional platform services will coalesce to modify and extend the platform across all five areas. Three specific targets for expanded platform services are driving digital technology platform as a top 10 strategic technology trend for 2017:

• Foundational Services. A common set of platform services that support cloud-native application design and mesh app and service architecture will underpin all five digital platform segments. These foundational services are key enablers for digital business ecosystems.
• IoT Platforms. Gartner estimates that, by 2020, there will be more than 21 billion interconnected devices,¹ 215 trillion stable connections and 63 million new connections emerging every second. IoT platforms enable businesses to extract their value. Establishing an IoT platform for multiple IoT initiatives will be an imperative for asset-based companies over the next three years (see Note 2).
• User Experience and AI Platforms. Additional platform services that extend, augment or replace traditional IT platform services to address new human interface models powered by AI and advanced machine learning will emerge over the next three years. These services are beginning as tightly coupled elements of a conversational application platform but will evolve to become semiautonomous user experience (UX) and AI service platforms within three to five years. Solution-specific conversational AI-powered platforms deployed in 2017 to 2019 will give way to cross-application UX and AI service platforms between 2020 and 2022.

Where Digital Technology Platforms Fits in the Top 10

Digital technology platforms are part of the mesh theme, which emphasizes the growing set of connections between people, devices, things, apps, services and content. Digital technology platforms provide critical services that support this mesh of connections and the intelligent digital business solutions delivered through the mesh (see Figure 2).

Figure 2. Where Digital Technology Platforms Fits in the Top 10 List of Strategic Technology Trends

Source: Gartner (March 2017)

Digital technology platforms provide the foundational technology services to build apps and things powered by AI and machine learning. These foundational services also:

• Underpin new experiences, such as augmented reality (AR)/virtual reality (VR) and conversational systems
• Support delivery of IoT and digital-twin solutions
• Provide the underlying application infrastructure for incorporating blockchains and distributed ledgers

MASA is built on digital platform services and drives digital platforms to incorporate new IT services to support event-driven solutions and fine-grained microservice architectures. Security must be an integrated element of digital technology platforms.

Foundational Services Underpin the Digital Technology Platform

IT system platforms include the traditional infrastructure and platform services used to run core IT systems. Elements include collaboration and workplace applications, back-office applications (such as HR and ERP), specialized industry applications (for example, industrial supply chain and core banking), endpoint computing and operational technologies. They also include the core infrastructure, operations, data management, security, integration, and application development and management capabilities used to build, deploy and manage off-the-shelf or custom-developed solutions. Many of these are mature technologies and will remain in place, but others will require modernization, augmentation or replacement with new technologies to support the evolving demands of digital business and its ecosystems.

In "Top 10 Strategic Technology Trends for 2017: Mesh App and Service Architecture," we described the architecture needed to deliver dynamic and flexible solutions. Technology innovation leaders are usually the first to use advanced services for Mode 2-style solutions. Eventually, the Mode 1 counterparts will want to use these services too. Technology innovation leaders should work with their enterprise solution architects to devise common standards sooner rather than later.

MASA-based applications are collections of components that are "glued" together with APIs and events, so managing APIs and events is critical to the success of these applications. APIs and events will link services together into an app, dynamically assembling services for customer-facing conversational systems and opening up services as a foundation of the digital ecosystem. The ecosystem foundation determines how a business interacts as an entity in the digital world with customers, partners, suppliers and even competitors. Top-performing organizations that participate in a digital ecosystem expect their average number of digital partners to double in the next two years.² APIs are vital because they make these connections possible. We expect that 70% of organizations will invest in tools to manage APIs. APIs implement business policies in the digital world, and enable platforms to connect to other platforms.

Many vendors have been adding new capabilities as software on top of infrastructure as a service (IaaS; aka IaaS+) or cloud platform services, but new abstractions for compute above virtual machines have been slow in coming. This has perpetuated a traditional server-centric application pattern and has not encouraged more dynamic and flexible MASA approaches. With the emergence of lightweight virtualization in the form of Docker images and containers for deployment agility and workload portability, this is changing. Serverless computing and function platform as a service (fPaaS) provide an even higher level of abstraction that fully relieves the developer of the need to manage infrastructure (see Figure 3).

Figure 3. Developer Concerns Shrink With Functions

Source: Gartner (March 2017)

Virtual machines (VMs) provide system administrators agility by abstracting the hardware away from the operating system. Containers provide further agility by abstracting your application runtime away from the operating system. Functions provide even more agility by abstracting the application logic away from the application runtime. Each of these abstractions decreases the size of the scale unit and deployment unit, which enables teams to narrow the developer's scope of concerns. However, the higher abstraction layers come with a cost in terms of flexibility and performance versus lower-level VM compute services (see Table 1 and "Emerging Technology Analysis: Serverless Computing and Function Platform as a Service").

Table 1. Characteristics of VM, Container and Function Cloud Services

AMI = Amazon machine image; OVF = open virtualization format
Source: Gartner (March 2017)

Beyond VMs – Containers

OS containers provide isolation that enables several applications to share an OS kernel and some OS libraries, while maintaining their own copies of specific OS libraries. Containers have been used for many years to increase the density of lightly used workloads, but the simplicity in packaging, shipping and ease of use of container management frameworks such as Docker have made the technology highly appealing. Container adoption has rapidly expanded within enterprise IT in the past two years as organizations focus on creating cloud-native applications that take advantage of the native elasticity and programmability of cloud infrastructure. The growing interest in microservice architecture and container affinity to this model also drives interest. Gartner has seen a 300% increase in container-related inquiries in 2016 and estimates adoption of containers for production workloads was under 10% at the end of 2015, growing to as much as 25% by YE16. Moreover, the formation of Open Container Initiative (OCI), which has member support from a large ecosystem of vendors to standardize runtime environments and image formats, has increased customer confidence due to reduced vendor lock-in and broader platform interoperability.

Lack of standards, limited container management and orchestration tools, and limited application development tools have limited users' shift beyond virtual machines as the main virtualization tool for infrastructure. However, cloud computing and the agility requirement for MASA-based applications are driving new interest in container technology. Meanwhile, vendors are moving to deliver more tools to support a containerized application environment (see Figure 4). OS containers hold the potential to significantly improve resource use and density. However, in most cases, OS containers will not replace hypervisor-based virtualization. Instead, they will offer a complementary virtualization option. EA and technology innovation practitioners, as well as application architects and infrastructure architects, need to factor containers into their technology roadmaps.

Figure 4. Container Ecosystem

Source: Gartner (March 2017)

Containers provide increased agility, resource utilization efficiency and application portability, but come at the cost of increased complexity. In regulated industries, implementing containers brings additional challenges due to immature embedded controls. Although running containers in production isn't impossible, it requires a steep learning curve as well as a willingness to commit significant time, money and resources. However, the pace of innovation is rapid in this technology segment, and that can hasten adoption cycles and improve overall technology maturity.

Recognize that overhype, open-source community fragmentation and ecosystem machinations between providers will create challenges, but that investing now can pay off. Containers support cloud adoption by providing a consistent deployment approach and runtime execution parity across on-premises and cloud servicers. Start by augmenting traditional virtualization with containers managed by Docker for development/test and CI automation. Prototype container-based architecture changes and continuous deployment (CD) in parallel. Deploy Docker in production if you are prepared for the disruption containers bring. Welcome this disruption, as it moves you closer to your DevOps, microservice architecture and hybrid cloud goals.

Beyond Containers – Serverless Computing and fPaaS

Imagine a service that allows you to simply submit code, and the service does the rest. You choose the programming language, runtime and framework, and the platform does the rest. There is no need to spend time "architecting" the solution, as your sole focus is to create the business logic and deliver business value. In this world, there is no need to configure autoscaling groups, maintain an in-memory cache, choose instance sizes, worry about input/output operations per second (IOPS), architect for resiliency, create load balancers, or put in business continuity measures for high availability (HA) and disaster recovery (DR). In this world, the platform delivers on your applications' nonfunctional requirements including scalability, availability, reliability and maintainability. The platform is intelligent in how it manages the underlying compute resources to meet the demands of your application. Whether there are 10 users or 10,000 users, the platform adjusts to meet the demand. Whether you are experiencing heavy reads, heavy writes or random IO, the platform configures itself to meet the demands regardless of when or where they occur in your application.

In the emerging serverless computing model, application logic executes in an environment with no visible underlying platform technology, container engine or container orchestration technology. This is a further abstraction beyond containers. fPaaS consists of extremely short-lived, ephemeral programs executed on demand with little control over the underlying infrastructure. fPaaS leverages the elasticity, rapid scale and virtualized shared resources of cloud computing natively in its core design to fully optimize resources. This is a highly efficient way to consume cloud infrastructure due to its fine granularity and event-triggered execution model. Functional PaaS aligns well with event-driven and microservice architecture, and supports advanced forms of agility. These design decisions maximize the benefits of cloud computing for both the provider and the consumer.

fPaaS implementations lack complete tooling, shared libraries and best practices for streamlined, repeatable and successful software development. Most production implementations are currently found at startups and ISVs, with some leading enterprises exploring fPaaS. Despite this immaturity, the value of the model has been demonstrated and maps well to key MASA elements such as microservices and event-driven applications. Over the next three to five years, we expect to see rapid maturation and increased adoption. Start planning now for fPaaS adoption to complement VM and container-based models. If using application PaaS offerings currently, ensure these services provide a comparable set of fPaaS features, or plan to do so in the future, allowing for the same ease of use, event-triggered invocation, and fine-grained billing and polyglot support as pure-play fPaaS offerings.

Functional PaaS is not appropriate for all scenarios, and many enterprises will use it for a subset of application functionality. Using fPaaS can reduce cycle time for narrowly scoped application logic, but will introduce additional complexity and overhead in development processes and runtime governance. Use fPaaS when your requirements are tightly scoped and can run in a constrained environment, and when processes are short-lived. Avoid using fPaaS to build full-blown applications, and recognize that fPaaS encourages cloud platform lock-in, since the event-driven nature of it depends on specific kinds of events from the various services in that specific cloud context. Use an API management service when using fPaaS to manage the sprawl of functions arising from fine-grained application services. Delay adoption of fPaaS until you have mastered the fundamental cloud computing and educated teams on event-driven and microservice architecture (see Figure 5).

Figure 5. Comparing Code-Sharing Models

Source: Gartner (March 2017)

IoT Platforms Are Evolving Rapidly

By 2020, we estimate that there will be more than 20 billion connected IoT devices, and IT leaders will struggle with provisioning, managing and connecting to them; analyzing data from them; and delivering functionality to them.³ Enterprise architects and technology innovation leaders need to consider the scope and composition of potential IoT solutions for future digital business, and determine the right investments in platforms to build, deliver and manage these "things." An end-to-end IoT business solution is a heterogeneous mix of IT and operational technology (OT) assets, often including many IoT endpoints, optionally including one or more IoT gateways, and including one or more IoT platforms, all integrated with existing enterprise systems (see Figure 6).

Figure 6. Representative IoT End-to-End Business Solution

Source: Gartner (March 2017)

An IoT platform begins with the capabilities to interact with the devices (via device gateways and digital twins) and then extends into stream analytics, event processing, contextual analysis, data and application integration, decision support and management, and interaction with back-end enterprise systems. An IoT platform can be delivered as software, cloud PaaS or a hybrid mix of these. An organization's IoT architecture consists of not only information architecture, but also the user interface, application logic and rules, analytics, and execution models. These elements can exist in the endpoint, gateways, the cloud or the organization. Architectures extend from the endpoint, through different types of gateways, into the cloud and to the organization. Different architecture models support different types of IoT solutions. EA leaders must understand IoT business solution architectures, how they will design them, where they will place different components of the application and how that placement might change over time. EA leaders must also understand that IoT will involve equipment that IT traditionally does not interface with a very broad range of vendors, requiring a flexible approach to interfaces and integration, plus thinking about a much broader security threat surface. A bigger challenge will be the interenterprise relationships, as IoT projects are most often owned by the business units. EA leaders will need to enhance their connections with the business unit leaders while emphasizing the business benefits that IT and EA bring to these IoT initiatives and projects.

IoT business solutions should address a specific set of digital business objectives, which can vary greatly by industry and by type of IoT project. Most of the time, you cannot buy an off-the-shelf solution designed to fully address your specific objectives – rather, you must build it. Some organizations require a broad set of legacy data and connectivity protocols. Others have specific compliance and regulatory requirements on reports and data storage. Requirements vary by region due to regulatory, language and culture variations, as well as local privacy laws. In practice, most IoT projects start small and are fairly limited in scope – for example, a point solution involving IoT endpoint devices and an IoT platform that analyzes data collected from those devices for a targeted business issue. Over time, however, these projects often evolve into more comprehensive IoT business solutions with multiple point solutions interacting with one another. Enterprise architects and technology innovation leaders working need to build a clear vision for the complete end-to-end architecture and its constituent building blocks, which leads toward assembling a set of IoT platform services that can be leveraged across projects.

Technology providers are increasingly assembling suites of IoT services into packaged IoT platforms. Entrants to the IoT platform market are driving rapid change from specialized IoT platforms toward more comprehensive offerings. Many companies claim to have the "core IoT solution," and we expect the market to remain complex for the next three to five years. IT leaders will need to compose end-to-end solutions from multiple best-of-breed providers. EA leaders must collaborate with business managers, infrastructure and security professionals, engineers, and vendors to design a flexible, but stable, IoT architecture and platform to support IoT solutions and connect them to business solutions. By 2020, most leading IoT platforms will be part of more comprehensive PaaS suites, and we will start to see a degree of consolidation. Leading organizations with multiple IoT initiatives create IoT centers of excellence (COEs) to facilitate the cross-disciplinary collaboration required for success.

Many IT vendors – including Microsoft, Oracle, Google, Amazon and SAP – are entering the IoT platform market. Companies such as GE are also established in the market. GE's Predix is an IoT platform for industrial companies and industrial systems, with specialized security, services and other elements. Other vertical-market IoT platforms will emerge.

Eventually, best-practice IoT solutions and solution architectures will emerge for specific business processes. Examples might include:

• Driver behavior monitoring for usage-based insurance
• Smart shelves in retail establishments, which may increase sales using a combination of Bluetooth beacons and incentive marketing software
• Queue optimization for mobile assets

We expect some industry-specific anchor points will survive, and some general-purpose ones will emerge.

User Experience and AI Platforms Are Emerging

The element of the digital technology platform with the most dramatic impact on future solutions combines a shift in the customer experience to a more "conversational" human-centric paradigm, with enhanced AI services to power the new experience. These AI-enhanced conversational platforms are used to deliver conversational systems. A conversational system is an app (e.g., virtual personal agent and chatbot) or thing (e.g., smart speaker and automobile) that uses a natural-language interface to interact with the user.

Most AI-related innovation has been in consumer-grade technologies. Messaging-based applications are becoming the norm among users, particularly millennials. Uses of voice (both speech to text and text to speech) have grown significantly since 2Q14. The industry is agog, chatting about chatbots. "Conversational AI first" will supersede "cloud first, mobile first" as the most important, high-level imperative for the next 10 years, and conversational AI platforms (CAPs) will be the next big paradigm shift in information technology. CAPs are already in market today, but more are coming. CAPs will likely be the strongest instigator of investments that exploit AI for the foreseeable future. This encompasses more than chatbots, virtual assistants and messaging-based applications: The emergence of CAP will stimulate significant growth in the exploitation of AI in general.

Almost every major industry player, including Amazon, Baidu, Google, IBM, Microsoft, Oracle, Salesforce and Tencent, has either delivered or will deliver its own version of a broadly applicable, conversational, AI-rich, general-purpose platform by YE17. Some will be in support of the supplier's own applications; some will be generally available for enterprise buyers and third parties to build upon; and most will be to serve both purposes. As these broad CAPs emerge, today's providers of AI-related applications that are built on narrow obscure platforms (such as IPsoft's Amelia and x.ai's calendaring agent) will come under market pressure to be acquired by one of the broad CAP players, migrate to one or more of the broad, general-purpose CAPs, or move into narrower, more specialized markets.

While enhanced user experience is a primary driver of AI platform services and the CAP will be a dominant trend, broader use of AI to drive advanced analytics and AI-enhanced business processes will also grow. Use of AI in IoT via digital twins will accelerate this use beyond conversational interfaces. In the three-to-five-year time frame, the conversational AI rich platform service segment will likely split into two separate segments

• User Experience Services: The shift to conversational systems will be the primary driver for the user experience platform. Natural-language processing (NLP) will replace rule-based synonym and phrase substitution approaches. Dynamic natural-language ontologies or knowledge graphs at multiple levels of specificity will be needed. Services to ingest, generate, and translate verbal and written language will be part of the platform. However, the user experience platform will not be tied to singular modalities (e.g., speech, handwriting, keyboarding, visual, gestures and touch), devices (e.g., PC, smartphone, tablet, smart speaker, personal fitness wearable, dress or large-scale screen), times and places. Context is key to pervasively and intelligently serving user needs. It's important to deliver an ambient UX that blends physical and virtual environments in a continuous experience to preserve continuity across a mesh of devices (may be viewed as pernicious and threatening if it is not also contextually sensitive).
• Intelligence Services: A core set of platform services – including natural language, dialogue management and text analytics driven largely by conversational system needs – will form the basis of a general-purpose AI platform. Deep learning, image recognition and predefined models as a service will also be delivered as AI platform building blocks. Scores of other intelligence services will also be needed to populate the AI-rich platform, including, for example, sentiment analysis, personality profiling, concept relationship extraction, and other methods for inferring intent from content and context. Many vendors are speeding to market with new general-purpose, AI platforms that will host a broad range of solutions that are lower cost, lower risk, faster to deploy and easier to manage to address enterprise business needs. Some of these applications will be entirely new, while others will have been migrated from earlier, narrow or private platforms.

EA and technology innovation leaders need to recognize the inherent volatility as the AI and UX elements evolve dramatically over the next five to 10 years. There will be many platforms – some broad and others narrow. Some will provide access to their platforms through only their applications, while others will be more aggressively open to use or extension by anyone. Most will give preferential treatment to their own services, raising the costs and complexity of projects using services. App providers will fight disintermediation by traditional platform providers, such as Microsoft and Google, which want to own the user interface layer and customer experience. Although IT leaders tend to prefer a single fully populated platform for a wide range of needs, this will unlikely be feasible from a single provider's offering or ecosystem for the foreseeable future.

Evaluate the use of chatbots built on conversational, AI-rich platforms as a new integration and user experience layer to simplify and speed up user interactions with legacy production applications and systems. Do lots of real-user testing. Don't depend on your engineers or QA teams to determine if the interface responds appropriately to "the average user." These systems are still brittle. If you ask them a question outside of their programming/pattern matching capabilities, they resort to stock answers if you don't train them otherwise. "I'm not sure how to answer that question" creates a credibility gap for your app if the question seemed reasonable. Delight can turn to disdain when you interact with a conversational interface in which the programmers didn't anticipate some of the most simple and obvious questions.

Actions

EA and technology innovation leaders:

• Use fPaaS when microservices are tightly scoped, can run in a constrained environment and are short-lived. Consider containers for microservices that do not meet these conditions. Use VMs where the best performance and maximum control are needed and the payload is large (e.g., a complete application versus a miniservice).
• Build expertise in event and API management for internally facing systems as well as externally facing customer and partner systems that are part of a digital business ecosystem.
• Get started with small, internal and external projects. Externally, where appropriate, extend externally accessible APIs with conversational, AI-based chatbots.
• Plan a phased approach to IoT platforms. Consider a pilot or agile approaches to maximize IoT project potential. Focus initially on IoT platform deployment, but recognize the need to integrate the IoT platform with other back-end systems, data and analytics.
• Establish centers of excellence focused on IoT, UE and AI. This role should explore the potential business value of IoT solutions and their impact on existing IT infrastructure. Establish it as a stand-alone responsibility or an add-on discipline to an existing COE role focused on, for example, service-oriented architecture, application management or business process management.

Source: Gartner Research Note G00319582, David W. Cearley, Alfonso Velosa, Brian Burke, Mike Walker, Samantha Searle, 21 March 2017