Don't Let OpenStack Hype Distort Your Selection of a Cloud Management Platform in 2012

Impacts

• Hype around open-source cloud management platforms (CMPs) is causing some customers to make unfounded assumptions that may lead to poor sourcing decisions when they are choosing a CMP to build a private cloud, or when buying cloud infrastructure as a service (IaaS) from a service provider.
• Vendor marketing is leading IT managers to believe that OpenStack is a stable, mature platform ready for widespread adoption, when it is an early-stage project with code stability challenges and a minimalistic feature set.
• IT managers may confuse adoption of an API with adoption of a CMP that supports the API, causing them to narrow their technology platform choices to their detriment.
• OpenStack proponents assert that it will eventually displace full-featured CMPs (notably VMware's vCloud suite) in the enterprise, but at present it is only implementing basic CMP capabilities.
• OpenStack may eventually become a solid open-source core at the heart of a large and successful ecosystem of commercial offerings.

Recommendations

• Don't assume that "open source" equates to open standards, broad interoperability and freedom from commercial interests. In reality, OpenStack is dominated by vendor interests, where they want customers to adopt their own offerings, potentially to include proprietary lock-in.
• Competitively assess OpenStack like you would any other solution from a vendor, keeping in mind that in its current state, it is most suited to early adopters with substantial engineering resources, high risk tolerance and a need for high-scale, low-cost cloud infrastructure.
• If interoperability is a concern, use a third-party cloud API library or cloud management tool that supports multiple cloud APIs, CMPs, or service providers.
• Do not plan the future of your data center with the assumption that OpenStack will be at its core. While it is a promising project, that is no guarantee of future success.

OpenStack is an open-source CMP which provides some of the basic capabilities necessary to build a cloud IaaS offering, including virtual infrastructure management and self-service provisioning (see "Technology Overview for Open-Source Cloud Management Platforms"). It is designed as a loose framework and is divided into multiple components where users can choose to adopt some components, but not others.

CMP = cloud management platforms

Source: Gartner (September 2012)

The initial and most important components are the compute service, Nova (similar to Amazon EC2) and the object-based storage service, Swift (similar to Amazon S3). Complementary, loosely-coupled components have been added over time, including Quantum (network controller), Keystone (identity and access management), Glance (VM image catalog) and Horizon (self-service portal).

OpenStack's source code is available under an Apache 2.0 license, and there are also commercial distributions available. There are milestone releases every six months; each release is given an alphabetical code word, starting with "Austin" in October 2010, with "Folsom" anticipated for September 2012 (see Table 1).

Source: Gartner (September 2012)

Swift originated as the software behind Rackspace's Cloud Files offering. It is the most stable and widely-adopted part of OpenStack, with a successful operational track record at scale, commonly used by cloud IaaS providers and has become the most common implementation of object-based, S3-style (but not S3-compatible) storage.

Nova originated at NASA, as a Python-based rewrite of Eucalyptus, an open-source Java EE-based CMP compatible with the Amazon Web Services (AWS) EC2 API. At the same time as NASA was developing Nova, Rackspace (a provider of managed hosting and cloud services) was working on a successor to the technology used for its Cloud Servers offering, which had been obtained via the 2008 acquisition of Slicehost. The two organizations decided to combine their efforts into an open-source project (see Note 1). Significant work needed to be done to merge their early efforts, resulting in a new and unstable code base.

As a result, Nova is without significant production operational history at scale. At present, OpenStack early adopters are organizations with strong technical skills and a high risk tolerance. They are typically service providers, financial services companies, or research institutions. Most either engage consultants for assistance, or commit significant technical resources to the project.

Two key service providers are in the process of adopting Nova at scale. Rackspace's Nova-based Cloud Servers were released into general availability (GA) in August 2012. New customers will be on Nova and existing customers can choose to launch new VMs on Nova, but this will simply result in a gradual ramp-up of their Nova-based cloud. Rackspace will operate the two platforms simultaneously until it can smoothly migrate existing customers to the new platform, but this means that it does not instantly achieve scale on Nova. HP has a Swift-based offering in GA, but its Nova offering is in beta and will not be GA until late 2012, so it has also not achieved scale.

The OpenStack project has historically been dominated by Rackspace, but will be moving to a foundation, with voting rights predominantly controlled through vendor sponsorships. Participating vendors hope that pooling their efforts will rapidly enhance the technical capabilities of OpenStack. Their goal is to provide an alternative to AWS and the VMware vCloud stack — the participating vendors feel that Amazon and VMware are much greater threats than each other.

Gartner has been fielding an increasing number of inquiry calls from IT managers, technical professionals, vendors, service providers and investors wanting to learn about OpenStack. We have found that the hype around OpenStack has led to a set of dangerous myths that are affecting sourcing, partnership and investment decisions, including the arbitrary selection of OpenStack without considering alternatives or doing a technical evaluation. This research examines the impact of the hype and provides recommendations that are based in the practical realities of OpenStack as it exists today.

Hype around open-source CMPs is causing some customers to make unfounded assumptions that may lead to poor sourcing decisions when they are choosing a CMP to build a private cloud, or when buying cloud IaaS from a service provider

Some people have been led to believe that because OpenStack is open source, it is an open and widely-adopted standard, with broad interoperability and freedom from commercial interests.

In reality, OpenStack is dominated by commercial interests, as it is a business strategy for the vendors involved, not the effort of a community of altruistic individual contributors. Some of the participants, notably Rackspace and other service providers are afraid of the growing dominance of AWS in the cloud IaaS market and do not believe that they have the ability to muster, on their own, the engineering resources necessary to successfully compete with AWS at scale, nor do they want to pay an ongoing license fee for a commercial CMP like VMware's vCloud stack.

Both Rackspace and HP believe that OpenStack will enable them to offer hybrid public/private cloud solutions, if they can drive OpenStack penetration for on-premises enterprise clouds. Others, like Piston Cloud Computing, Nebula, and Cloudscaling, provide commercial distributions of OpenStack, along with professional services and support. Many are vendors, such as Cisco, Citrix (see Note 2), Dell and Red Hat, that want to ensure OpenStack works well with their products, as well as limiting VMware's future market power.

OpenStack has drawn a large vendor community, which makes source code contributions, but many exclusively contribute vendor-specific code, such as drivers for their own hardware and resist the creation of interoperability inducing common frameworks that would facilitate switching between vendors. Vendor interests also heavily affect the determination of what features to add and how to implement them, since every vendor wants each feature to work best with its own products.

OpenStack is not an open standard, as it does not provide for either interoperability or portability. It does have an open community and anyone is welcome to contribute to discussions and to contribute code. The technical leadership is elected by contributors, although most contributors are employees of the sponsoring vendors and in practice, the technical leadership has come from vendor employees. Voting rights for its proposed foundation's governing board of directors are largely controlled via vendor sponsorships. The open community process, in combination with closed-room discussions between the sponsors, sometimes results in intense infighting between the participants in the community, particularly as Rackspace works to establish the OpenStack foundation.

There are interoperability issues between OpenStack versions, due to the nascent, constantly-changing nature of the project, although this is lessening as the project matures and compatibility becomes a community priority. There are also interoperability issues between distributions, especially since commercial distributions of OpenStack typically contain proprietary extensions, which fragment commonality.

Additionally, as OpenStack is essentially a framework for components and not a product, some distributions even have completely different components than the ones in the OpenStack core (for example, the use of Ceph, another open-source project, rather than Swift for storage) or may not include certain components and associated functionality at all. These differences between versions and distributions extend to the API, making it more difficult for third-party libraries and tools to reliably support OpenStack. This lack of interoperability, even within OpenStack, will also limit the future potential for fully open, hybrid public-private or federated OpenStack clouds; instead, hybrid or federated clouds will likely require using specific service providers and their specific supported distributions.

Since the vendors involved want to drive business through the OpenStack ecosystem, there is considerable incentive for proprietary lock-in. There is no significant difference in lock in for customers adopting OpenStack than those customers adopting proprietary CMPs. In fact, as OpenStack is not a widely-adopted standard, the number of solution providers is highly limited (whether in hardware, software, or services), restricting overall customer choice far more than adopting VMware's vCloud. Nor are there multiple implementations of an OpenStack CMP. On a practical level, the AWS API has the broadest ecosystem, including multiple CMPs (both open source and proprietary) offering compatible implementations.

Recommendations:

• Assess OpenStack with no less scrutiny than you would give any proprietary commercial offering.
• Do not treat OpenStack as the preferred standard when specifying API and CMP platform compatibility in the context of sourcing technology or services, unless you have business-partnership or technical reasons to do so.
• Government entities should be particularly careful not to mistake OpenStack for a commonly adopted open standard. This is especially important when developing sourcing requirements. As there is only a single OpenStack CMP implementation, few OpenStack service providers and more limited ecosystem choices (in hardware, management software, tools and libraries), due to its lack of significant market share, OpenStack currently restricts choice, rather than increasing it.

Vendor marketing is leading IT managers to believe that OpenStack is a stable, mature platform ready for widespread adoption, when it is an early-stage project with code stability challenges and a minimalistic feature set

Swift storage has a solid track record and code-base longevity, but the rest of OpenStack is very new. OpenStack has historically had challenges with code stability, although recent changes made in the development process should improve the stability of future releases. However, it continues to have difficulties with project management and scope management, attaching risk to the timely delivery of planned features and overall project progression.

Quality assurance (QA) has improved over time, but it remains a challenge. Vendors and service providers have developed better QA and testing tools, but are not contributing them to the open-source project, because they are competitive advantages for building service offerings and improving time-to-market for distributions and related software.

Vendors involved in OpenStack typically anticipate that it will take until at least late 2013 before it reaches a level of stability and maintainability comparable to typical commercial software sold to the enterprise. Furthermore, after each new major release, it takes several months for the code to reach stability again, as bug fixes are applied against all the new code that has been introduced with the release. The projected time frame for OpenStack stability is constantly moving. Developers typically estimate that it will be achieved in another 18 to 24 months, but they have been estimating this for several releases now and there is no specific project plan to allow accurate projection of a time frame for stability. Commercial software normally reaches stability by this age and the projected time frame is exceptionally long, even for complex open-source projects.

The constant changes within the project also make it difficult for individual organizations — whether customer organizations, or vendors in the OpenStack ecosystem — to build and maintain extensions to the open-source code. For instance, one of the advantages of using open source is that you can implement any missing features yourself, but if you implement the feature in a way that is incompatible with the next version, you will have to modify and possibly entirely rewrite your code. Due to OpenStack's present minimal functionality, many organizations have to extend OpenStack to get it to meet their business needs, so this is a serious concern and it limits OpenStack's utility vs. competing open-source CMPs.

The overall project difficulties are causing many vendors to re-evaluate their OpenStack-related strategies. Vendors, as well as OpenStack customers, often say very different things in public about OpenStack than they do in private. The dissonance between public and private statements stems from customers' desire to associate themselves with a project hyped as the future of the cloud. However, their reservations about the project's ability to produce a stable product within a commercially meaningful time frame run very deep and consequently, many vendors are unwilling to fully commit significant engineering resources to the project at present.

Vendor desire to achieve competitive advantage over others in the OpenStack ecosystem has led to their reluctance to commit operations and maintenance-related capabilities to the open-source core. For example, the core contains neither an installer nor an update/upgrade mechanism, as both of these critical capabilities are left to the commercial distributions. Upgrades require a re-install and "forklift" move of workloads. As the project is in its early stages, new versions can be drastically different, breaking compatibility and requiring extensive pre-migration preparation and testing. Many adopting OpenStack early on are still on old versions.

Service providers are interested in building differentiating features and vendors are interested in differentiated proprietary distributions, or even selling additional software that complements OpenStack (including commercial CMPs operating at higher levels of the CMP stack). They have a careful balancing act between contributing enough into OpenStack to make it successful, while still withholding enough to ensure their own success.

Recommendations:

• If you are considering OpenStack, conduct a careful technical assessment, determine whether you have the technical skills to deploy and operate it and ensure that you consider your total cost of ownership of the solution — open source is not always less expensive.
• Consider proprietary CMPs as well as free and commercial distributions of OpenStack and other open-source CMPs, particularly Eucalyptus (which has officially licensed the AWS API from Amazon and is relatively easy to deploy) and CloudStack (which is a common choice for service providers and others wanting to run an AWS-compatible cloud at scale).

IT managers may confuse adoption of an API with adoption of a CMP that supports the API, causing them to narrow their technology platform choices to their detriment

OpenStack's original API was the AWS API, but it is transitioning over to its own native API and continued AWS API support is controversial in its community. AWS competitors, notably Rackspace, are concerned that it strengthens AWS's ecosystem, so AWS compatibility may eventually be removed from the core project. There is currently only modest support for third-party tools for the OpenStack native API.

Fortunately, since cloud APIs are relatively straightforward, it's a trivial step to build connectors that translate one API into another. Regardless of whether you believe OpenStack will eventually be the most popular API, you do not need to use the OpenStack CMP to use the OpenStack API; you can simply use a connector.

Recommendations:

• Keep your options open. Instead of writing directly to a cloud IaaS API, use a third-party, multi-cloud library such as libcloud or jclouds. Consider using a cloud management tool with multi-cloud capabilities, such as enStratus, RightScale, or ServiceMesh. Keep in mind that the capabilities of cloud providers vary widely, so anything that works with multiple clouds can only provide "lowest common denominator" cross-cloud functionality.
• Choose a CMP that can easily support the use of multiple cloud API connectors, so that you can easily plug in libraries and tools that work with other APIs.

OpenStack proponents assert that it will eventually displace full-featured CMPs (notably VMware's vCloud suite) in the enterprise, but at present it is only implementing basic CMP capabilities

OpenStack proponents, notably Rackspace and HP, have asserted that OpenStack will eventually become the CMP of choice for private clouds, including enterprises, as well as service provider clouds (whether public and private). However, OpenStack's minimalistic feature set makes it unlikely to displace more full-featured solutions. OpenStack is mostly a resource manager and it's possible that it may become a common component at this layer, but CMP functionality — and value creation — exists primarily above this layer (see "How to Build an Enterprise Cloud Service Architecture").

OpenStack may add additional components over time that provide improved higher-level functionality, but they will probably not to be as fully-featured as their proprietary counterparts, as is usual for open-source solutions in the rest of the IT operations management tools market. Each of the vendors involved in either OpenStack services or commercial distributions have a different go-to-market strategy, although in some cases, they may eventually want to offer full-featured commercial CMPs that use OpenStack at the resource management tier.

It is unlikely that OpenStack will displace VMware's vCloud suite or Microsoft's System Center, or competing products such as BMC Cloud Lifecycle Management, over the next five years. OpenStack is, in theory, hypervisor-neutral, but in reality it supports KVM or Xen. It' support for VMware's vSphere and Microsoft's Hyper-V is not usefully functional for enterprise deployments. It is also unlikely that OpenStack will lead to the displacement of vSphere or Hyper-V, in favor of Xen or KVM, in the enterprise.

OpenStack will face a very difficult battle against VMware, which already has a significant installed base, channel to market and suite of products that provides far more than resource management. Microsoft is also rapidly growing its market share, thanks to its deep stronghold in IT organizations and its frequent role as a strategic vendor. Customers want to be able to extend their internal virtualization environments with hybrid cloud capability and usually look to their existing vendors for solutions.

It is certainly true that many businesses are seeking more economical alternatives to VMware for both the hypervisor and the CMP and that many are interested in the potential of open-source solutions to offer lower costs. However, it is currently far more common for such businesses to add Microsoft as a second vendor, than for them to adopt an open-source solution.

Our view is that over the next five years, most businesses will adopt CMPs that integrate with the tools that they already have, or which offer an easier-to-adopt and more mature solution than OpenStack. That does not mean that OpenStack will not eventually become successful, but it is unlikely to be a near-term mainstream enterprise solution.

Recommendations:

• Do not plan the future of your data center with the assumption that OpenStack will be at its core. OpenStack may be a promising project, with many vendors eager to join its marketing bandwagon, but its future success is by no means assured. Rather than counting the number of vendors joining the project, judge OpenStack's progress by its ability to deliver future releases on-time, with the planned features and minimal bugs and achieving mainstream adoption.
• To maximize deployment flexibility and interoperability to enable the potential for multi-vendor substitution, try to choose CMP solutions that allow the layers of the service (access management, service management, service optimization, resource management and the underlying resources), to be logically independent of one another.

OpenStack may eventually become a solid open-source core at the heart of a large and successful ecosystem of commercial offerings

OpenStack is a promising project for a number of reasons, which include:

• The technical design and implementation is solid and architected for true cloud-scale deployments.
• The breadth of the vendor community and OpenStack's commercialization-friendly approach has quickly made OpenStack one of the four most important cloud infrastructure ecosystems. (The other three are from Amazon, VMware and Microsoft.)
• OpenStack's approach of providing a framework for components allows for innovative proposed developments to go through an incubation stage. Additionally, the open-source approach provides the ability to drive innovation in a decentralized and cooperative manner.

We believe that there are three key ways in which OpenStack is likely to succeed over the long term:

• The open-source core will evolve into a successful solution for those looking for a basic, low-cost CMP.
• Successful commercial products and services will be built around the open-source core, with vendors adding proprietary extensions and capabilities.
• Cloud IaaS providers will use the open-source core, in conjunction with capabilities they've built themselves (or sourced from commercial vendors), to power their offerings at scale.

OpenStack is in the process of being transferred from Rackspace to the OpenStack Foundation. Once the foundation takes the reins of governance, it should begin to answer critical questions for the maturation of the project, including determining what OpenStack really is (for example what can be branded "OpenStack" and in what context).

This may potentially resolve some of the interoperability and fragmentation issues, as well as providing overall stronger project governance and management and will be crucial for moving OpenStack beyond a niche, early-adopter solution, to something that is suitable for mainstream enterprise adoption.

Recommendations:

If you adopt OpenStack:

• Involve yourself in the community, which needs stronger representation of operators and users.
• Contribute your own bug fixes and improvements back to the open-source core, if your organization permits it.