LICENSED FOR DISTRIBUTION

Magic Quadrant for Distributed File Systems and Object Storage

Published: 17 October 2017 ID: G00318930

Analyst(s):

Summary

IT leaders are looking to deliver agile, scalable and cost-effective storage for ever-increasing amount of unstructured data. This research assesses the key attributes, vision and executional prowess of distributed file systems and object storage vendors.

Strategic Planning Assumption

By 2021, more than 80% of enterprise data will be stored in scale-out storage systems in enterprise and cloud data centers, up from 30% today.

Market Definition/Description

Storage systems based on distributed file systems and object storage are growing fast in both number and capacity, and are becoming the platform of choice to tackle the growth of unstructured data in enterprise data centers. With data growth exceeding 50% year over year, infrastructure and operations (I&O) leaders are looking for extensible on-premises storage products that can address an increasing number of digital business use cases with lower acquisition and operational costs. Enterprises are demanding scalability, automation and programmatic access, such as in cloud infrastructures with their self-healing and ease of management. Infrastructure software-defined storage (SDS), deployed on commodity hardware, is emerging as a threat to external controller-based (ECB) storage arrays in environments with a steep growth of unstructured data. New and established storage vendors are continuing to develop scalable storage clustered file systems and object storage products to address cost and scalability limitations in traditional, scale-up storage environments.

Gartner defines distributed file systems and object storage as software and hardware solutions that are based on "shared nothing architecture" and support object and/or scale-out file technology to address requirements for unstructured data growth. A shared nothing architecture is a distributed computing architecture in which each node is independent and self-sufficient, and there is no single point of contention across the system. More specifically, none of the nodes share memory or disk storage. People typically contrast shared nothing with systems that keep a large amount of centrally stored state information, whether in a database, an application server or any other similar single point of contention.

  • Distributed file system storage uses a single parallel file system to cluster multiple storage nodes together, presenting a single namespace and a storage pool to provide high-bandwidth data access for multiple hosts in parallel. Data is distributed over multiple nodes in the cluster to deliver data availability and resilience in a self-healing manner, and to provide high throughput and scale capacity linearly.

  • Object storage refers to devices and software that house data in structures called "objects," and serve clients data via RESTful HTTP APIs, such as Amazon Simple Storage Service (S3) and OpenStack Swift.

Magic Quadrant

Figure 1. Magic Quadrant for Distributed File Systems and Object Storage
Research image courtesy of Gartner, Inc.

Source: Gartner (October 2017)

Vendor Strengths and Cautions

Caringo

Company info: Caringo is a privately held company, founded in 2005, based in Austin, Texas. The company sells object storage and file archiving software. The company's last round of funding was nearly $9 million raised from venture capitalists.

Product description and enhancements over past 12 months: Caringo develops Swarm, an object storage product. Recent updates to Swarm include a revamped user interface, quotas, metadata enhancements and the introduction of Swarm appliances.

Delivery model: Swarm is most often deployed using server hardware from Cisco, Dell and Supermicro. Since Swarm is primarily software, it can also be deployed as a virtual machine instance.

Pricing model: The vast majority of Swarm's customers purchase the product in a perpetual license model, though the company also offers an annual subscription option.

Popular use cases: Caringo has a history of being among the first generation of object storage vendors, which often focused on a Mode 1 style of operation that supported governance and compliance use cases. Nevertheless, the company is attempting to expand its focus to include Mode 2 style of operations that support workloads in the media and entertainment and cloud storage use cases.

Strengths
  • Caringo has released multiple generations of object storage products with a regular cadence of software releases over a long period of time.

  • Caringo customers frequently make repeat purchases of Swarm, indicating an overall satisfaction and confidence with the product.

  • Reference customers frequently mention a positive experience and receive prompt attention when issues arise that require Caringo's support.

Cautions
  • Gartner clients do not frequently shortlist Caringo's object storage product for large, petabyte-scale deployments. Additionally, the average Caringo deployment is small relative to the vendors in this Magic Quadrant and in the overall market, in terms of both capacity deployed and number of sites.

  • Caringo continues to trail the market in terms of product innovation. Caringo is not among the first to release category-defining features, but typically follows the market leaders.

  • Caringo's ability to serve large enterprise clients with complex requirements is constrained by the company's very small field sales staff compared to its object storage competitors.

Cloudian

Company info: Cloudian is a privately held company founded in 2011. The company raised $41 million in its most recent funding round in October 2016.

Product description and enhancements over past 12 months: Cloudian develops HyperStore, an object storage product that uses Apache Cassandra. Notable recent enhancements to HyperStore include erasure coding across sites, node management gains and improvements to its S3 API.

Delivery model: HyperStore is sold as both an appliance and as software that can be deployed in a virtual machine. It is available as a preintegrated appliance through a relationship with Lenovo and offers compatibility with servers from Dell, Cisco, Quanta and Supermicro. HyperStore can also be purchased and billed through the Amazon Web Services (AWS) Marketplace.

Pricing model: Cloudian offers HyperStore in both perpetual and subscription pricing models.

Popular use cases: The earliest target segment of HyperStore was service providers offering storage as a service. Today, HyperStore is frequently deployed as a target for enterprise backup solutions such as Commvault and Rubrik. HyperStore is also used as an archive for rich media and video surveillance.

Strengths
  • Cloudian has grown its overall number of petabyte-scale customers since initially being evaluated in Gartner's Critical Capabilities for Object Storage, published in March 2016.

  • Reference customers frequently cite Cloudian's close compatibility with the Amazon S3 API as being a significant factor in their decision to purchase HyperStore, and many customers indicated an overall satisfaction with the product after deployment.

  • HyperStore appliances provide robust, modular, high-density and resilient hardware options for a wide range of capacities.

Cautions
  • Most Cloudian customers do not use HyperStore for application development. Customers frequently require a product from a third-party independent software vendor (ISV), such as those focused on backup or cloud storage gateways, which increases the cost of the overall solution substantially.

  • Cloudian marketing continues to incorrectly state that HyperStore is 100% S3 API compatible. The company attempts to sell end-to-end capabilities for running analytics workloads such as Hadoop and Spark on HyperStore itself when no such turnkey solution exists.

  • According to end-user references, Cloudian's file gateway has significant stability and performance problems. Further, the file implementation is not a native, distributed part of the platform, which, when compounded with the stability issues, can result in a single point of failure.

DDN

Company info: DDN is a privately held company, founded in 1998, based in Santa Clara, California. DDN is not funded by outside investors, but rather has been solely funded by its cash flow from product sales. DDN is a mature storage supplier to the high-performance computing (HPC) market, where its Web Object Scaler (WOS) product is positioned as an archive tier for data that no longer requires high-performance primary storage. DDN is focused on large systems and the high-end market with a high-touch direct sales model, while fulfillment typically happens through DDN partners.

Product description and enhancements over past 12 months: DDN develops the WOS, an object storage product with a focus on performance. Notable recent enhancements to WOS include a new appliance model in addition to erasure coding and compliance enhancements.

Delivery model: WOS is offered in two appliance options and as a software-only distribution.

Pricing model: WOS is only offered in a traditional, perpetual license pricing model.

Popular use cases: WOS is often used as an archive tier for DDN's HPC-focused products.

Strengths
  • DDN has a heritage in the HPC market and has the resources in place to effectively serve the object storage needs of these customers with WOS.

  • Enterprises can select from a variety of WOS hardware and software deployment options that suit their respective infrastructure, cost and management preferences.

  • DDN offers seamless integration between WOS and its parallel file system appliances, GRIDScaler and EXAScaler, for handling warm and cold datasets between these environments transparently.

Cautions
  • Gartner clients have been critical of DDN's postsales service and support once WOS is operational.

  • DDN's vision for object storage trails other vendors that are innovating in the direction of public and hybrid cloud capabilities.

  • WOS prioritizes performance over security features, and lacks server-side encryption and HTTP authentication. Further, WOS's S3 API compatibility, which uses Apache Hbase, is implemented on an architecture that does not scale well beyond a single site.

Dell EMC

Company info: EMC, founded in Hopkinton, Massachusetts, in 1979, is part of privately held Dell Technologies and is referred to as Dell EMC. The company primarily sells on-premises storage hardware, software, servers and networking equipment.

Product description and enhancements over past 12 months: This Magic Quadrant, considers two products: Isilon, a distributed file system, and Elastic Cloud Storage (ECS) object storage. Notable ECS enhancements include support for Dell PowerEdge servers, metadata search, life cycle policies and compliance enhancements. Notable Isilon enhancements include support for solid-state drives, branch office support and CloudPools, which offers support for Google Cloud Storage. The Isilon F800, a new solid-state-based Isilon offering, is designed for commercial high-performance file workloads such as media and entertainment, life sciences and analytics.

Delivery model: Both Isilon and ECS are most frequently sold with Dell EMC hardware, but both can be deployed as software only. Further, both products can be acquired as private, dedicated infrastructure in Virtustream's cloud infrastructure as a service (IaaS) environment.

Pricing model: Both ECS and Isilon are most frequently sold in a traditional, perpetual model. Isilon OneFS license includes both a per-node base license and per-TB capacity, and price varies by the type of nodes. The ECS product offer flexible pricing options such as an amortized pricing model or pay as you go.

Popular use cases: ECS is primarily used as an archive for unstructured data when object storage is preferred. Isilon is primarily used to support high-throughput, file-based workloads.

Strengths
  • Isilon dwarfs all other distributed file system vendors in this Magic Quadrant in terms of revenue. Isilon alone represents nearly 50% of the total revenue in the market covered by this Magic Quadrant, reflecting the product's continued dominance.

  • ECS has a well-designed architecture, built by engineers who were responsible, in part, for building early parts of Microsoft Azure. The distributed, pragmatic designs used by public cloud IaaS providers are also found in ECS.

  • Gartner clients evaluating products for large-scale, unstructured data workloads frequently include Isilon or ECS on their shortlist due to a positive track record and support capabilities.

Cautions
  • The Isilon F800 is a nascent offering that is not designed for online transaction processing (OLTP).

  • Gartner reference customers are frequently concerned with the high cost of maintenance and support when purchasing ECS and Isilon.

  • Isilon and ECS trail the emerging market for hybrid cloud storage solutions. Isilon and ECS support very simple tiering of data to remote storage, but not the two-way synchronization that customers are beginning to deploy.

HGST

Company info: HGST, a brand of Western Digital, acquired Amplidata object storage technology in 2015. That same year it launched its product as the HGST Active Archive System, a high-density appliance that is designed for multipetabyte object storage deployments and high-throughput operations.

Product description and enhancements over past 12 months: Its current product, ActiveScale, is based on the new iteration of the hardware and software that was released to production December 2016; while Active Archive, which is based on the 4.x version, is no longer being sold to new customers. The object storage products feature all-inclusive software and grid-based storage architectures based on self-healing erasure code algorithms and are engineered to take full advantage of high-capacity HGST and Western Digital hard-disk drives (HDDs). There are two products available. ActiveScale P100 modular object storage system ranges from 720TB to 19PB. Active Scale X100 is a full-rack engineered storage appliance that ranges from 840TB to 52PB. Recently, the vendor has added a cloud-based storage analytics tool ActiveScale Cloud Management (CM), support for 10TB He HDDs, and partnered with Avere Systems and others to provide file access support to the ActiveScale platform.

Delivery model: ActiveScale is delivered as an integrated appliance system.

Pricing model: Base configurations of ActiveScale are priced inclusive of hardware and software and are priced by capacity, with cloud-based storage analytics included as well. In addition, HGST offers flexible capacity, capacity on demand and leasing options.

Popular use cases: ActiveScale is best-suited for large-scale deployments of archive, content and backup when a turnkey appliance is the preferred delivery model and no public cloud integration is required.

Strengths
  • Integration of Western Digital, HGST and SanDisk will enable the ActiveScale solution to reach a large customer base across many verticals and geographies.

  • Western Digital's deep insight into the latest underlying HDD technology results in higher efficiency and higher density of its ActiveScale products compared to the competition.

  • Deployment simplicity and installation speed of ActiveScale appliances were ranked high by end-user references.

Cautions
  • ActiveScale is a new product, just launched in 4Q16, and is structurally different from the previous generation Active Archive System, hence requiring a forklift upgrade from Active Archive product deployments.

  • ActiveScale P100 and X100 start with 720TB/840TB minimum capacity and grow in large (720TB/840TB) capacity increments, which do not suit smaller implementations with slower data growth scenarios.

  • ActiveScale lacks native file system support, thus requiring users to integrate additional partner products.

Hitachi Vantara

Company info: Hitachi Vantara, a Santa Clara, California-based subsidiary of Hitachi, first entered the object storage market with Hitachi Content Platform (HCP) in 2006 and now has more than 1,800 customers globally. HCP has evolved and expanded its object storage portfolio with integrated products like enterprise file sync and share (EFSS), cloud storage gateway and search analytics. Hitachi Vantara leverages HCP's unique custom metadata storage and query capabilities to empower integrations with other Hitachi Vantara technologies, such as Pentaho, providing core data management components to big data analytics projects and Hitachi's goals for "Social Innovation" projects.

Product description and enhancements over past 12 months: Today, Hitachi Vantara offers four products under the umbrella of object storage: Hitachi Content Platform, object storage solution; HCP Anywhere, for file synchronization and sharing and end-user data protection; Hitachi Data Ingestor (HDI), a cloud file gateway; and the recently released Hitachi Content Intelligence, providing search and analytics insights. Hitachi Vantara is reporting more than 50% of HCP users are using more than one of the portfolio products today. Over the past 12 months, HCP has released a new major version of HCP, v.8.0, which focuses on capacity, density and cost-optimization features such as geodistributed erasure coding with data reduction features, KVM hypervisor support, the latest HDD and SSD support as well as a better object recovery procedure.

Delivery model: HCP can be deployed as an engineered appliance, software only running on a virtual machine or as a managed service with back-end capacity provided by its own S series nodes or external storage area network (SAN) arrays.

Pricing model: Pricing options include capacity perpetual, capacity enterprise license agreement (ELA), financial lease and a managed service solution monthly fee based on capacity consumption.

Popular use cases: HCP is best-suited for traditional archiving solutions as well as new diverse use cases such as EFSS, public cloud onramp and data management for distributed workloads.

Strengths
  • HCP is a mature product that is deployed across a wide range of industries and geographies with strong expertise in compliance-focused archiving.

  • Hitachi Vantara's vision of HCP evolving from a stand-alone object storage solution to a platform portfolio is resonating with reference customers and prospects.

  • HCP delivers a wide range of capacity and deployment options, including service provider partners and hybrid cloud storage enablement, and it has one of the broadest ISV support ecosystems in the industry.

Cautions
  • Hitachi Vantara lacks a distributed file system product, which can be a limiting factor for end users looking for a unified solution.

  • Based on customer references, HCP's pace of product innovation and responsiveness to new feature requests and enhancements tend to be slower compared to its competitor's.

  • Some HCP end users have reported issues with the quality of Hitachi Vantara support and timeliness of troubleshooting, escalation and problem resolution.

Huawei

Company info: Huawei is a global provider of ICT solutions headquartered in Shenzhen, China. The company provides products and services for carrier, enterprise and consumer customers with a broad portfolio of storage solutions.

Product description and enhancements over past 12 months: Huawei offers multiple products in this category. The OceanStor 9000 is primarily deployed as a scale-out network-attached storage (NAS) product, while FusionStorage 6.0, which was launched less than a year ago, is positioned as a block and object storage product. Huawei's OceanStor 9000 series had minor enhancements in the past 12 months including new hardware controllers.

Delivery model: OceanStor 9000 is delivered as a hardware appliance, while FusionStorage 6.0 is delivered as software on certified Huawei servers .

Pricing model: The OceanStor 9000 is priced on a per-node basis, while the FusionStorage 6.0 software is priced on a capacity basis.

Popular use cases: For OceanStor 9000: HPC, big data analytics, home directories and video surveillance. For FusionStorage 6.0: Cloud storage, backup and archiving.

Strengths
  • Huawei continues to exhibit strong momentum in this market segment. It added hundreds of PB-scale customers and revenue grew at a triple-digit growth rate in 2016.

  • Both products from Huawei offer high scalability and resilience, supporting up to 100PB of storage on up to 288 nodes and a wide range of interfaces.

  • Huawei's efforts in enabling a common storage fabric across on-premises and public cloud environments can engender more hybrid cloud use cases, although its public cloud presence is mostly restricted to China today.

Cautions
  • Huawei's product lines have overlapping capabilities, which causes confusion among customers regarding the right product fit for the use case. Moreover, FusionStorage 6.0 is an early stage product that lacks key features such as compression and distributed erasure coding.

  • Huawei's sales in North America are small, and it continues to struggle to grow its customer and channel partner base in that continent due to geopolitical and brand perception issues.

  • Huawei provides limited flexibility in consumption models and licensing with OceanStor 9000 and FusionStorage 6.0. OceanStor 9000 is only available as an appliance with node-based a la carte pricing, while FusionStorage 6.0 has certified a limited number of third-party hardware OEMs.

IBM

Company info: IBM is a large IT vendor headquartered in Armonk, New York, that provides hardware, software and services for enterprises and midsize businesses.

Product description and enhancements over past 12 months: IBM Spectrum Scale is a scale-out file system based on General Parallel File System (GPFS). Recent enhancements to the product in the 4.2.2 software release include file set data locality for Docker, Hadoop Distributed File System (HDFS) transparency improvement and Securities and Exchange Commission (SEC) 17a-4 certification, including immutability for file. IBM Cloud Object Storage (COS) is a highly scalable and resilient object storage product that IBM acquired through Cleversafe. Recent product enhancements include Network File System (NFS) file access, support for Federal Information Processing Standard (FIPS) 140-2 Level 1 certification, and support for locked vault.

Delivery model: IBM Spectrum Scale can be deployed as software only, as a hardware appliance sold by IBM as Elastic Storage Server (ESS) or as a reference architecture by certified third-party OEMs. IBM COS System is delivered as software only, or in the form of hardware appliances and supported on a number of compatible hardware platforms from suppliers such as Cisco, Supermicro, Hewlett Packard Enterprise (HPE), Seagate, Dell and Lenovo.

Pricing model: Spectrum Scale has a new capacity-based licensing model. Elastic Storage Server (Spectrum Scale appliance) pricing is either on per disk (HDD/SSD) basis or available through node-based pricing (per socket) for customers that wish to continue on the older model. IBM Cloud Object Storage is licensed on a raw capacity basis.

Popular use cases: Use cases for IBM Spectrum Scale include commercial HPC, big data analytics, home directories, backup and active archiving. For IBM COS, they include archiving, backup and content distribution.

Strengths
  • IBM offers customers a wide choice of deployment options across both products — preintegrated appliances or software certified on a wide variety of third-party hardware OEMs. IBM's exit from the x86 server hardware business has made it more appealing as a software partner.

  • IBM Cloud Object Storage delivers competitive distributed erasure coding capabilities that provide protection against site failures, has vault-level granularity and is tightly integrated with its encryption technology.

  • IBM Spectrum Scale supports efficient cloud tiering with integrated compression and encryption. The cloud-tiering feature can be used with both Amazon S3 and IBM Bluemix, as well as with IBM Cloud Object Storage as a private storage target.

Cautions
  • There has been staff turnover in many customer-facing roles after IBM's acquisition of Cleversafe. Also, the pace of innovation for enterprise customers has slowed down, with features such as write once, read many (WORM) still missing as IBM prioritizes the product development needs of its Bluemix cloud.

  • Gartner inquiries reveal that IBM's presales and postsales support needs improvement due to its complex hand-off process and the process of coordinating resources scattered across a large organization.

  • IBM's pricing model continues to be perceived as being complex due to continuance of legacy pricing models and as having higher maintenance costs, when compared to other products in this category.

NetApp

Company info: NetApp is a publicly held company, founded in 1992, based in Sunnyvale, California. The company primarily sells on-premises storage infrastructure, but is expanding to include hybrid cloud data services.

Product description and enhancements over past 12 months: NetApp sells StorageGRID, an object storage product whose primary access method is the Amazon S3 API. Notable enhancements over the past 12 months include support for bare-metal servers, multitenancy enhancements, additional erasure coding schemes and a new user interface.

Delivery model: StorageGRID is software that is primarily deployed as part of NetApp packaged hardware, but can also be deployed in a series of Docker containers or virtual machine instances.

Pricing model: StorageGRID can be acquired as a traditional, perpetual license or in a subscription model based on capacity consumed per month.

Popular use cases: StorageGRID's roots are in the healthcare space, serving as storage infrastructure for unstructured data such as medical images. But newer iterations of StorageGRID have broadened its appeal to modern application use cases, such as applications that utilize the product's S3 API.

Strengths
  • NetApp is showing early, positive signs in its multiyear transformation from selling primarily on-premises storage hardware to selling hybrid cloud storage solutions. StorageGRID in particular employs innovative hybrid cloud storage capabilities compared to other object storage vendors.

  • NetApp has the strongest relationship, from both an engineering and go-to-market perspective, with AWS of any other vendor in this Magic Quadrant. This results in unique, jointly developed offerings supported end to end, both on-premises and in the public cloud.

  • StorageGRID is beginning to attract large, well-known brand names to its platform spanning a diverse set of industries including manufacturing, media and entertainment, and transportation.

Cautions
  • StorageGRID 10.x uses Apache Cassandra for metadata storage, but this architectural decision is unproven and questionable, particularly when juxtaposed against smaller number of customers with petabyte-scale StorageGRID deployments relative to the leaders in this market.

  • StorageGRID is a small business inside of NetApp compared to high-growth products such as SolidFire. This results in StorageGRID not having the same internal attention as the other NetApp products with which StorageGRID competes for resources.

  • StorageGRID was rated the lowest of all products in this Magic Quadrant by NetApp's reference customers in terms of satisfaction related to integration and deployment.

Qumulo

Company info: Qumulo, headquartered in Seattle, was founded in 2012 and raised $130 million in four rounds of funding. The Qumulo Core scale-out NAS product was first released on 16 March 2015. On 7 June 2017, Qumulo entered the HPE Complete reselling program.

Product description and enhancements over past 12 months: Qumulo Core is a scale-out file system product that was designed for large-scale, high-throughput file workloads that require granular performance analytics and capacity management. Qumulo core cluster can scale up from four nodes and 96TB capacity to 1,000 nodes or 6PB capacity. Over the past 12 months, Qumulo has added a lot of enterprise features such as erasure coding, real-time capacity quotas and replication, and it has expanded its capacity trends analytics reports. It now has additional hardware SKUs and certifications with HPE Gen9 hardware. Qumulo uses an agile development methodology and has delivered new features and releases on a biweekly basis. The product has successfully gained traction and rapidly grew to over 100 customers in data-intensive industries such as media and entertainment, life and earth sciences, telecommunications, automotive, manufacturing, oil and gas, and higher education.

Delivery model: Qumulo Core software is delivered as an appliance or a software-only product on precertified commodity hardware: Qumulo QC-Series and HPE Apollo 4200 Gen9 servers.

Pricing model: Qumulo is priced as a software subscription based on raw capacity of the hardware for terms from one to five years; it includes all new releases, features and software support, and is transferable across hardware.

Popular use cases: Qumulo Core is best-suited for commercial HPC, analytics, content distribution and large home directories.

Strengths
  • Qumulo real-time analytics provides end users with easy-to-use insight into clusterwide performance, capacity and trending on the granular level.

  • Qumulo's aggressive roadmap on hardware, software and public cloud integration support is resonating well with Qumulo customers and is resulting in growing numbers of prospects.

  • End users gave the highest scores for the Qumulo support team and customer-focused engagement process.

Cautions
  • Gartner clients are questioning Qumulo viability and are hesitant to make a large strategic investment with a relatively new technology provider.

  • Today Qumulo is mostly deployed in North America. The HPE Complete partner program will be required to engage with customers in other geographies.

  • Qumulo Core is lacking support for object storage S3 API, newer file protocols such as SMB3 and NFSv4, and key features such cluster node removal.

Red Hat

Company info: Red Hat is a publicly traded company headquartered in North Carolina, with a strong pedigree of open-source products across OS, middleware, cloud management, containers, platform as a service (PaaS) and storage.

Product description and enhancements over past 12 months: Red Hat Ceph Storage is an open-source software product that supports block and object storage access with file access available as a technical preview (not production-certified). Recent enhancements include better Active Directory/LDAP support, native SSL support, SigV4 support (S3), Ansible integration and tech preview of Ceph Filesystem (Ceph FS). Red Hat Gluster Storage is also an open-source product primarily sold as a scale-out NAS product in a disaggregated form factor. Recent key enhancements include support for containerized workloads across the hybrid cloud, storage tiering and metadata as well as small file performance enhancements.

Delivery model: Red Hat Ceph Storage is sold as software direct to customers, as a reference architecture or as embedded software via OEM appliance partners. Red Hat Gluster Storage is sold as a software subscription direct to customers or through reference architectures.

Pricing model: Red Hat Ceph Storage has a subscription software pricing model based on capacity. Red Hat Gluster Storage has a subscription software pricing model based on number of nodes.

Popular use cases: Both products serve a variety of Mode 1 and Mode 2 use cases. Use cases for Red Hat Ceph Storage include OpenStack private cloud storage, backup and archiving. For Red Hat Gluster Storage they include archiving, backup, container/PaaS storage, home directories and media streaming.

Strengths
  • Red Hat Ceph Storage is a versatile product that is increasingly being deployed as both a block and an object storage system, with new object storage capabilities such as multisite failover and close fidelity to the S3 API.

  • Red Hat Gluster Storage is tightly integrated with both Docker and Kubernetes, enabling data persistence and protection for containerized workloads in either hyperconverged or disaggregated form factors.

  • Both Red Hat Ceph and Gluster Storage are certified across a broad spectrum of server hardware with reference architectures available from leading server OEMs such as HPE, Cisco and Supermicro.

Cautions
  • Red Hat has chosen to have separate pricing models for Ceph and Gluster, with Ceph being licensed on a capacity basis and Gluster being licensed on a node basis. This complicates subscription management, as well as makes Red Hat Ceph Storage expensive relative to other open-source alternatives.

  • Ceph FS is still in technical preview and isn't certified to be production-ready. It lacks several key data services such as snapshots and replication.

  • The skills required to deploy and manage open-source software-defined storage products such as Ceph and Gluster are rare within the enterprise, and there is typically a steep learning curve for most storage admins.

Scality

Company info: Scality is a venture-backed privately held company that was founded in 2009, with R&D based in France and sales and marketing based in Silicon Valley. Scality has been delivering its object and scale-out file storage software-only product since 2010. Scality Ring's seventh iteration was released to general availability in July 2017.

Product description and enhancements over past 12 months: Scality Ring is a scale-out, peer-to-peer distributed shared-nothing software that can be deployed on any x86 commodity hardware. It offers file and object capabilities as well as support for OpenStack protocols. Scality Ring features an integrated virtual file system that provides file storage services without the need for external file gateways, unlike the majority of other object storage solutions. In its 7.0 release, Scality Ring has expanded enterprise capabilities for both file system (WAN georeplication, versioning and file access coordination) and S3 connector areas. Over the past 12 month, Scality has extended its focus toward developers of cloud-native applications and released an open-source S3 Server. In July 2017, Scality released Zenko — an S3 multicloud data controller and search engine, which is a strategic investment for Scality growth in the area of public cloud integration for content delivery and bursting.

Delivery model: Scality Ring is offered as software only, running on any x86 standard Linux distribution. Scality provides reference architecture with hardware OEM partners and has strategic alliances and reseller partnerships with Cisco, Dell and HPE.

Pricing model: The product is priced as a perpetual or hardware-lifetime-usable capacity-based license, with optional add-on for geodistribution, email connector and compliance connector.

Popular use cases: Scality Ring is best-suited for multipetabyte geographically distributed deployments of unstructured data for content distribution, email, video, backup, active and HPC archiving.

Strengths
  • Scality is a well-funded software company with an established customer base of multipetabyte deployments in North America and EMEA.

  • Scality customers praise technical support and responsiveness to end-user needs, which result in a high level of repeat business.

  • Scality's S3 Server and Zenko releases are resonating with reference end users looking to expand Ring to enable cloud-native and hybrid cloud capabilities with unified data management.

Cautions
  • When replacing a traditional NAS filer, end users must validate performance and features of Scality, as Ring's file system is not designed for general-purpose enterprise file share workloads.

  • Scality Ring does not have native compression or deduplication features.

  • Gartner clients are becoming concerned with Scality product pricing as large OEM storage vendors now offer high-density solutions with deep discounts.

StorageCraft (Exablox)

Company info: Exablox, a StorageCraft company, launched the OneBlox scale-out file system storage product in 2010, raised $45.5 million and has been mostly tackling the Tier 2 unstructured data market. In January 2017, Exablox was acquired by StorageCraft, the data protection company with headquarters in Draper, Utah.

Product description and enhancements over past 12 months: Exablox OneBlox was designed for unstructured storage needs and can scale up by adding disk drives or scale out by adding nodes (up to seven) to the OneBlox cluster. OneBlox distributed file system supports NFS and SMB protocols and features in-line variable-length deduplication, compression and comprehensive snapshot support. Over the past 12 month, Exablox has added file auditing, multisite many-to-many remote replication, encryption at rest, and support for all-flash hardware SKUs to expand for latency-sensitive workloads. Historically, OneBlox has been competing with secondary storage products; it is now being used as a foundation for StorageCraft's disaster recovery as a service (DRaaS) offering.

Delivery model: OneBlox is offered as a hardware appliance model with "bring your own disk drives."

Pricing model: Pricing is based on a per-appliance basis regardless of capacity. Add-on replication, encryption and management software is licensed separately. Customers must buy hardware disks to populate appliances from a hardware compatibility list.

Popular use cases: OneBlox is well-suited for backup, archiving and file repository.

Strengths
  • StorageCraft Exablox has gained adoption as an easy-to-scale and manage platform for Tier 2 file-based workloads ranging from a few TB to PB implementations.

  • Exablox has comprehensive support for data service technologies such as global in-line data deduplication, continuous snapshots, compression and encryption.

  • Exablox customers like the simplified purchasing model including the per-node (and not capacity) pricing and the ability to buy hard drives of their choice separately without a storage vendor premium.

Cautions
  • Exablox has low mind share with enterprise IT buyers beyond the backup or file repository use case.

  • The vast majority of Exablox implementations are currently in North America.

  • Following the StorageCraft acquisition, some end users are expressing concern about Exablox's future as a stand-alone storage product outside of the StorageCraft's DRaaS offering.

SUSE

Company info: SUSE, founded in 1992 and operating as a semiautonomous business unit within the Micro Focus Group since 2014, is a diversified open-source software company that is well-known as the second-largest Linux vendor.

Product description and enhancements over past 12 months: The SUSE Enterprise Storage (SES) product is based on Ceph and provides unified access for block, file and object protocols. Recent enhancements include the launch of openATTIC management tool, support for Ceph FS, multisite object replication and support for ARM64.

Delivery model: SES is delivered as software or as a reference architecture through hardware OEMs.

Pricing model: SES has a node-based subscription pricing model.

Popular use cases: SES use cases include OpenStack private cloud storage, backup and archiving.

Strengths
  • SUSE is an active contributor to the Ceph open-source community with a product that is differentiated from that of the larger, incumbent Red Hat and from the upstream open-source project.

  • SUSE has an aggressive pricing model for Ceph that is node-based with a low upfront cost and incremental costs for adding nodes.

  • SUSE's acquisition of HPE's software assets gives it a larger software portfolio and stronger foothold in OpenStack private cloud deployments, which are still the primary use case for Ceph deployments.

Cautions
  • SUSE has an aggressive software release cycle, which has resulted in certifying features that aren't considered fully production-ready by members of the Ceph community (such as Ceph FS), nor as extensively tested as its peers.

  • SES is managed through the OpenATTIC tool, which needs better dashboards for file system and object management.

  • SUSE lacks a robust hardware OEM strategy — the list of server products certified under its "Yes Certification program" is quite minimal.

SwiftStack

Company info: SwiftStack, based in California, is a software defined storage vendor and the main contributor to OpenStack Swift project. SwiftStack, has developed a product based on Swift with proprietary software functionality for better deployment and management and has been shipping the product since 2013.

Product description and enhancements over past 12 months: The SwiftStack system provides native object storage as well as a file system gateway layered on top of the object storage core. SwiftStack version 5 was launched in March 2017, with SwiftStack adding support for Cloud Sync to and from Google Cloud Platform. Also included in this release were multiregion erasure coding and more scalable containers/buckets.

Delivery model: SwiftStack delivers a software-only solution that is also available as a bundled solution from Cisco and its resellers. In addition, SwiftStack has joint reference architectures with HPE, Dell and Supermicro.

Pricing model: SwiftStack is licensed on a subscription basis based on capacity under management. It also offers an optional perpetual licensing option.

Popular use cases: Use cases include rich media editing and streaming, backup, and archive.

Strengths
  • SwiftStack leverages open-source innovation while focusing its engineering on enterprise data services and integration. Its business model makes it an attractive partner for server OEMs and protects it from the uncertainty of low-margin hardware business.

  • SwiftStack offers flexible consumption options for organizations — it is software-based with a subscription licensing model based on usable, rather than raw, capacity. It also offers a perpetual licensing model based on capacity for organizations that prefer a capital expenditure (capex)-based model.

  • SwiftStack supports bidirectional synchronization of objects with both AWS and Google, with data being retained in its native format, enabling easier data access, analysis and protection in public cloud storage.

Cautions
  • The initial setup and implementation of the product can be cumbersome due to hardware selection and performance-tuning challenges, leading to the need to use professional services for large-scale implementations.

  • While SwiftStack has joint reference architectures with a number of leading hardware OEMs, none except Cisco provides any level of support for the SwiftStack software.

  • SwiftStack has been slow to leverage the convergence of file and object access. Its file system gateway does not provide POSIX-compliant interfaces and, as a result, not all file system operations are supported.

Vendors Added and Dropped

We review and adjust our inclusion criteria for Magic Quadrants as markets change. As a result of these adjustments, the mix of vendors in any Magic Quadrant may change over time. A vendor's appearance in a Magic Quadrant one year and not the next does not necessarily indicate that we have changed our opinion of that vendor. It may be a reflection of a change in the market and, therefore, changed evaluation criteria, or of a change of focus by that vendor.

Added

Qumulo: Qumulo expanded its business to geographies beyond just North America and now qualifies for this Magic Quadrant.

HGST: ActiveScale has crossed the threshold for production customers and now qualifies for this Magic Quadrant.

StorageCraft Exablox: OneBlox product deployments are now addressing multiple use cases, which qualifies it for this Magic Quadrant.

Dropped

Panasas: For the 2017 Magic Quadrant, Gartner made a change to the inclusion criteria to only include vendors that address multiple use cases, which does not apply to Panasas ActiveStor as it addresses only commercial high-performance computing.

Inclusion and Exclusion Criteria

To qualify for inclusion, vendors must meet all of the following requirements:

  • Revenue must be more than $10 million per year for the distributed file systems and/or object storage product between 1 May 2016 and 30 April 2017 or should have at least 50 production customers each consuming more than 300TB capacity through distributed file or object storage protocols only. The vendor must provide reference materials to support this criterion.

  • The product must be installed in at least two major geographies (among North America, EMEA, Asia/Pacific and South America).

  • The product deployments must reflect multiple use cases that are outlined in the "Critical Capabilities for Object Storage" or the "Critical Capabilities for Distributed File Systems."

  • The product must be designed for primarily on-premises workloads and not as a pass-through solution where data will be permanently stored elsewhere.

  • The vendor must own the storage software intellectual property and be a product developer. If the product is built on top of open-source software, the vendor must be one of the top 10 active contributors to the community (in terms of code contribution).

Packaging:

  • The product must be delivered as either an appliance or software-based solution.

  • The product must be available for purchase as a stand-alone storage product and not an integrated or converged system with a compute and hypervisor bundle.

Product capabilities:

The product must have:

  • File and/or object access to the common namespace/file system.

  • A shared nothing architecture where data is replicated or erasure coded over the network across multiple nodes in the cluster. The product must also have the ability to handle disk, enclosure or node failures in a graceful manner without impacting availability.

  • A single file system capable of expanding beyond 500TB.

  • A global namespace capable of 1PB expansion.

  • A cluster that spans more than three nodes.

  • Support for horizontal scaling of capacity and throughput in a cluster mode or in independent node additions with a global namespace/file system.

Vendors to Watch

The following vendors did not meet the criteria for inclusion in the Magic Quadrant, but may be worthy of consideration as they are starting to get more traction in the unstructured data storage market:

  • Elastifile

  • Microsoft

  • Hedvig

  • Minio

  • Nexenta

  • Pure Storage

  • Inspur

Evaluation Criteria

Ability to Execute

We analyze the vendor's capabilities across broad business functions. Ability to Execute reflects the market conditions and, to a large degree, it is our analysis and interpretation of what we hear from the market. Gartner analysts evaluate vendors on the quality and efficacy of the processes, systems, methods or procedures that enable IT provider performance to be competitive, efficient and effective, and to positively impact revenue, retention and reputation within Gartner's view of the market.

Table 1.   Ability to Execute Evaluation Criteria

Evaluation Criteria

Weighting

Product or Service

High

Overall Viability

High

Sales Execution/Pricing

Medium

Market Responsiveness/Record

High

Marketing Execution

Low

Customer Experience

High

Operations

Low

Source: Gartner (October 2017)

Completeness of Vision

Completeness of Vision distills a vendor's view of the future, the direction of the market and the vendor's role in shaping that market. We expect the vendor's vision to be compatible with our view of the market's evolution. A vendor's vision of the evolution of the data center and the expanding role of distributed file and object storage are important criteria. In contrast with how we measure Ability to Execute criteria, the rating for Completeness of Vision is based on direct vendor interactions, and on our analysis of the vendor's view of the future.

Table 2.   Completeness of Vision Evaluation Criteria

Evaluation Criteria

Weighting

Market Understanding

High

Marketing Strategy

Low

Sales Strategy

Medium

Offering (Product) Strategy

High

Business Model

Medium

Vertical/Industry Strategy

Medium

Innovation

High

Geographic Strategy

Medium

Source: Gartner (October 2017)

Quadrant Descriptions

Leaders

Vendors in the Leaders quadrant have the highest scores for their Ability to Execute and Completeness of Vision. A vendor in the Leaders quadrant has the market share, credibility, and marketing and sales capabilities needed to drive the acceptance of new technologies. Market leaders will typically be able to execute strongly across multiple geographies with products that cover both distributed file systems and object storage offerings. They will also have consistent financial performance, broad platform support and flexible deployment models.

Challengers

Challengers are typically vendors with proven global presence and market achievement that only target a narrower subset of the market, or have not yet established themselves across the broader market for both distributed file systems and object storage areas. They have strong products, as well as sufficient credible market position and resources to sustain continued growth in the future, but currently fall behind on influence and thought leadership for this market segment.

Visionaries

These are typically vendors that are focusing on strong innovation and product differentiation, but are smaller vendors with limited reach or achievement to date, or larger vendors with innovation programs that are still unproven. A vendor in the Visionaries quadrant delivers innovative products that address operationally or financially important end-user problems on a broad scale, but has not demonstrated the ability to capture market share or sustainable profitability.

Niche Players

Many distributed file system or object storage vendors will address a more narrow market niche, or they may be vendors with market programs that have not yet established their differentiation and/or execution ability. However, Niche Player vendors may address their specific market category and excel by focusing on specific market or vertical segments.

Context

This Magic Quadrant represents vendors that sell products for unstructured data growth for enterprise data centers. The distributed file system and object storage market emerged as a response to the tremendous increase in unstructured data generation that is fueled by new business requirements. To address it, the storage platform has to be based on a scale-out software approach to enable seamless data growth with a strong emphasis on long-term data efficiency for cost optimization. I&O leaders are now looking for distributed scale-out storage products to build new platforms based on software-defined approaches where performance comes from hardware innovation of a commodity hardware layer and data resiliency comes from a scale-out software layer where data is distributed across multiple nodes.

Across many products in this market, vendors are providing appliances, software-only products and preintegrated storage systems to fit the needs of the different deployment strategies of enterprise end users.

As the distributed file system and object storage market matures, storage software and hardware vendors are expanding their product portfolios to provide more differentiated and agile offerings. New consumption models and procurement offerings are emerging to provide end users with different ways to purchase storage. Advances in software technology and the commoditization of the hardware will make it possible for I&O leaders to enjoy web-scale economics and scalability of the storage platform for unstructured data growth of bimodal IT.

Market Overview

The markets for distributed file systems and object storage are actively merging. That is the reason Gartner is publishing a single Magic Quadrant on the combined segments —- it will eventually be one market. The distinctions between the two segments are slowly blurring, but the buyers are already treating it as one market and requiring both file and object access for unstructured dataset solutions.

Enterprises are often deciding between public cloud and on-premises infrastructure for given workloads. Organizational culture and sensitivity to security and governance mandates are typically the leading factors that enterprises consider when deciding whether to move applications and data to the public cloud or to keep them on-premises.

When customers choose to keep the applications and data on-premises, they are increasingly choosing between products such as Dell EMC Isilon and IBM Cloud Object Storage for large sets of unstructured data. In many cases, customers seeking solutions in this market would be better-suited with a single product that has both file and object personalities, so that workloads can seamlessly interact with data using the most appropriate protocol for the specific task and environment.

Startups and Innovation

Formerly risk-averse enterprises have become receptive to buying from storage startups that are using clean-sheet designs and a wealth of knowledge to build more efficient systems. This is illustrated by the popularity of products such as solid-state arrays, hyperconverged integrated systems (HCISs), distributed file systems and object storage.

The startups in the distributed file system segment are still emerging and did not meet the inclusion criteria outlined for this Magic Quadrant as a result, but the object storage startups did. The object storage startups are challenging the incumbents with new paradigms for deployment and operation that ultimately lower total cost of ownership (TCO).

Choice in Deployment

The vendors in the market for distributed file systems and object storage are offering mixed deployment options to give customers choices in how they deploy infrastructure. Common deployment options include turnkey appliances or software-only option that can be deployed either on bare-metal industry-standard hardware as virtual machines or on Docker containers. Increasingly, vendors in this market are offering their products as software-defined storage precertified to run on x86 industry-standard hardware.

Amazon S3 API Standardization

The current object storage segment can be thought of as a two-sided market: There are providers of object storage protocols and consumers of these protocols consisting of applications. There were more providers than consumers until the Amazon S3 API became the de facto standard for object storage. Vendors deploying object storage platforms in enterprise data centers adopted Amazon S3, a protocol mainly used in the public cloud, because of the developer community that formed around it. Now there are many consumers and providers, all using Amazon S3. The object storage market is finally in equilibrium.

The interest in using public cloud services such as AWS has brought customer awareness to the object storage market. Software developers building Mode 2 web and mobile applications are sometimes asked to repatriate these applications back to enterprise data centers. Enterprise IT seeks control of applications and data, while software developers seek novel and efficient ways of programmatically interacting with infrastructure. The market for on-premises object storage products solves both of these.

Evidence

Placement on the Magic Quadrant for Distributed File Systems and Object Storage is based on Gartner's view of a vendor's performance against the criteria noted in this research. Gartner's view regarding vendor placement on the Magic Quadrant is heavily influenced by more than 2,000 inquiries and one-on-one meetings with Gartner clients on the topic of object storage and distributed file systems solutions, conducted since the publication of the last Magic Quadrant. Gartner also utilizes worldwide end-user surveys, Gartner conference kiosk surveys, Gartner conference session polling data, gartner.com Research Circle polls and Gartner Peer Insights. The Magic Quadrant methodology includes the solicitation of references from each vendor; for this Magic Quadrant, Gartner conducted over 144 reference checks (via electronic survey and/or live interview) from a set of customers provided by each vendor. The included vendors submitted comprehensive responses to Gartner's Magic Quadrant survey on this topic, which were used as the basis for subsequent vendor briefings and follow-up meetings, product demonstrations, and correspondence.

Additionally, this research drew input from other Gartner analysts, industry contacts and public sources, such as U.S. Securities and Exchange Commission filings, articles, speeches, published papers and public domain videos.

Evaluation Criteria Definitions

Ability to Execute

Product/Service: Core goods and services offered by the vendor for the defined market. This includes current product/service capabilities, quality, feature sets, skills and so on, whether offered natively or through OEM agreements/partnerships as defined in the market definition and detailed in the subcriteria.

Overall Viability: Viability includes an assessment of the overall organization's financial health, the financial and practical success of the business unit, and the likelihood that the individual business unit will continue investing in the product, will continue offering the product and will advance the state of the art within the organization's portfolio of products.

Sales Execution/Pricing: The vendor's capabilities in all presales activities and the structure that supports them. This includes deal management, pricing and negotiation, presales support, and the overall effectiveness of the sales channel.

Market Responsiveness/Record: Ability to respond, change direction, be flexible and achieve competitive success as opportunities develop, competitors act, customer needs evolve and market dynamics change. This criterion also considers the vendor's history of responsiveness.

Marketing Execution: The clarity, quality, creativity and efficacy of programs designed to deliver the organization's message to influence the market, promote the brand and business, increase awareness of the products, and establish a positive identification with the product/brand and organization in the minds of buyers. This "mind share" can be driven by a combination of publicity, promotional initiatives, thought leadership, word of mouth and sales activities.

Customer Experience: Relationships, products and services/programs that enable clients to be successful with the products evaluated. Specifically, this includes the ways customers receive technical support or account support. This can also include ancillary tools, customer support programs (and the quality thereof), availability of user groups, service-level agreements and so on.

Operations: The ability of the organization to meet its goals and commitments. Factors include the quality of the organizational structure, including skills, experiences, programs, systems and other vehicles that enable the organization to operate effectively and efficiently on an ongoing basis.

Completeness of Vision

Market Understanding: Ability of the vendor to understand buyers' wants and needs and to translate those into products and services. Vendors that show the highest degree of vision listen to and understand buyers' wants and needs, and can shape or enhance those with their added vision.

Marketing Strategy: A clear, differentiated set of messages consistently communicated throughout the organization and externalized through the website, advertising, customer programs and positioning statements.

Sales Strategy: The strategy for selling products that uses the appropriate network of direct and indirect sales, marketing, service, and communication affiliates that extend the scope and depth of market reach, skills, expertise, technologies, services and the customer base.

Offering (Product) Strategy: The vendor's approach to product development and delivery that emphasizes differentiation, functionality, methodology and feature sets as they map to current and future requirements.

Business Model: The soundness and logic of the vendor's underlying business proposition.

Vertical/Industry Strategy: The vendor's strategy to direct resources, skills and offerings to meet the specific needs of individual market segments, including vertical markets.

Innovation: Direct, related, complementary and synergistic layouts of resources, expertise or capital for investment, consolidation, defensive or pre-emptive purposes.

Geographic Strategy: The vendor's strategy to direct resources, skills and offerings to meet the specific needs of geographies outside the "home" or native geography, either directly or through partners, channels and subsidiaries as appropriate for that geography and market.