LICENSED FOR DISTRIBUTION

Critical Capabilities for Distributed File Systems

Published: 09 November 2017 ID: G00274893

Analyst(s):

Summary

The importance of storing and analyzing large-scale, unstructured data is bringing distributed architectures to the forefront of IT infrastructure planning. Gartner compares six distributed file system storage products against eight critical capabilities in five use cases relevant to I&O leaders.

Overview

Key Findings

  • Big data analytics, cloud storage and support for microservice applications are emerging use cases for distributed file systems. Distributed file system storage products face competition from object storage products, due to their superior scalability and easier management, as well as from traditional, scale-up network-attached storage (NAS) products, due to their growing capacity and better interoperability.

  • The nascent trend of deploying distributed file system products via public cloud is expected to challenge the growth of on-premises deployments due to the promise of low entry costs from public cloud infrastructure as a service (IaaS), rapid scalability and a growing ecosystem of independent software vendors.

Recommendations

I&O leaders focused on infrastructure agility should:

  • Select distributed file system storage products based on their interoperability with the ISV solutions that are dominant in their environment.

  • Validate all performance claims with proof-of-concept deployments, given that performance varies greatly by protocol type and file sizes.

  • Prioritize products with software-defined storage capabilities. This approach will enable you to extend distributed file systems to the public cloud and edge deployments.

  • Shortlist vendors with the ability to run natively in the public cloud and that enable hybrid cloud storage deployments with bidirectional tiering, as this emerging paradigm is experiencing positive, early traction with enterprises.

Strategic Planning Assumption

By 2021, more than 80% of enterprise data will be stored in scale-out storage systems in enterprise and cloud data centers, up from 30% in 2017.

What You Need to Know

This document was revised on 14 November 2017. The document you are viewing is the corrected version. For more information, see the Corrections page on gartner.com.

The growing demand for storage products that can scale linearly in capacity and performance to manage unstructured data is propelling distributed file system products to the forefront for five use cases (such as high-performance computing [HPC], analytics, file sharing, backup and archiving).

In this research, we rate six distributed file system storage products on their ability to support five use cases by means of those products' capabilities, which are critical to those cases. As revealed by the analysis in this research, the evaluated products, for the most part, vary greatly in their architecture, capabilities and alignment with the aforementioned use cases. Although many vendors continue to fine-tune their products to focus on specific use cases, the leading vendors in this research cater to a wide variety of use cases in enterprise environments.

I&O leaders should carefully select a distributed file system storage product through a rigorous planning process that involves thoroughly evaluating the products' critical capabilities. Furthermore, the majority of distributed file system products are now available as a software-defined storage (SDS) deployment option, which not only increases flexibility in hardware deployment choices but also enables the ability to connect on-premises storage to the public cloud and the edge computing. In addition, because expertise in distributed file system storage and SDS is uncommon in enterprise IT organizations, I&O leaders should allocate time and budget for training on the technology.

Analysis

Critical Capabilities Use-Case Graphics

Figure 1. Vendors' Product Scores for the Archiving Use Case
Research image courtesy of Gartner, Inc.

Source: Gartner (November 2017)

Figure 2. Vendors' Product Scores for the Backup Use Case
Research image courtesy of Gartner, Inc.

Source: Gartner (November 2017)

Figure 3. Vendors' Product Scores for the Commercial HPC Use Case
Research image courtesy of Gartner, Inc.

Source: Gartner (November 2017)

Figure 4. Vendors' Product Scores for the Large Home Directories Use Case
Research image courtesy of Gartner, Inc.

Source: Gartner (November 2017)

Figure 5. Vendors' Product Scores for the Analytics Use Case
Research image courtesy of Gartner, Inc.

Source: Gartner (November 2017)

Vendors

Dell EMC Isilon

Dell's acquisition of EMC in 3Q16 (Dell had previously acquired Isilon Systems in 4Q10) established Dell as a leading supplier of clustered file systems. The OneFS operating system, which powers Isilon solutions, has a proven track record of 15 years and is now on version 8.1. Depending on the specific model, Isilon solutions scale to a maximum cluster size of 68PB in 144 nodes. Dell EMC is currently shipping Generation 5 and Generation 6 hardware products, which can coexist in a single cluster. This makes long-term ownership simpler and reduces migration costs and complexity. Generation 6 products are available in all-flash (F800), hybrid (H400, H500 and H600) and archive models (A200 and A2000), to support commercial HPC, home directory, file sharing and archive workloads. For new-generation products, Isilon's minimum cluster configuration is four nodes. With the Generation 6 of Isilon solutions, Dell has unbundled Isilon hardware from OneFS software, which allows customers to choose their own hardware and control costs. The base OneFS software includes support for all versions of NAS protocols (NFS/SMB), OpenStack SWIFT and Hadoop HDFS protocols. Isilon does not have all-inclusive pricing, and separate licenses are available for Isilon SnapshotIQ, SmartQuotas, SmartConnect Advanced, SmartDedupe, SyncIQ, SmartLock and SmartPools software. Isilon is available in a variety of product families to meet user-specific capacity, throughput, latency and cost requirements. Isilon has also offered customers deployment flexibility by adding software-only IsilonSD Edge products. IsilonSD Edge runs as a software-defined storage VM under VMware ESXi 6.0 or 6.5, it is compatible with any server on the VMware VSAN hardware compatibility list, and it scales to 36TB. The Isilon CloudPools feature provides policy-based automated tiering of inactive data from Isilon clusters to public cloud data centers (such as AWS and Microsoft Azure) and to private cloud object platforms based on ECS.

Customers have high satisfaction from Isilon's flexibility, ease of deployment, architecture, scalability and reliability, but have raised concerns regarding acquisition and support costs. Some customers conducting evaluations during Dell's acquisition of EMC have also raised concern over the quality of support.

Huawei OceanStor 9000

Huawei began shipping the OceanStor 9000 product in 4Q13. OceanStor 9000 supports both file and object storage. However, it is primarily used for file storage. OceanStor's base software includes support for CIFS, NFS, FTP, HDFS, NDMP, Amazon S3/OpenStack Swift, NIS, Microsoft Active Directory and LDAP protocols. It is most commonly used for commercial HPC, large home directories and analytics applications. The product is a fully symmetrical distributed architecture delivered as a hardware appliance, which is dynamically and nondestructively expandable from three to 288 nodes, supporting a global namespace of up to 100PB. The OceanStor 9000 series had minor enhancements in the past 12 months, including new hardware controllers. Huawei sells OceanStor 9000 direct, through reseller partners and as an OEM component of application-specific appliances in the video surveillance and oil and gas industries. OceanStor 9000 supports SSD, SATA, SAS and NL-SAS disk drives, which provide a useful variety of cost and performance options. Optional software includes storage tiering, automated load balancing, quota management, snapshot, WORM, performance acceleration, remote replication, data migration, third-party antivirus integration and surveillance video and imagery restoration.

In addition to functionality and performance, Huawei's long-term vision and strong customer focus are the primary reasons for selecting OceanStor 9000 in the APAC region. Huawei end users' references noted OceanStor 9000's ease of implementation. Some customers expressed the desire for all-flash options, lower-cost cold storage, integration with public clouds, simplified management and greater granularity of monitoring capabilities.

IBM Spectrum Scale

IBM Spectrum Scale, formerly known as General Parallel File System (GPFS), is a mature product with a proven track record. IBM Spectrum Scale has been available on IBM AIX since 1998, on Linux since 2001 and on Windows Server since 2008. Spectrum Scale is a distributed file system with unified file and object capabilities, and is available as a software-only solution, as an IBM Elastic Storage Server appliance or as a cloud service. The software is offered in three editions: Standard, Advanced and Data Management. The Advanced edition provides asynchronous data replication and encryption for data at rest, while the Data Management edition has capacity-based licensing. Spectrum Scale supports up to 8YB in a single namespace and a maximum file size of 8EB. Spectrum Scale is used for commercial HPC, analytic applications, file sharing and large content repositories, across a broad range of industries including media and entertainment, life sciences, healthcare and financial services. IBM's Elastic Storage Server, which incorporates Spectrum Scale software in a hardware appliance, is available in all flash and disk models and reduces implementation complexity. With a heritage in HPC, media and entertainment, Spectrum Scale is well-suited for very high capacity and very fast throughput requirements. The appliance version of Spectrum Scale has made the offering a viable alternative for non-technical-computing users. In addition to its own Elastic Storage Server appliance, IBM partners with DDN, Lenovo, Seagate, Bull, Cray, Fujitsu and NEC, as alternative appliance delivery channels.

Customers often cite performance, scalability, reliability, breadth of capabilities and vendor expertise as primary reasons for selecting IBM Spectrum Scale. It can be complex to implement, and although IBM has developed a team of technical experts to work with customers and reseller partners to assist with proofs of concept and implementations, early mistakes in deployment can be complex to correct. Customers also noted the need for tighter integration with AWS and Azure as concerns that were addressed in September 2017 by the introduction of Quick Start Spectrum Scale deployment on AWS.

Qumulo

Qumulo Core scale-out software was generally available on 16 March 2015. Qumulo is deployed on its own QC-Series hybrid appliances, but it can also run x86 commodity hardware platforms (such as HPE Apollo 4200 servers) and can be deployed in AWS public cloud. Qumulo Core is a distributed file system that lives on top of a distributed block store and utilizes Reed-Solomon erasure codes, which provides better resiliency than a RAID algorithm for large disk drive implementations. Protection occurs at the block layer, rather than the file system layer, allowing Qumulo Core to deliver high efficiency regardless of file size. Qumulo offers transparent, all-inclusive pricing based on raw capacity. The smallest on-premises Qumulo cluster size is four nodes, 96TB raw capacity and can scale up to 1,000 nodes in a single cluster. The product features file system analytics to provide visibility to performance and capacity trends and make better data management decisions. Over the last year, Qumulo added erasure coding, real-time capacity quotas, replication, capacity trending and more hardware options. Qumulo Core is typically used for animation rendering, genomic sequencing, scientific research, digital content stores, Splunk data storage and enterprise home directories.

End users often cite ease of management and product vision as some of the differentiating factors for Qumulo selection. New features are released and delivered to customers biweekly, which some users find too frequent to keep up with. Gartner clients are reporting high-quality customer support but note that the product is still missing some enterprise features (such as NFS4 and SMB3 support, compression, deduplication, seamless firmware upgrade and node removal).

On 19 September 2017, Qumulo expanded its offering with the announcement of Qumulo File Fabric (QF2), which is now available using AWS. QF2 is designed as interconnected file fabric that connects Qumulo Core clusters running in the public cloud with Qumulo clusters running in on-premises data centers for workload mobility and cloud bursting.

Red Hat Gluster Storage

The Red Hat Gluster Storage (formerly Red Hat Storage Server) product has been offered since 2011, when open-source-based Gluster was acquired by Red Hat. Since then, Red Hat has launched Gluster's software-defined storage product, GlusterFS, with more stability, better features and additional prepackaged software. The product is a software-only scale-out, multiprotocol (NFS, RESTful APIs, Server Message Block [SMB]) and open-source storage software solution with PB-scale capacity and improved snapshot and local and geo replication capabilities. Red Hat Storage Server is a preintegrated software product consisting of Red Hat Enterprise Linux (RHEL), GlusterFS and the extensible file system (XFS). It can be installed on bare-metal hardware, in a kernel-based virtual machine (KVM) or in VMware hypervisors to pool storage resources as well as container image. Gluster can scale from three to 128 nodes in the cluster and can be deployed on-premises or in the public cloud (AWS, Azure and Google Cloud).

The product benefits from Red Hat's complementary open-source community projects and technical support capabilities, which include community-, standard- and premium-support options. Red Hat Gluster Storage is priced per node, which makes it an attractive option for end users implementing high-capacity nodes. However, the product lacks some capabilities that enterprise IT buyers aspire to provide in a file system product, such as public cloud tiering and WORM. Gluster has erasure coding but lacks native support for compression and deduplication. End users also report that Gluster is best performing for large file operations and streaming workloads, but is not optimized for small and random I/O workloads.

Red Hat Gluster is often implemented for rich-media content, backup and recovery and near-line archives. Red Hat Gluster Storage is now integrated with Red Hat OpenShift Container Platform solution and has been offered as part of Red Hat Hyperconverged Infrastructure.

StorageCraft Exablox

Exablox first launched its scale-out file system product in 2010, and was acquired by StorageCraft in January 2017. The Exablox OneBlox scale-out product provides distributed file system capabilities with NFS and SMB support and continuous and unlimited snapshots, in-line deduplication, compression and off-site replication. Exablox file system design is based on objects that can be stored in highly scalable, location independent, flat key-value stores. OneBlox is delivered as hardware appliance and has recently added support for all-flash model in order to expand toward latency-sensitive workloads. Exablox operates on a "bring you own drives" approach and accepts a wide range of HDD and SSD drives from various vendors, which reduces costs and avoids being dependent on one supplier with high cost of OEM drives. The vendor sells the product based on a per-node license, and can start with single-node implementation. Exablox is often used as an alternative to backup target solutions. Over the last 12 months, the product has added variable-length deduplication, compression, file auditing, multisite many-to-many remote replication and encryption.

Exablox, combined with StorageCraft ShadowProtect, can replicate to StorageCraft's cloud services for DRaaS. Exablox management is done through OneSystem as a mutitenant cloud-based management service. Exablox is most often used as a backup and archiving target and secondary storage repository. The product is missing some enterprise features, such as tiering, erasure coding, NFS 4 support and support for nondisruptive upgrades. Gartner clients outline ease of installation and expansion as one of the product highlights but would like OneBlox to have deeper integration with additional backup ISV and more detailed monitoring and analytical capabilities.

Context

Traditionally, the major market for distributed file system storage has been academic and commercial HPC environments for workloads (such as genomic sequencing, financial modeling, 3D animation, weather forecasting and seismic analysis). As such, distributed file system storage solutions have focused on scalable capacity, raw computing power and aggregated bandwidth, with data protection, security and efficiency as secondary considerations.

However, ever-increasing data growth — chiefly, unstructured data growth — in the enterprise has led many I&O leaders in these organizations to deploy the technology to support large home directories, backup and archiving. For these use cases, better security and multitenancy, easier manageability, robust data protection and ISV interoperability are becoming more important.

In addition to simply supporting these four use cases (academic HPC not included), I&O leaders are embracing distributed file system storage for its added benefits. First and foremost, the technology includes embedded functionality for storage management, resiliency and security at the software level, easing the tasks related to those functions in the I&O organization. The technology also offers nearly linear horizontal scaling and delivers highly aggregated performance through parallelism. This means that distributed file system storage enables pay-as-you-grow storage capacity and performance, making it a cost-effective alternative to scale-up storage, in particular when I&O leaders are forced to purchase more storage than needed to ensure storage growth does not outpace capacity. Lastly, most distributed file system storage vendors use standard x86 hardware and can be purchased as a software-only option, thus providing supplier and cost flexibility, which can reduce the hardware acquisition costs.

In big data analytics, scenarios in which these file systems can run map/reduce processing jobs, cloud storage for file sync and share and other SaaS workloads are emerging use cases for scale-out file system storage products.

Product/Service Class Definition

Distributed file system storage uses a single-parallel file system to cluster multiple storage nodes together, presenting a single namespace and storage pool to provide high bandwidth for multiple hosts in parallel. Data is distributed over multiple nodes in the cluster to handle availability and data protection in a self-healing manner and to provide high throughput and capacity linearly.

The products in this category must have "shared nothing architecture." A shared nothing architecture is a distributed computing architecture in which each node is independent and self-sufficient, and there is no single point of contention across the system. More specifically, none of the nodes share memory or disk storage. People typically contrast shared nothing architecture with systems that keep a large amount of centrally stored state information, whether in a database, an application server or any other similar single point of contention.

Critical Capabilities Definition

Capacity

The ability of the product to support growth in storage capacity in a nearly linear manner, with capacity requirements often extending from hundreds of TB to the PB scale.

Storage Efficiency

The ability of the product to support storage efficiency technologies (such as compression, deduplication, thin provisioning and automated tiering) to reduce TCO.

Interoperability

The ability of the product to support third-party ISV applications, public cloud APIs and multivendor hypervisors.

Manageability

The ability of the product to support automation, management and monitoring, and to provide reporting tools.

Performance

The aggregated IOPS, bandwidth and low latency that can be delivered by the cluster functioning at maximum specifications and observed in real-world configurations.

Resiliency

The ability of the product to provision a high level of system availability and data protection.

Security and Multitenancy

The depth and breadth of a product's native security and multitenancy features, including granular access control, user-driven encryption, malware protection and data immutability.

Pricing

The ability to deliver good value for the money as well as flexible consumption models and pricing options.

Use Cases

Archiving

In this use case, an enterprise uses distributed file system storage to meet the requirements of long-term data retention.

Backup

Enterprises use distributed file system storage to meet the requirements of large-scale, disk-based backup for low RTOs and RPOs.

Commercial HPC

In this use case, an enterprise uses distributed file system storage to provide high throughput and parallel read-and-write access to large volumes of data.

Large Home Directories

Enterprises use distributed file system storage to support large home directories, as they would with scale-up NAS, only on a larger scale.

Analytics

In this use case, enterprise uses this platform for analytics workloads (such as big data, log and sensor analysis and real-time analytics).

Inclusion Criteria

The products covered in this research include distributed file system storage offerings with a sizable footprint in the market. To qualify for inclusion, vendors must meet all of the following requirements:

  • Revenue above $10 million per year for the distributed file systems product from 1 May 2016 through 30 April 2017, or should have at least 50 production customers each consuming more than 300TB capacity through distributed file storage protocols only. The vendor must provide reference materials to support this criterion.

  • The product must be installed in at least two major geographies (among North America, EMEA, Asia/Pacific and South America).

  • The product should be deployed across multiple use cases that are outlined in this research.

  • The product must be designed primarily for on-premises workloads rather than a pass-through solution where data is permanently stored elsewhere.

  • The vendor must own the storage software intellectual property and be a product developer. If the product is built on top of open-source software, the vendor must be one of the top 10 active contributors to the community (in terms of code contribution).

  • The product must be:

    • Delivered as either an appliance or software-based solution.

    • Available for purchase as a stand-alone storage product and not an integrated or converged system with compute and hypervisor bundle.

  • The product must have:

    • File access to the global file system.

    • A shared nothing architecture where data is replicated or erasure coded over the network across multiple nodes in the cluster.

    • The ability to handle disk, enclosure or node failures in a graceful manner without impacting availability.

    • A single file system capable of expanding beyond 500TB.

    • A cluster that spans more than three nodes.

    • Support for horizontal scaling of capacity and throughput in a cluster mode or in independent node additions with a global file system.

Table 1.   Weighting for Critical Capabilities in Use Cases

Critical Capabilities

Archiving

Backup

Commercial HPC

Large Home Directories

Analytics

Capacity

20%

15%

20%

10%

15%

Storage Efficiency

8%

10%

3%

20%

5%

Interoperability

8%

10%

3%

6%

10%

Manageability

12%

20%

9%

12%

15%

Performance

7%

5%

40%

10%

15%

Resiliency

14%

10%

10%

20%

15%

Security and Multitenancy

20%

5%

5%

12%

14%

Pricing

11%

25%

10%

10%

11%

Total

100%

100%

100%

100%

100%

As of November 2017

Source: Gartner (November 2017)

This methodology requires analysts to identify the critical capabilities for a class of products/services. Each capability is then weighed in terms of its relative importance for specific product/service use cases.

Critical Capabilities Rating

Each of the products/services has been evaluated on the critical capabilities on a scale of 1 to 5; a score of 1 = Poor (most or all defined requirements are not achieved), while 5 = Outstanding (significantly exceeds requirements).

Table 2.   Product Rating on Critical Capabilities

Critical Capabilities

Dell EMC Isilon

Huawei OceanStor 9000

IBM Spectrum Scale

Qumulo

Red Hat Gluster Storage

StorageCraft Exablox

Capacity

3.0

3.5

4.5

3.5

4.5

4.5

Storage Efficiency

3.5

3.3

3.5

2.8

3.0

2.8

Interoperability

4.0

3.0

3.0

2.5

2.5

2.0

Manageability

3.0

3.0

2.0

4.0

2.5

2.6

Performance

4.0

3.0

4.5

3.0

2.5

1.5

Resiliency

4.5

4.0

4.3

3.0

3.5

3.0

Security and Multitenancy

3.5

2.5

3.0

2.0

2.5

2.7

Pricing

3.0

2.3

2.0

4.0

3.8

4.0

As of November 2017

Source: Gartner (November 2017)

Table 3 shows the product/service scores for each use case. The scores, which are generated by multiplying the use-case weightings by the product/service ratings, summarize how well the critical capabilities are met for each use case.

Table 3.   Product Score in Use Cases

Use Cases

Dell EMC Isilon

Huawei OceanStor 9000

IBM Spectrum Scale

Qumulo

Red Hat Gluster Storage

StorageCraft Exablox

Archiving

3.50

3.09

3.40

3.07

3.22

3.10

Backup

3.38

3.01

3.03

3.41

3.28

3.19

Commercial HPC

3.62

3.11

3.86

3.22

3.15

2.71

Large Home Directories

3.62

3.18

3.44

3.08

3.13

2.92

Analytics

3.57

3.09

3.41

3.14

3.12

2.90

As of November 2017

Source: Gartner (November 2017)

To determine an overall score for each product/service in the use cases, multiply the ratings in Table 2 by the weightings shown in Table 1.

Vendors to Watch

The following vendors did not meet the criteria for inclusion in the critical capabilities, but may be worthy of consideration as they are starting to get more traction in the distributed file systems storage market:

  • DDN

  • Elastifile

  • Microsoft

  • Nexenta

  • Pure Storage

  • Inspur

  • Veritas

Evidence

Scoring for the eight critical capabilities was derived through vendor responses to a comprehensive primary-research questionnaire administered by the authors. Extensive follow-up interviews were conducted with all participating vendors. Scoring also took into consideration Gartner inquiries and the responses of vendors' reference customers to a primary-research survey that covered ease of use, performance and whether products met the claims of their related vendors.

Critical Capabilities Methodology

This methodology requires analysts to identify the critical capabilities for a class of products or services. Each capability is then weighted in terms of its relative importance for specific product or service use cases. Next, products/services are rated in terms of how well they achieve each of the critical capabilities. A score that summarizes how well they meet the critical capabilities for each use case is then calculated for each product/service.

"Critical capabilities" are attributes that differentiate products/services in a class in terms of their quality and performance. Gartner recommends that users consider the set of critical capabilities as some of the most important criteria for acquisition decisions.

In defining the product/service category for evaluation, the analyst first identifies the leading uses for the products/services in this market. What needs are end-users looking to fulfill, when considering products/services in this market? Use cases should match common client deployment scenarios. These distinct client scenarios define the Use Cases.

The analyst then identifies the critical capabilities. These capabilities are generalized groups of features commonly required by this class of products/services. Each capability is assigned a level of importance in fulfilling that particular need; some sets of features are more important than others, depending on the use case being evaluated.

Each vendor’s product or service is evaluated in terms of how well it delivers each capability, on a five-point scale. These ratings are displayed side-by-side for all vendors, allowing easy comparisons between the different sets of features.

Ratings and summary scores range from 1.0 to 5.0:

1 = Poor or Absent: most or all defined requirements for a capability are not achieved

2 = Fair: some requirements are not achieved

3 = Good: meets requirements

4 = Excellent: meets or exceeds some requirements

5 = Outstanding: significantly exceeds requirements

To determine an overall score for each product in the use cases, the product ratings are multiplied by the weightings to come up with the product score in use cases.

The critical capabilities Gartner has selected do not represent all capabilities for any product; therefore, may not represent those most important for a specific use situation or business objective. Clients should use a critical capabilities analysis as one of several sources of input about a product before making a product/service decision.