Rightsizing the Enterprise Data Center Network

20 March 2013 ID:G00236105
Analyst(s): Mark Fabbi, Caio Misticone

VIEW SUMMARY

New fixed form factor switches can reduce the data center core network's longstanding reliance on costlier and oversized chassis-based switches. We discuss what network managers need to know about rightsizing the enterprise data center network.

Overview

Impacts

  • Fixed format switches will enable network managers to overcome performance, reliability and scalability differences, compared with chassis-based switches.
  • Newer small form factor core devices can enable network managers to reduce capital costs by 30% to 70%, and save 30% or more on operations expenses, compared with chassis-based switches.
  • Fixed format switches offer network managers internal nonblocking performance, low latency and generally very good oversubscription solutions.

Recommendations

  • Use fixed form factor switches as an alternative to costlier chassis-based switch architectures, and shortlist fixed form factor vendors that allow you to scale to meet your needs, ensuring that scalability requirements are included in the RFP.
  • Enterprises should not only compare the capital cost differences between fixed form factor switches and chassis-based switches, but also should look at operating expenditures (opex) such as power consumption and maintenance, which can cut costs by at least 30%.
  • Select vendors with offerings that best align with your enterprise's key evaluation criteria, including latency and oversubscription ratios, because these play important roles in application performance.

Analysis

The battle for massive scale and large fabrics continues to evolve among data center network equipment vendors. Although vendors elect to differentiate and expend significant marketing effort on which owns the biggest fabric, Gartner does not consider this a particularly relevant criteria for the majority of enterprise data center networks, because most of them reside within the 200 to 1,000 physical server range. As enterprises virtualize and consolidate their server infrastructure and offload applications to cloud services providers, most can deploy simpler, smaller and ultimately more cost-effective data center network solutions using small form factor switches.

This research investigates how enterprises can use different architectures, with fixed form factor switches connected to top of rack (ToR) switches and blade switches, to replace big chassis platforms that are often underutilized, overpowered (from both a performance and an electrical standpoint) and overpriced.

Figure 1. Impacts and Top Recommendations for Rightsizing the Enterprise Data Center Network
Figure 1.Impacts and Top Recommendations for Rightsizing the Enterprise Data Center Network

Source: Gartner (March 2013)

Impacts and Recommendations

Fixed format switches will enable network managers to overcome performance, reliability and scalability differences, compared with chassis-based switches

With the emergence of fixed format switches, enterprises now have viable alternatives when deploying their data center core network. Historically, the critical data center network requirements of performance, reliability, scalability and management necessitated the use of big chassis-based solutions. However, vendors such as Alcatel-Lucent, Arista Networks, Dell Force10 and others have developed new architectural designs using fixed port switches with one- or two-tier architectures to offer an alternative for many enterprise data centers.

Performance

With server virtualization reducing the number of physical servers and consolidating traffic, enterprise data centers are migrating to 10G attached servers with 40G switch interconnections. The new generation of fixed form factor switches typically offers 40 to 48 ports of 10G with four to six ports of 40G (which often can be configured as multiple 10G ports). We are also seeing examples of full 40G support on all ports. These switches offer internal nonblocking performance, low latency and generally very good oversubscription solutions. In most cases, fixed form factor will offer significantly improved server-to-server latencies (in many cases, by as much as 90%), because they do not require the complex internal architecture of chassis-based switches.

Reliability

Chassis-based switch reliability is based on features such as redundant supervisor modules, fabric modules, power supplies and fans. Fixed form factor switches distribute the control logic and reliability risk, and, when combined with MC-LAG and other architectural approaches, provide a strong failover capability and resiliency, compared with chassis-based solutions. Newer switches offer dual power supplies to provide additional resiliency.

Scale

While absolute scale is still largely the ultimate advantage of chassis-based solutions, fixed form factor switches in a simple two-tier architecture can support up to 500 physical servers. Single-tier mesh solutions available from a number of vendors can support more than 200 server-facing ports. Furthermore, scale can be accomplished by using more-complex fabric solutions and leaf-and-spine Clos network solutions to support the needs of most enterprise data centers with server port requirements from 250 to more than 1,000 ports.

Operational Considerations

A chassis solution provides a single point of management for the core data center network. With technologies such as MC-LAG, improved fabric implementations, and simplified one- and two-tiered architectures, the fixed form factor solutions offer a similar or improved operational experience and an alternative to traditional approaches. The elimination of the Spanning Tree Protocol (STP) and the reduction of ports and devices due to the rightsizing approach further help these solutions.

Good examples of fixed form factor switches are Alcatel-Lucent's OS6900, Arista's 7150 and Dell Force10's Z9000.

Recommendation:

  • Use fixed form factor switches as an alternative to costlier chassis-based switch architectures, and shortlist fixed form factor vendors that allow you to scale to meet your needs, ensuring that scalability requirements are included in the RFP.

Newer small form factor core devices can enable network managers to reduce capital costs by 30% to 70%, and save 30% or more on operations expenses, compared with chassis-based switches

Gartner has seen capital costs for fixed form factors at 30% to 70% less than typical chassis-based solutions. The main reason for the delta is that modular chassis have expensive redundant supervisors and fabric, as well as expensive line cards and software licenses. In many proposals, the chassis is only partially filled, resulting in the fixed cost of the chassis being allocated across a relatively small number of ports. Savings vary according to the size of the network and specific vendor architectures. We've seen greater savings (typically 50% to 70%) for smaller network sizes (with 100 to 500 physical server ports) and smaller savings (25% to 50%) when networks start to grow and move beyond 1,000 servers.

From an opex perspective, fixed form factor switches have many advantages:

  • Typically, they are 1 rack unit (U) or 2U tall, compared with chassis-based switches that range between 7U and 25U; therefore, they save significant space.
  • They usually use much less power, with savings ranging from 30% to 70%, compared with more-traditional solutions.
  • In contrast to chassis-based switches, fixed form factor switch maintenance costs savings can be 30% to 90%, because some vendors offer lifetime warranties for these products.

Recommendation:

  • Enterprises should not only compare the capital cost differences between fixed form factor switches and chassis-based switches, but also should look at operating expenditure (opex) such as power consumption and maintenance, which can cut costs by at least 30%.

Fixed format switches offer network managers internal nonblocking performance, low latency and generally very good oversubscription solutions

Vendors often have different architectures and products to meet different requirements, such as latency, oversubscription rates, performance and scale within the data center. In a traditional three-tier architecture, latency tends to be higher, because the packet may have to traverse all three layers before it gets to its final destination. With high-performance fixed form factor hardware and a one- or two-tier network, extremely low latency results can be achieved.

Oversubscription rates can also vary widely, depending on the relationship between the uplink bandwidth and the available bandwidth for the downlink connections to the servers. In a nonblocking scenario, the device's internal backplane exceeds the full-duplex bandwidth requirements of its ports. So, if one port is oversubscribed, it will not affect the performance or delay for other ports in the same switch. A low oversubscription helps prevent application bottlenecks and ensure consistent application performance. Gartner has found that the typical ratio for oversubscription is 3-to-1 or lower, depending on bandwidth and delay requirements

Another important factor to consider is that the adequate network architecture should be based on application and business requirements. For example, in high-frequency trading, cluster/computing modeling or deep analytics, success may be measured in nanoseconds. In this case, an architecture that provides the lowest possible latency is critical and should be one of the most important factors to consider during the evaluation process. Examples of ultra-low-latency switches are Arista's 7150 and the Cisco Nexus 3000 Series. Alternatively, if your environment is highly virtualized and cloud-oriented, an architecture that provides the ability to be elastic, scalable and virtualization-aware would prevail over an architecture with latency as the prime factor.

Recommendation:

  • Select vendors with offerings that best align with your enterprise's key evaluation criteria, including latency and oversubscription ratios, because these play important roles in application performance.

Download Attachments