How to Turn Old Data Centers Into Critical IT Assets

Infrastructure and operations leaders often focus on new ideas, technologies and ways to deliver business value, while older data centers get minimal attention. With a clear strategy, older data centers can be updated to support new and emerging business services, while reducing operating costs.

Overview

Key Findings

• Data center floor space may be underutilized, due to reduced workloads. However, it may need to be optimized to reduce rack space, power and cooling requirements, freeing up floor space to reduce operating costs, as well as handle higher-density workloads.
• Compute requirements continue to grow, but the use of existing floor space is inefficient. IT must redesign the data center white space to support projected growth.
• New (or existing) workloads are driving a need for higher power and cooling densities, and the current facilities may not support them. If moving workloads elsewhere is not an option, then alternative solutions must be evaluated.

Recommendations

Infrastructure and operations leaders responsible for enterprise architecture should:

• Increase floor space capacity and reduce energy costs by replacing or upgrading existing IT racks with self-contained cooling environments.
• Maximize horizontal and vertical growth opportunities by remodeling data center floor space to support hot or cold containment techniques.
• Structure IT delivery with extreme density and reduced energy consumption, by incorporating alternative liquid cooling techniques for specific workloads.

Strategic Planning Assumption

By 2025, data centers deploying specialty cooling and density techniques will see 20% to 40% reductions in operating costs.

Introduction

CIOs and IT leaders are faced with the difficult task of continuing to deliver IT services to a broader audience, with greater complexity, at an ever-increasing rate. Couple this with the impact of a difficult economy, and many data center managers find themselves trying to implement long-needed data center upgrades or improvements with static or decreasing budgets.

Enterprises with existing workloads that will remain in their data centers must decide how best to restructure their physical infrastructures to improve efficiencies and extend the data center’s useful life. The following challenges are most common, and vary from enterprise to enterprise:

• Workloads are migrating out of on-premises data centers, as clients move to software as a service (SaaS) and platform as a service (PaaS), creating increased unused power and cooling capacity.
• The global pandemic has moved IT organizations’ emphasis to delivering more-agile services; in part, they can be fulfilled by the cloud, colocation and existing infrastructure.
• Data residency and sovereignty requirements are being strengthened through new legislation, requiring more emphasis on geographic location of data and, therefore, data centers.
• Because latency, legacy and legal reasons prevent workloads from moving to the cloud, on-premises data centers remain critical to IT planning.

Analysis

Many enterprises face the dilemma of what to do with their existing data centers. During the past few years, the outward focus of IT has been all about cloud migrations, edge strategies and getting workloads closer to the customer; however, a core set of workloads will remain on-premises. These workloads often remain on-premises, because they rely on mainframe or classic UNIX systems; have strict compliance, audit, or latency requirements; or are a set of tightly integrated production applications requiring significant investment (and potential risk) to replatform. Although continued investment in an older, more traditional data center may seem contradictory to current trends, if done wisely, it can yield significant benefits to short- and long-term planning. An emerging factor is also the impact of postpandemic remote work strategies and the desire for those compute environments that are required on-premises to be as contactless as possible.

Data center managers and IT leaders can optimize existing data centers by using one of the three top practices shown in Figure 1. Each will yield a highly efficient data center that will increase capacity, while reducing operating costs.

Figure 1. Three Top Practices for Data Center Optimization

Enhance Delivery

In data centers nearing operational capacity, the primary limiting factor usually revolves around a lack of physical space to place more equipment, power to support additional equipment or adequate cooling infrastructures to maintain the equipment within acceptable operating temperatures. The preferred solution for most data center planners is to build the next-generation data center to support long-term growth, or to use colocation, cloud or hosting services.

These options are viable alternatives; however, they each entail moving workloads away from the traditional on-premises operation, which introduces risk and adds complexity to the operating environment. There is another alternative for long-term upgrades of existing data centers: using self-contained rack solutions.

Self-contained racks are manufactured enclosures that contain a group of racks (e.g., two, four or six) designed to support medium to high compute densities, and will often integrate their own cooling mechanism. Gartner has recommended this approach with many clients, where appropriate, and believes most data center planners should include these retrofit options on the list of candidate solutions for improving data center space.

Self-contained rack solutions from vendors such as Vertiv, Schneider Electric, IBM, Panduit and Rittal were originally designed for use in high-density computing environments. There, intense and focused cooling was needed, or existing infrastructures would not support higher densities without expensive retrofits.

Break Floor Space Into Discrete Sections

The least-intrusive retrofit technique involves clearing out a small section of floor space for one of these self-contained units (perhaps four racks of space). This could be as small as 120 square feet of space and could reside on an existing section of raised floor, or on a slab floor. Depending on the vendor, the self-contained rack unit will require power from an existing power distribution unit (PDU), or, in some cases, a refrigerant or cooling distribution unit will be needed (approximately the size of a rack). Assume an increase in per-rack space of approximately 20% to take into account additional supporting equipment. Because in-rack cooling solutions are self-contained cooling solutions, you do not have to worry about putting new racks in a hot-aisle/cold-aisle configuration or worry about using containment. This will enable more flexibility in the placement of the new racks on the data center floor.

Once the unit is installed, a phased migration of workloads begins from other sections of the floor onto the new racks. This is not a one-for-one migration, because these rack units can support higher densities of cooling (e.g., 20 kilowatts [kW] per rack). Often, an existing data center would average only 50% or 60% of rack capacity utilized, because higher- density racks cause hot spots on the floor, but the new contained racks can easily support higher densities. With all cooling done at the rack level, the amount of workload migrated is often 40% to 50% greater. In this case, a new, self-contained, four-rack unit might absorb the workloads from between six and eight racks on the existing floor.

Enterprises often map this activity to equipment refresh cycles, using new servers and storage form factors to populate self-contained rack environments that maximize rack densities during workload migration.

Reconfigure

Workloads moved to the new enclosure are unlikely to come from the same racks. Consequently, the older section of the server area will be heavily fragmented. The next phase in the project entails defragmenting the environment, moving workloads out of underutilized racks to free up additional floor space for the next self-contained installation. This reconfiguration will take time and affect servers, storage, and networking components and connections. Because it’s happening in a live data center, much of the activity is likely to occur during off-hours or on weekends, so integrating this work into your change control process becomes critical.

Once these workloads are moved, the process of physically relocating equipment and clearing out the next section of floor space begins. This could take weeks to accomplish. Once it’s completed, the next self-contained rack installation begins.

Reconfigure Again

As each subsequent unit is installed, the overall density of computing per rack is much higher than the original data center (since high-density cooling is done at the unit level). The end result is a significantly higher compute-per-square-foot ratio and a smaller overall data center footprint. This creates more room for growth.

Depending on where existing servers are in their economic life cycles, this migration phase might also be an excellent time to consider a server refresh. Implementing smaller server form factors can increase the rack density, while reducing overall power and cooling requirements (relative to the previous generation).

The key to all of this remains the input power to the data center — it must be adequate for the higher-density racks. One offsetting benefit here is that, as more workloads move to these high-density racks, the overall kW of cooling load can actually decrease. This is because much of the cooling air flow is handled inside the rack, thus reducing the amount of air flow needed across the entire data center space.

Maximize Floor Space

The most common top practice for a data center that uses computer room air conditioning (CRAC) units and a raised floor for the cold air plenum involves arranging the equipment racks in a hot-aisle/cold-aisle configuration. This requires that equipment racks be arranged so the cold-air inlet side faces the same way for all racks in a row. Then each row alternates the direction the cold-air inlets face (see Figure 2). This helps air flow, so that the impact of the hot air on the cold air before it can be used to cool the equipment is limited.

Figure 2: Hot-Aisle Containment

With a data center that has been organized around hot and cold aisles, dramatically improved separation of cold supply air and hot exhaust air through containment becomes an option. Another advantage is that efficient air flow management, using cold or hot containment, will increase the utilization of rack densities, thus freeing up additional floor space for future growth.

Hyperconverged infrastructures (HCIs) can also be used as an effective means of space efficiency for applications that are applicable to scale-out approaches.

Reinvent Infrastructure

Although new chip designs attempt to lower the heat footprint of processors, increased computing power requirements are leading to increased equipment densities. These densities increase cooling requirements. With growth in the number of high-density servers, infrastructure and operations (I&O) leaders must provide adequate cooling levels for computer rooms. For those looking to retrofit data centers for extreme densities in a small footprint, perhaps for scientific computing, artificial intelligence (AI) and machine learning (ML), or research organizations, liquid and immersive cooling have become viable options.

The power for cooling a data center can take as much as 60% to 65% of the total power used. Higher-density racks of 15kW to 25kW can often require more than 1.5kW of cooling load for every 1kW of IT load, just to create the cool air flow needed to support those racks. Rear-door heat exchangers (RDHx) are field-replaceable rack doors (in most instances) that cool the hot air as it exits the rack door, rather than relying on air flow in the data center as the solution. One benefit of the RDHx is that, not only do you have more-efficient racks, but much of the power once used for cooling becomes available for reuse by facilities to support other building systems or rerouted as additional IT load. Common RDHx vendors include Futjitsu, Vertiv, Schneider Electric, Nortek Air Solutions, Cool IT Systems and Opticool.

The use of liquid cooling can solve the high-density, server-cooling problem, because water (conductive cooling) conducts more than 3,000 times as much heat as air and requires less energy to do so. Liquid cooling enables the ongoing scalability of computing infrastructure to meet business needs. It may not be obvious that the RDHx solution can save money, so customers must be willing to build the business case. Depending on heat load and energy costs, ROI can be attained within a few years. In many cases, previously planned facilities upgrades (with typical ROIs between 15 and 20 years) may not be required.

More recently, most vendors have started providing a refrigerant solution, instead of water. A low-pressure refrigerant solution can alleviate water leakage concerns, because, if leaks occur, then refrigerants can boil off as nontoxic, noncorrosive gases. Although this may add extra cost for coolant distribution units, it will remove any of the worry of water leaks damaging the equipment.

Immersive cooling systems are also gaining acceptance, especially where self-contained, high-density (40kW to 100kW and beyond) systems are needed. Direct immersion and liquid cooling systems are now available. They can be integrated into existing data centers with air-cooled servers. However, adoption has been slow, considering the heavy investment in mechanical cooling methodologies and the continually improving power efficiency of modern systems. Examples of immersive cooling vendors include Green Revolution Cooling, Iceotope, LiquidCool, TMGCore and Stulz.

Costs and Metrics

Every environment is different, so estimating your benefits from these actions or unique cost savings will require detailed work. We often see cost analyses based on ROI or cost avoidance, rather than a traditional total cost of ownership (TCO) model. For example, we recently spoke with a firm that was planning to retrofit its existing data center or build a new one, but struggling to justify the costs. They chose to reconfigure their existing floor space with self-contained rack environments to maximize vertical density and cooling efficiency. Coupling this with an aggressive technology refresh (using new form factors and lower power to compute ratios), they freed up enough floor space and power to support an additional five years of projected growth. The significant capital expense (capex) of the original plan was replaced by a smaller expense, while reducing energy consumption and overall operating expenditures (opex).

Power Usage Efficiency (PUE) is a common metric developed by The Green Grid. It is used to determine how efficiently a data center uses energy. It is the ratio of total amount of energy used by the data center compared to the energy delivered to computing equipment. The Green Grid has also developed water usage and carbon usage metrics (WUE and CUE, respectively). Although useful from a design perspective, many organizations with older facilities focus on capacity metrics, such as compute per kilowatt or rack density. As a result, they can easily track how efficiently IT is using the existing floor space (and power) in the data center. Gartner has written about a rack density metric called Data Center Space Efficiency (DCSE), which looks at both horizontal (floor) and vertical (rack) space capacities.

Bottom Line

By implementing a phased retrofit, data center managers may attain significant growth in their existing facilities, while reducing the cooling requirements and freeing up power for additional IT workloads. This activity is not without risk, because any physical equipment move within a live production data center is risky. However, if executed as a long-term project, and broken into small, manageable steps, the benefits can far outweigh the risks. From a budgeting point of view, it is also easier to implement, because the capital requirements are spread over multiple quarters, versus a traditional data center build project. In addition, the overall costs will be significantly less than a new build, with many of the same benefits.