This research looks at the benefits of allocating the energy used in data centers to the workloads that run in those data centers. This approach will enable IT organizations to better put a useful value on every kilowatt (kW) of energy consumed.

• Most organizations are measuring data center energy consumption across their hardware.

• Allocating energy to a workload is more complex than hardware energy measurements.

• Workload-based energy management measurement will enable organizations to put a value on every kW of energy they use in their data centers.

• Allocate energy consumed to specific workloads, and correlate with the importance of the workloads to the business.

• Do not focus on an absolute energy measurement per workload; instead, use a consistent process to get a ratio of the energy consumed for a given set of similar workloads.

• Use a method of apportioning fixed energy used and measured energy to get an energy/workload figure.

• Prioritize the applications in a manner similar to the recovery schedule for a disaster recovery/business continuity plan.

• Start using tools from vendors such as HP, 1E and Schneider Electric, which purchased American Power Conversion (APC) in 2006.

This document was revised on 16 September 2010. For more information, see the Corrections page.

An integrated energy management software process and toolset is essential for operating modern data centers, whether they are internal, hosting or cloud service sites. This is because energy consumption is the fastest-growing part of the data center budget and, without proper management, will cause financial and infrastructure deployment problems.

Moreover, the issue is likely to worsen during the next few years, as more organizations grow their technology infrastructures and we come out of this recessionary period. With greater than 5% growth in server shipments per year projected for the next two years, organizations need to "forcefully" control their energy consumption and costs.

"DCIM: Going Beyond IT" shows the benefits of an emerging set of energy management tools called data center infrastructure management (DCIM). These solutions are being developed by various organizations as an essential part of the integrated energy management toolset. "Data Center Infrastructure Management: The Benefits of an Integrated Energy Management Software Approach" shows how such toolset should be deployed. This research illustrates the mechanisms through which IT organizations can begin to develop energy dashboards for their data centers.

As these tools and processes become more pervasive, and the basic energy and carbon-dioxide data is collected, organizations will start to look at the value they gain from every unit of energy that they spend on IT. If a comparison is made with manufacturing industries, many factories measure output and productivity of their industrial building according to the value of the energy used to produce the goods or services. For example, Schneider Electric uses the concept of production energy optimization and has metrics such as energy consumed per unit, which helps users understand where energy use should be used to trigger events.

Gartner has started having conversations with leading-edge data center designers and operators about applying such an approach to energy consumption in data centers. The benefits and issues are outlined in the sections that follow.

Most software tools that measure energy consumption in the data center are based on hardware components. For example, they measure the energy flowing through an uninterruptible power supply (UPS), product data utility (PDU), rack or even a server. This equates to a kilowatt-hour (kWh) rating per hardware device. This provides useful information for overall data center energy improvements, capacity planning and power utilization efficiency (PUE) readings. As a first stage, this is useful; however, in a virtualized world, a single server will have multiple applications running on it. These applications will probably be from different business units because it's prudent to spread applications across a number of hardware boxes to limit the impact of hardware failure on a particular business unit.

There will also be a mix of workload types; some production applications and some development work will be mixed in a single partition. Hence, a hardware-based energy management approach, although extremely useful, has a limited level of granularity. This is especially the case in cloud services, where a number of workloads, potentially from different users, may be running in different virtual machines (VMs) on the same physical server.

Another problem with the hardware approach to energy management is that it is difficult to allocate the energy used by a given application set across the storage and networking devices. Even where software-based energy management tools are built into the hardware (storage devices, routers, switches, etc.) there will be no breakdown of how much energy used by that device should be allocated to a particular workload. An alternative approach is to measure the energy used by workloads.

An emerging concept in data centers, and one that will prove to demonstrate lasting value, is to allocate the energy used to the beneficial work for which that energy is used. The best way to specify the beneficial work is to look at the workload that is run. The workload may be an application, a set of applications or a specific application set of jobs, such as batch processing. With this approach, a data center should be able to state with a reasonable degree of confidence that a particular workload uses a specified amount of energy to run over a 24-hour period. This provides a direct relationship between energy consumed and the application benefits of using that amount of energy. The application in this case is the best proxy for end-user benefits and can be considered analogous to a physical product that a factory may produce.

For example, consider a standard online transaction processing (OLTP) application that is used to access a database and report the answer back to the user. By measuring (where possible) or simply approximating the energy that that application uses (for all users) over a period of time (e.g., 24 hours), the organization derives a more meaningful energy metric than the amount of energy that a particular server uses. It becomes a method to correlate the energy used (and carbon-dioxide expended) directly to the application. The more important the application is to the business, the more energy it should be able to use.

This workload-based approach is used in many other data center processes. The most-business-critical applications get the fastest servers, the best networking routers and, of course, are the first applications to be brought back online after a disaster. Moreover, specific hardware designs are tailored to specific workloads, such as database-intensive or high-transaction applications. In the same way, the most-critical applications should get the highest allocation of energy.

As server-based power capping becomes more prevalent in data centers, this workload approach will become more critical. It's also a prelude to sophisticated energy-based financial accounting for internal data centers, hosting companies and cloud services. In this way, the energy cost is not allocated on hardware basis, but is specifically tied to the application. Hence, if users have applications that are energy-hungry, they will pay more. These may be badly written applications that use too many resources (memory, cache, etc.) or engineering types of applications that simply require a lot of energy.

Allocating energy consumption to applications is not as easy as it may seem. It requires a set of processes that mixes specific costs (where measurable) and a degree of apportionment. In much the same way as activity-based accounting uses fixed and variable costs, workload-based energy consumption will require a degree of financial judgment. Vendors such Schneider, IBM and HP are developing tools to solidify this approach. HP's Power Capping and Smart Grid approaches are good, but the leader is 1E, with its NightWatchman Server Edition tool and its Useful Work process. Gartner has worked with a small number of companies attempting to implement this workload methodology.

The approach taken by one user is to set up a test platform consisting of a virtualized server, network and storage environment. On this system, a single workload is run for a 24-hour period, and the energy consumption is measured. This is then allocated to that workload. All other applications are then run, one by one, and the measurements are taken. This gives the first data set and a ratio of relative amounts of energy consumed. As these applications are run on a shared, virtualized environment, the fixed proportion of the data center's energy use is apportioned in the same ratio as the energy used by the set of applications running on a particular server. Hence, cooling, power loss and general data center energy consumption (PDU, UPS, etc.) are all apportioned. In this way, a model is built up of the energy associated with a particular workload.

Although the absolute accuracy of the energy consumed by a single workload may not be completely measured, it's the relative values across a portfolio of workloads that is important. Correlating this with the importance of the workload is the critical step. Again, as an example, if a physical server is running 10 applications, each in a separate VM, then allocating the energy equally across all 10 VMs may not be the best approach, if one of those applications is significantly more important to the business than the other nine. In this case, a higher ratio of energy should be related to the more valuable application, through server power-capping tools or by moving that application into a physical machine where more energy is available.

Users will gain a number of benefits from taking a workload-based approach to data center energy management. Some of these will be long-term and may take between five and seven years to fully mature.

Many operations tools focus on workloads, so a maturing of the energy management toolset will bring convergence. For example, application-monitoring and performance, workload-balancing and job-scheduling/batch-processing tools will eventually be tied into the workload-based energy management toolset. Although this will take more than five years to happen, it will yield benefits in energy-based chargeback and general internal cost allocation to different business units with different sets of applications.

The workload approach will enable data center operations groups to schedule the running of applications based on energy cost and availability. Hence, if a company has a number of data centers around the world, then it may well be possible to run a set of applications in a "follow the moon" mode. Because energy tariffs usually drop at night, the organization would simply run the applications in regions where it's night. This may not be appropriate for all applications; however, for many, any slight increase in latency or performance drop could be justified by a lower cost for running the applications.

For organizations using cloud service providers, the cost of running the applications has a general energy component built in. This will not typically be visible; it will be an apportioned cost. By linking energy usage to specific workloads, providers and consumers will get greater transparency of the energy costs associated with each workload. This is a necessary step to ensure that the financial models around external cloud services are robust enough to deal a large range of different application types. Furthermore, where workloads fluctuate a great deal, the energy associated with the workload can be used for modeling and financial metrics. For example, Google measures milliwatts per query, and this sort of approach can be tailored for cloud service use.

Another possible long-term benefit of workload-based energy tools is that they will enable users to compare the efficiency of software. In general, well-written applications are optimized to use minimal resources (memory, cache, registers, etc.) for a given set of transactions. Hence, these applications should have a relatively low energy use measurement. However, when applications are badly written, they will have a high energy use measurement. In this way, similar applications can be compared before purchase to work out the total cost of ownership involved with running the applications. Perhaps, this could lead to some form of licensing model that takes code efficiency into account.

Allocating energy to a workload and then prioritizing the workloads gives a clear picture of the usefulness every kilowatt of energy the data center consumes. As tools enter this market, we encourage organizations to start enhancing their data center processes to adopt this approach. This is especially true for those companies developing internal clouds or planning to use hosting services.