Solution Comparison for Public Cloud Third-Party Cost Optimization Tools

Strengths

• The combination of Cloudability and CBM presents a large breadth of functions, which allows Apptio to provide at least the minimum requirements in every area of cost optimization considered in this assessment except object storage optimization.

• Apptio provides more transparency than other solutions assessed on how it achieves certain functionality such as rightsizing or reservation purchase management. Public blogs and articles showcase the complex decision framework that allows Cloudability to produce high-quality recommendations.

• Apptio provides an array of widely adopted ITFM tools. Organizations selecting Apptio can count on a vendor with significant experience in managing the costs of IT and their business implications.

• Organizations that already use Apptio for managing their IT financials can easily extend their use to embrace public cloud cost management. Apptio’s strategic acquisitions allowed the vendor to quickly ramp up its public cloud cost management functionality and make it comparable to other leading vendors in this space.

Weaknesses

• Apptio provides multiple solutions with a high degree of overlap. Despite Apptio’s direction being clearly focused on Cloudability, it will take some time before the vendor can effectively incorporate the CBM functionality within Cloudability and supply its customers with a single comprehensive solution. Furthermore, history shows that, post-acquisition, vendors direct their investment to the integration efforts and away from building new features to keep their product competitive. History also show that some functionality may be eventually left behind when it is difficult to integrate. Therefore, there is no guarantee that Apptio will preserve all functionality of both solutions once the integration is completed.

• Apptio provides strong support for AWS but only minimal support for Microsoft Azure, GCP and on-premises stacks. This makes the multicloud management value proposition of Apptio weaker than vendors with higher feature parity between cloud platforms.

• Apptio provides no public documentation for CBM, making it difficult for its customers to research and assess its features. Furthermore, CBM is no longer represented within Apptio’s marketing website. Despite the product having significantly contributed to the scores in this assessment, the vendor actively sells it only in limited scenarios (specifically, only where FedRAMP certification is required). As a consequence, the majority of customers may not be able to buy CBM functionality until the integration with Cloudability is completed.

• Apptio Cloudability offers minimal configuration options to customize the policies that power the recommendation engine. Although the built-in policies are certainly applicable to the majority of use cases, customer organizations would benefit from a higher degree of configurability to handle exceptions and minimize false positives.

Strengths

• CloudCheckr provides advanced functionality for decommissioning old Amazon EBS snapshots. The functionality meets all requirements of the “old snapshot decommissioning” criterion except for the indication of estimated savings. However, customers can find cost information for each existing snapshot in the product’s inventory report.

• CloudCheckr provides strong support for reservation portfolio management. Its RI “rebalancer” provides actionable recommendations on how to modify existing AWS reservation to maximize coverage and minimize waste.

• CloudCheckr’s utilization heatmap for compute instances helps organizations visually identify utilization patterns and use those patterns to plan for scheduled shutdowns. However, CloudCheckr doesn’t provide recommendations for how to implement schedules.

• CloudCheckr is a cloud management tool that goes beyond cost management. CloudCheckr provides more than out-of-the-box 500 best practices for cost and security concerns. This large number allows customers to quickly get their cloud deployments in control with minimal configuration burden. Its security capabilities allow customers to use the same tool to also enforce a security baseline and manage broader cloud governance, especially where cost and security intersect. This makes CloudCheckr a good choice for organizations that prioritize minimizing the number of management tools.

Weaknesses

• CloudCheckr scores low in the majority of the assessment criteria except for old snapshot decommissioning and reservation portfolio management (both features are available only for AWS). The majority of its cost optimization functionality is basic and fails to provide the depth that organizations require to maximize savings while preserving performance. CloudCheckr’s breadth of features, even beyond cost optimization, seems to be traded with lower depth in the cost optimization space.

• CloudCheckr’s approach to policy-based management focuses primarily on reporting issues rather than solutions. With the exception of its rightsizing and snapshot cleanup functionality, CloudCheckr’s “best practice checks” do not include the indication of the recommended actions that customers should undertake to address the issues. This approach does not allow CloudCheckr to meet the minimum requirements in some of the criteria in this assessment.

• CloudCheckr does not provide any cost optimization capability for GCP or on-premises cloud platforms. Organizations should not select CloudCheckr if either of these is their primary cloud platform.

• CloudCheckr’s public documentation has proven to be incomplete. Specifically, most of the documentation for Azure-specific capabilities is missing. Only a few pages mention the availability of certain functions for Azure, but they do not provide any detail. This lack of documentation makes it difficult for organization to assess the available functionality prior to the adoption of the tool.

Strengths

• Flexera Optima is highly programmable and provides a proprietary policy language that customers can use to extend the product’s functionality. This extensibility comes with a comprehensive developer’s guide that makes developing extensions an approachable task. This capability allows client organizations to potentially support a larger set of cloud services than those included by default, without having to wait on Flexera to do that.

• Flexera Optima provides great feature parity among supported cloud platforms. Flexera is the only vendor that scores “High” for AWS, Microsoft Azure and GCP support. Furthermore, it is the only vendor that supports cost optimization for GCP for more than 75% of its functionality.

• Flexera is the leading vendor within the SAM space, according to Before the acquisition, RightScale was the leading vendor in the CMP space, according to The combination of two vendors that know how to establish leadership positions can certainly increase confidence in the resulting organization. Furthermore, Flexera’s latest acquisition of RISC Networks in June 2019 and the addition of cloud migration functionality has further enhanced the vendor’s cloud management portfolio. This makes Flexera a good choice for organizations that want to minimize the number of vendors in their cloud management strategy.

• Flexera Optima’s documentation is excellent. It provides a dedicated documentation website that is comprehensive, well-organized and easy to navigate. Customers can easily find the information they need to assess product features or to learn how to operate the solution.

Weaknesses

• Flexera Optima does not provide the analysis of cost implications for most of its recommendations (with the exception of unused Amazon EBS volumes). Information such as estimated savings or additional expenditure is fundamental to help organizations decide whether to act on the suggested actions. Most of the criteria used in this assessment require the indication of estimated savings to allow a vendor to earn a “Medium” score.

• The provided cost optimization policies are just a handful, and not all of them include the ability to execute on recommendations. Many of the criteria within this assessment could be met by developing the required code, but this effort is left to the customer. This is possibly due to the vendor’s initial strategy of providing Flexera Optima in conjunction with professional services.

• Flexera Optima does not provide any reservation-related recommendation. Its reservation policies simply retrieve recommendations from AWS and Microsoft Azure and native cost optimization tools. With such an approach, it is the cloud provider that evaluates resource utilization and produces insights, while Flexera Optima simply retrieves and aggregates this information.

• Flexera lacks functionality such as rightsizing of block storage volumes or software infrastructure services other than databases. Client organizations can implement such functionality only through the development of a custom policy.

Strengths

• Turbonomic automation capabilities are superior to most of the other vendors assessed. Customers can automate the execution of recommended actions either directly or through an integration with external orchestration tools such as Ansible, ServiceNow (for reservation purchase approvals) or Terraform.

• Turbonomic is the only vendor that provides feature parity between some public cloud platforms (namely AWS and Microsoft Azure) and on-premises environments through the same tool. However, this parity doesn’t apply to GCP, and it excludes the resource scheduling capabilities provided by ParkMyCloud.

• Turbonomic is very focused on the mission of providing automated application resource management. For Turbonomic, cost optimization is just a benefit of the perfect alignment between infrastructure resource supply and performance requirements. This focus allows Turbonomic to achieve excellence in what it does and earn the highest score in some of this assessment’s criteria.

• ParkMyCloud provides the most advanced resource scheduling capabilities. The acquisition of ParkMyCloud allowed Turbonomic to grow its functionality in line with its philosophy of staying focused on a few key areas and achieving excellence in each of them.

Weaknesses

• Turbonomic does not address some important areas of cost optimization. It does not address the decommissioning of unused resources other than compute instances and storage volumes. For example, it does not provide recommendations to decommission unused database services, load balancers, IP addresses or old snapshots.

• Turbonomic does not provide reservation portfolio management. When using reservations, customers must continually evaluate their reservation coverage and change modifiable attributes, such as reserved allocation size, to maximize coverage and minimize waste. Turbonomic does not provide functionality to assess coverage and modify or exchange purchased reservations.

• Turbonomic’s support for GCP is minimal. Although ParkMyCloud fully supports GCP for resource scheduling, the Turbonomic Platform provides minimal cost optimization for GCP. Specifically, the platform provides rightsizing and decommissioning for GKE clusters, as part of the platform’s functionality for Kubernetes and container management.

• Turbonomic does not go beyond resource optimization. It does not address broader aspects of cost management such as budgeting, tracking or reporting. When using Turbonomic, organizations need to adopt other tools to address all of their cost management requirements.

Strengths

• CloudHealth’s new rightsizing engine provides a highly configurable policy that customers can use to define an “efficiency target.” At the time of writing, CloudHealth uses this new engine only for Microsoft Azure SQL Database rightsizing. However, the vendor expects to extend it to all supported resource types in the near future.

• CloudHealth excels in reservation purchase and portfolio management for AWS, meeting all the requirements in this assessment. AWS clients using CloudHealth can count on mature functionality to suggest purchase and subsequent ongoing management of reservations.

• VMware provides strong resource optimization for on-premises environments through vRealize Operations. Sometimes, like in the case of compute instance rightsizing, vRealize Operations exceeds the capabilities that CloudHealth provides for public cloud services.

• Between internal development and acquisitions, VMware has built a broad portfolio of public cloud management tools. Customers can select CloudHealth for cost management in addition to Secure State (cloud security), Cloud Assembly (provisioning and orchestration), Code Stream (continuous integration/continuous delivery [CI/CD]) and Service Broker (service request management). This makes VMware a good choice for organizations that want to minimize the number of vendors in their cloud management strategies.

Weaknesses

• CloudHealth provides uneven breadth of functionality for each supported public cloud platform. Although support for AWS is strong, with nearly all functionality available, less than 75% of its functionality is available for Microsoft Azure. Furthermore, some functionality available for Microsoft Azure is not available for AWS (such as database as a service rightsizing). Cost optimization support for GCP is negligible. Such uneven support makes the multicloud management value proposition of VMware weaker than vendors with higher feature parity between cloud platforms.

• CloudHealth’s built-in actions that execute and automate optimization opportunities are limited. Missing actions include the ability to change size for storage volumes and the ability to change size or decommission software infrastructure services such as databases. Furthermore, the list of available actions differs significantly for each supported cloud platform.

• CloudHealth does not provide resource scheduling recommendations. Although it does provide a scheduling engine, it’s up to the user to decide which schedules to apply to which resource, and CloudHealth provides no help with this decision.

• VMware provides on-premises resource optimization through a separate product (vRealize Operations), which is only minimally integrated with CloudHealth. The two products have separate identity management, data models and user interfaces. The only available integration allows vRealize Operations to visualize dashboards by pulling data from CloudHealth. As a consequence, organizations that want a hybrid cost management solution through a single vendor still have to deal with two separate tools.

Definitions

This solution comparison makes use of the following terms:

• Rightsizing: The action of adjusting the size of a compute instance or a software infrastructure service allocation to match the actual performance demand. Rightsizing can happen in both directions: “down” (for smaller recommended sizes) or “up” (for bigger recommended sizes).

• Downsizing: Rightsizing action in the “down” direction (for smaller recommended sizes).

• Upsizing: Rightsizing action in the “up” direction (for bigger recommended sizes).

• Same-sizing: Rightsizing action that doesn’t actually include the change of resource size. Same-sizing is typically performed to change other resource parameters that may affect cost, such as a cloud provider’s subregion.

• Subregions: Distinct cloud provider’s regions that are geographically close enough to deliver a cloud service with equivalent performance and latency but may have different costs associated. For example, AWS US-West-1 (N. California) and US-West-2 (Oregon).

• Utilization policy: The utilization-based conditions that determine when a cost optimization opportunity should be pursued. Utilization policies can be based on multiple metrics such CPU, memory, storage space or hours of running time.

• Observation period: The time period during which a resource is observed and its utilization is tracked to match it against the utilization policy.

• Risk profile: A combination of utilization policy attributes that determines the risk of producing a wrong recommendation. Higher risk profiles can lead to more cost benefits but may affect performance (like in the case of rightsizing) or may generate spending waste (like in the case of reservations).

• Software infrastructure services: Software infrastructure includes databases, middleware and other infrastructure elements typically delivered as software. Some of these services may be classifiable as PaaS. Some examples of software infrastructure services are Amazon ElastiCache, Azure Load Balancer and Google Cloud Spanner.

• Allocation-based software infrastructure services: A software infrastructure service for which end users must specify an allocation size at the time of resource provisioning. End users pay for these services on the basis of the provisioned allocation, regardless of the service utilization. For example, Amazon RDS is allocation-based, while AWS ELB is not.

• Scheduling engine: A cron-like engine that can be configured to issue specific cloud operations (such as resource start/stop) at certain date and time.

• Warmer class: A cloud provider’s object storage class that is designed for more frequent access to the objects. Warmer classes normally deliver faster retrieval time but are more expensive than colder classes.

• Colder class: A cloud provider’s object storage class that is designed for less frequent access to the objects. Colder classes normally deliver slower retrieval time but are less expensive than warmer classes.

• On-premises cloud platform: An on-premises environment that supports the dynamic provisioning of (at a minimum) compute, network and storage resources through an API. VMware vSphere and Microsoft Azure Stack are examples of on-premises cloud platforms.

Criteria

This solution comparison is based on 16 criteria grouped into four categories. Gartner’s defined set of criteria is based on common topics of discussion on public cloud cost-optimization and related inquiry calls, in concert with what Gartner believes is important for organizations evaluating ways to optimize their public cloud spend.

Cost optimization tools are part of the larger category of analytics tools. The value of analytics tools grows as they take on more responsibilities of the processes that lead from data to actions. A basic analytic tool will tell you what happened. An advanced one will tell you what you should do about it and automate the entire decision process.

Gartner used a similar framework to distribute the features that allow vendors to move between the scores of “Low,” “Medium” and “High” based on the growing value provided by the vendor’s solution. Additionally, with some exceptions, Gartner used the following scoring principles:

• Recommendations must be original and independent. Simply collecting information from other sources, such as the provider’s API (e.g., AWS Trusted Advisor or Microsoft Azure Advisor), does not qualify for the criteria in this assessment.

• The configurability of the utilization policy that drives the identification of optimization opportunities is often required to earn a “Medium” score. Although vendors apply default values to their policies that cover the majority of use cases, customers may require the customization of certain utilization parameters to fine-tune the analytics engine.

• The ability to display cost implications of identified optimization opportunities is often required to earn a “Medium” score. Information such as estimated savings or additional expenditure is fundamental to help organizations decide whether to act on the provided recommendations.

• The ability to execute actions with “one click” directly from the console of the cost optimization tool is often required to earn a “Medium” score.

• Criteria that relate to compute instances are often more “prescriptive” than criteria for software infrastructure services, storage or other resources. This is due to the higher maturity of the IaaS market and the more sophisticated management requirements that organizations have in this area.

• Functionality for software infrastructure services often requires specific support for database solutions to earn a “Low” score and for caching or container orchestration services to earn higher scores.

• The availability of parameters that can be used to define a risk profile is often required to earn a “High” score. Such parameters allow the tool to produce more or less aggressive recommendations that prioritize savings over performance or vice versa.

• The ability to automate optimization actions is often required to earn a “High” score. To meet these automation criteria, customers must be able to configure the solution to execute actions automatically without user interaction. It is not sufficient for vendors to provide the availability of actions in their API. They must orchestrate the entire automation workflow.

This assessment exclusively considers features that vendors provide out-of-the-box and that are accessible via the tool’s point-and-click interface. This methodology excludes all features that can only be accomplished through the development of code such as an AWS Lambda function or similar. The exclusion also applies to features that can only be accomplished by modifying lines of vendor-provided code.

The criteria have been developed to evaluate vendor support for the following capabilities:

• Compute instance rightsizing: This criterion assesses recommendations, execution and automation of rightsizing for compute instances. The assessed functionality includes downsizing, upsizing, utilization policy configuration (including externally injected metrics), cross-family and cross-region rightsizing.

• Block storage rightsizing: This criterion assesses recommendations, execution and automation of rightsizing for block storage volumes. The assessed functionality includes size-based and IOPS-based volume type changes and utilization policy configuration.

• Software infrastructure services rightsizing: This criterion assesses recommendations, execution and automation of rightsizing for allocation-based software infrastructure services. Assessed services include database solutions, caching services and container orchestration services.

• Idle compute instance decommissioning: This criterion assesses recommendations, execution and automation of the decommissioning of idle compute instances. The assessed functionality includes the configuration of utilization metrics to define the “idle” state.

• Unused storage decommissioning: This criterion assesses recommendations, execution and automation of the decommissioning of unused block storage volumes. The assessed functionality includes the configuration of when a storage volume should be considered “unused” (i.e., the number of days it has been unattached to instances).

• Old snapshot decommissioning: This criterion assesses recommendations, execution and automation of the decommissioning of old snapshots. The assessed functionality includes the configuration of when snapshots should be considered “old” (i.e., the number of days from creation).

• Unused service decommissioning: This criterion assesses recommendations, execution and automation of the decommissioning of provisioned capacity for allocation-based software infrastructure services that are not being used. Assessed services include database solutions, caching services and container orchestration services.

• Unused resource decommissioning: This criterion assesses recommendations, execution and automation of the decommissioning of provisioned resources that are not being used. Assessed resource types include IP addresses, network gateways and load balancers.

• Resource scheduling: This criterion assesses the ability to determine and configure resource scheduling. The assessed functionality includes the identification of utilization patterns and the schedule configuration using an internal or external scheduling engine. In addition, this criterion assesses auto shutdowns (after a time from the start), resource grouping, dependency support and the ability to execute arbitrary commands before and after start/stop operations.

• Reservation purchase management: This criterion assesses the ability to recommend and execute the purchase of reservations such as Amazon EC2 RI, Google CUDs and Microsoft Azure RIs. The assessed functionality includes the configuration of the utilization policy, the definition of a risk profile and the ability to handle reservations for software infrastructure services (when these are supported by the provider).

• Reservation portfolio management: This criterion assesses the ability to modify the flexible attributes of purchased reservations (when these are supported by the provider). The assessed functionality includes reservation expiration alerts, backward analysis of past coverage and waste and the ability to automate the modifications.

• Object storage optimization: This criterion assesses the ability to move storage objects between available classes based on time or utilization conditions. The assessed functionality includes the movement within three storage classes (hot, warm and cold) based on access patterns and the ability to automate the movement in all directions.

• AWS support: This criterion assesses the solution’s functionality coverage for AWS. To achieve higher scores, the solution must provide at least 50% (for a “Medium” score) or 75% (for a “High” score) of its features for AWS.

• Microsoft Azure support: This criterion assesses the solution’s functionality coverage for Microsoft Azure. To achieve higher scores, the solution must provide at least 50% (for a “Medium” score) or 75% (for a “High” score) of its features for Microsoft Azure.

• GCP support: This criterion assesses the solution’s functionality coverage for GCP. To achieve higher scores, the solution must provide at least 50% (for a “Medium” score) or 75% (for a “High” score) of its features for GCP.

• On-premises cloud platform support: This criterion assessed the solution’s functionality coverage for one or more on-premises cloud platforms such as VMware vSphere. Because not all of the criteria are applicable to on-premises environments, only the capabilities marked as “[OP]” in Table 6 are considered for this score. To achieve higher scores, the solution must provide at least 50% (for a “Medium” score) or 75% (for a “High” score) of its features (also marked as “[OP]”) for an on-premises cloud platform.

Each criterion corresponds to a set of features or capabilities that solutions must implement to attain a particular rating for that criterion. The rating for each criterion is based on compliance with the attributes that belong to a particular rating value. The reader should note that the criteria’s attributes are cumulative. To achieve a higher rating, a solution must implement the features or capabilities required by the preceding rating and then add additional ones.

Table 6 provides the definitions for each rating for each criterion evaluated in this Solution Comparison. Attributes marked as “[OP]” in the following table are those relevant for the “on-premises cloud platform support” criterion.