How Markets and Vendors Are Evaluated in Gartner Magic Quadrants


Archived Published: 22 July 2014 ID: G00266763

Analyst(s): |

  Free preview of Gartner research

Summary

Magic Quadrants offer visual snapshots, in-depth analyses and actionable advice that provide insight into a market's direction, maturity and participants. Understanding our methodology will help you when evaluating a market, choosing a technology or service provider or managing vendor relationships.

Overview

Key Findings

  • Gartner Magic Quadrants are based on rigorous analysis backed up by a highly structured methodology.

  • Your interpretation and understanding of a Gartner Magic Quadrant is crucial. It will enable you to get the most from the market analysis in alignment with your unique business and technology needs.

Recommendations

  • Read this document and gain insight as to how Gartner can help you evaluate markets and the technology and service providers in those markets to support your evaluation and investment decisions.

  • Read this document in conjunction with a Magic Quadrant to provide context to the analysis.

  • Consult a Magic Quadrant to narrow your list of choices, but don't base your decision on the model only.

  • Consider scheduling an inquiry session with a Gartner analyst who specializes in the market to gain further details and insight.

Analysis

Assessing the market and its technology and service providers is a business-critical task. Vendor differentiation based on differing company size, level of complexity and strategies can make it challenging to compare vendor offerings. In addition, the market's overall direction is often unclear.

Gartner's Magic Quadrants solve these problems by offering snapshots of markets and their participants. These documents enable you to map vendor strengths against your current and future needs.

Where appropriate, the Magic Quadrant will provide "Additional Perspectives" based on market differentiators identified as important to consider when selecting technology and service providers. These perspectives will provide context for either a specified geography, industry or company size depending on the market in question.

Alternatively, Gartner may provide insights about a specific market with a Gartner Market Guide rather than a Magic Quadrant note.

Distinguish Between Gartner Magic Quadrant and Market Guide Research

Markets vary in many ways, but all follow a predictable life cycle with five phases:

  • Emerging

  • High growth

  • Consolidating

  • Maturity

  • Declining

When a market emerges, it lacks definition, and typically has a multitude of vendors offering point solutions, and it is not always possible to comparatively assess vendors. Low barriers to entry mean that many vendors can enter the market without significant investment, and they can leave without significant implication for their broader business too. Similarly, when the market is highly mature, the services and products on offer will typically be provided by only a few vendors and can lack differentiation in solution features or service. In both these cases, a Gartner Market Guide provides a meaningful perspective of the underlying market, without a deep evaluation of vendor products or services.

Magic Quadrants provide a graphical comparative positioning of technology and service providers where market growth is high and provider differentiation is distinct. Magic Quadrants depict markets in the middle phases of their life cycle by using a two-dimensional matrix that evaluates vendors based on their "Completeness of Vision" and "Ability to Execute." The Magic Quadrant has 15 weighted criteria that are used plot vendors on the graph shown in Figure 1 based on their relative strengths in the market.

Understanding Gartner's Qualitative Market Analysis

To evaluate vendors for Magic Quadrants, Gartner uses a comprehensive process that defines the boundaries of the market, research focus and the steps taken to form the analysis.

Identifying the Market

To be considered for a Magic Quadrant, a market must be distinct and viable. Gartner selects a market for analysis based on the impact of emerging trends and users' need to understand changing market dynamics. Gartner analysts focus on markets in which their insights can assist clients with planning, investment decision making and the ongoing support of vendor relationships.

Selecting Vendors

A Magic Quadrant is not intended to be an exhaustive analysis of every vendor in a market, but rather a focused analysis. The criteria for inclusion may consist of market share, revenue, number of clients, installed base, types of products/services, target market or other defining characteristics. These criteria help narrow the scope of the research to those vendors that Gartner considers to be the most important — or best suited to the evolving needs of Gartner's clients as buyers in the market.

Consideration is given to vendors that cannot meet the broad Magic Quadrant inclusion criteria. Occasionally, vendors of particular note are recognized for their ability to meet specific business needs and are included in the "Honorable Mentions" section of the report. Vendors with notable offerings in a specific submarket may also be highlighted in a Magic Quadrant additional perspective.

Defining the Rating Criteria

Magic Quadrants use standard criteria in two categories: Completeness of Vision and Ability to Execute (see Figure 1). We then adapt the rating criteria to a market by prioritizing and weighting them based on a high, low or standard scale of importance. In some cases, a criterion may have a "no rating" weight because it has low relevance for the market or does not provide sufficient differentiation.

Completeness of Vision

The category includes these criteria:

  • Market understanding: The ability of a vendor to understand buyer needs and translate these needs into products and services. A vendor that shows the highest degree of vision listens and understands what a buyer wants and needs, and can use that information to shape or enhance the relationship.

  • Marketing strategy: A clear, differentiated set of messages consistently communicated throughout the organization and publicized through online presence, advertising, customer programs, events and positioning statements.

  • Sales strategy: A strategy for selling products that uses the appropriate network of direct and indirect sales, marketing, service and communication affiliates to extend the scope and depth of a vendor's market reach, skills, expertise, technologies, services and customer base.

  • Offering (product) strategy: A vendor's approach to product development and delivery that emphasizes differentiation, functions, methodology and feature set in relation to current and future requirements.

  • Business model: The validity and logic of a vendor's underlying business proposition.

  • Vertical/industry strategy: A vendor's strategy to direct resources, skills and offerings to meet the needs of market segments, including vertical industries.

  • Innovation: Marshaling of resources, expertise or capital for competitive advantage, investment, consolidation or defense against acquisition.

  • Geographic strategy: A vendor's strategy to direct resources, skills and offerings to meet the needs of regions beyond the vendor's "home" or native area, directly or through partners, channels and subsidiaries, as appropriate for that region and market.

Ability to Execute

This category includes these criteria:

  • Products/services: Core goods and services offered by the vendor that compete in and serve the market. This category includes product and service capabilities, quality, feature sets and skills (offered natively or through original equipment manufacturers) as defined in the market definition and may be further detailed by other criteria.

  • Overall viability: Includes an assessment of the vendor's overall financial health, the financial and practical success of the relevant business unit, and the likelihood of that business unit to continue to invest in and offer the product within the vendor's product portfolio.

  • Sales execution/pricing: The vendor's capabilities in presales activities and the structure that supports them. This criterion includes deal management, pricing and negotiation, presales support and the overall effectiveness of the sales channel.

  • Market responsiveness and track record: Ability to respond, change direction, be flexible and achieve competitive success as opportunities develop, competitors act, customer needs evolve and market dynamics change. This criterion also considers how responsive the vendor has been over time.

  • Marketing execution: The clarity, quality, creativity and efficacy of programs designed to deliver the vendor's message to influence the market, promote its brand and business, increase awareness of its products and services, and establish a positive identification with the product, brand or vendor with buyers. A combination of publicity, promotions, thought leadership, word of mouth and sales activities can drive this mind share.

  • Customer experience: Relationships, products, and services and programs that enable clients to succeed with the products evaluated. This criterion includes the ways customers receive technical support or account support. It can also include ancillary tools, customer support programs (and their quality), availability of user groups and service-level agreements.

  • Operations: The vendor's ability to meet its goals and commitments. Factors include the quality of the organizational structure, such as skills, experiences, programs, systems and other vehicles that enable the vendor to operate effectively and efficiently.

Researching the Market

Research activities include, but are not limited to:

  • Vendor briefings

  • Surveys

  • Vendor-provided references

  • Industry contacts

  • Client interviews

  • Public sources, such as U.S. Securities and Exchange Commission filings, articles, speeches and published papers

  • Input from other Gartner analysts

Evaluating Vendors

Teams of analysts collaborate to evaluate and rate each vendor against each of the weighted criteria. The resulting scores are used to generate a Magic Quadrant.

Publishing the Research

The Magic Quadrant is published as a research document that explains the vendor positions and ratings, as well as new developments in the market. As a result, these documents provide a context in which to use the models. During this step, the research undergoes rigorous internal peer review and validation, and a factual review by the vendors included in the research.

The Structure of a Gartner Magic Quadrant

The Magic Quadrant graph (see Figure 1) is based on two axes:

  • Ability to Execute: Summarizes factors such as the vendor's financial viability, market responsiveness, product development, sales channels and customer base.

  • Completeness of Vision: Reflects the vendor's innovation, whether the vendor drives or follows the market, and if the vendor's view of how the market will develop matches Gartner's perspective.

Figure 1. The Magic Quadrant
Research image courtesy of Gartner, Inc.

Source: Gartner (July 2014)

Understand the Four Quadrants

The vendors positioned in the four quadrants — Leaders, Challengers, Visionaries and Niche Players — share certain characteristics.

Leaders

Leaders provide mature offerings that meet market demand and have demonstrated the vision necessary to sustain their market position as requirements evolve. The hallmark of leaders is that they focus and invest in their offerings to the point that they lead the market and can affect its overall direction. As a result, leaders can become the vendors to watch as you try to understand how new offerings might evolve.

Leaders typically possess a large, satisfied customer base (relative to the size of the market) and enjoy high visibility within the market. Their size and financial strength enable them to remain viable in a challenging economy.

Leaders typically respond to a wide market audience by supporting broad market requirements. However, they may fail to meet the specific needs of vertical markets or other more specialized segments.

Challengers

Challengers have a strong ability to execute but may not have a plan that will maintain a strong value proposition for new customers. Larger vendors in mature markets may often be positioned as Challengers because they choose to minimize risk or avoid disrupting their customers or their own activities.

Although Challengers typically have significant size and financial resources, they may lack a strong vision, innovation or overall understanding of market needs. In some cases, Challengers may offer products nearing the end of their lives that dominate a large but shrinking segment.

Challengers can become leaders if their vision develops. Over time, large companies may fluctuate between the Challengers and Leaders quadrants as their product cycles and market needs shift.

Visionaries

Visionaries align with Gartner's view of how a market will evolve, but they have less proven capabilities to deliver against that vision. In early markets, this status is normal. In more mature markets, it may reflect a competitive strategy for a smaller vendor — such as selling an innovation ahead of mainstream demand — or a larger vendor trying to break out of a rut or differentiate.

For vendors and customers, Visionaries fall in the higher risk/higher reward category. They often introduce new technology, services or business models, and they may need to build financial strength, service and support, and sales and distribution channels. Whether Visionaries become Challengers or Leaders may depend on if companies accept the new technology or if the vendors can develop partnerships that complement their strengths. Visionaries are sometimes attractive acquisition targets for Leaders or Challengers.

Niche Players

Niche Players do well in a segment of a market, or they have a limited ability to innovate or outperform other vendors in the wide market view. This may be because they focus on a functionality or geographic region, or they are new entrants to the market. Alternatively, they may be struggling to remain relevant in a market that is moving away from them. Niche Players may have reasonably broad functionality but with limited implementation and support capabilities, and relatively limited customer bases. They have not yet established a strong vision for their offerings.

Assessing Niche Players is more challenging than assessing vendors in other quadrants because some could make progress, while others do not execute well and may not have the vision to keep pace with broader market demands.

A Niche Player may be a perfect fit for your requirements. However, if it goes against the direction of the market — even if you like what it offers — then it may be a risky choice because its long-term viability will be threatened.

Using a Magic Quadrant for Your Requirements

Your needs and circumstances should determine how you use the Magic Quadrant, not the other way around. To evaluate vendors in the Leaders quadrant only, and ignore those in other quadrants, is risky. As a result, Gartner discourages this practice. For example, a vendor in the Niche Players quadrant could offer functions that are ideally suited to your needs. Similarly, a Leader may not offer functions that meet your requirements — for example, its offerings may cost more than its competitors, or it may not support your region or industry.

A vendor's position on a Magic Quadrant illustrates useful information about that vendor. For example, if you want to make a strategic investment in a technology, then a vendor's viability will be critical. Therefore, weigh a vendor's ability to execute more heavily than its Completeness of Vision, and evaluate Challengers before Visionaries. Conversely, if you can gain a competitive advantage by investing in an emerging technology, then evaluate Visionaries before Challengers.

The comparative positioning of all of the vendors in a Magic Quadrant offers a high-level view of the market. In an established, mature market, a preponderance of vendors may cluster in the Leaders quadrant because most vendors will tend to offer a complete set of products and services, with little differentiation. By contrast, in a growing market, most vendors will cluster in the Niche Players quadrant, and most offerings will be incomplete, yet distinctive.

Gartner's interactive Magic Quadrant features (available to entitled Gartner clients) enable you to get additional insight into a market and its vendors. You can use these features to tailor your view of the Magic Quadrant in alignment with your business goals, needs and priorities.

Using the interactive features, you can adjust the weightings applied to each of the evaluation criteria to generate a new Magic Quadrant graphic for that market that is tailored to your own requirements. You can then save and share these customized Magic Quadrants for your internal analyses and decision making. A client-customized view of the Magic Quadrant is intended for you to gain relevant insight into a market and does not represent Gartner's view of the market. Client-customized views of the Magic Quadrant are for internal use only and are subject to standard Gartner usage guidelines.

Some Magic Quadrants will include Additional Perspectives that emphasize what is distinct in select markets where differentiation is relevant and provides unique considerations for technology and service selection for the given context. The Notable Vendors included are a sampling of vendors in the given market that clients should consider that may or may not have been included in the full Magic Quadrant.

Frequently Asked Questions

Why does the analyst create a Magic Quadrant instead of a Market Guide? The lead analyst makes the recommendation on which research type is most appropriate to the market under consideration. By the very nature of these markets, determining the growth and maturity point of a rapidly evolving market is not an exact science. The key characteristics of the market outlined earlier in this document are all considered, and the decision is made using that information.

What is a Gartner Critical Capabilities document? Critical Capabilities is an essential companion to the Magic Quadrant. A Critical Capabilities helps Gartner clients gain a view of the positioning of vendors product and service offerings, allowing comparison against a critical set of differentiators to support their strategic technology investment decisions. It shows which products or services are the best fit in various use cases and provides actionable advice to end-user clients on which products/services they should add to their vendor shortlists for further evaluation.

How Does a Gartner Critical Capabilities differ from a Magic Quadrant? Critical Capabilities is a companion note to the Magic Quadrant. The Magic Quadrant positions vendors in a market, rather than products or services by usage scenario.

Can a Magic Quadrant be used as a vendor selection tool? A Magic Quadrant analyzes a market and can help you to focus your search on important criteria. However, it will not provide the details needed during the vendor selection process to align your requirements to a particular vendor. Gartner provides an extensive range of complementary research — such as Critical Capabilities and Vendor Ratings — that can assist you further in your vendor selection process. Analyst inquiries can also help during the vendor selection phase.

Why aren't all vendors from a market included in a Magic Quadrant? In most cases, it would be impossible to cover every vendor in a market — there are too many, or some are irrelevant based on the criteria. Therefore, Gartner focuses its research in a way that will provide the greatest value to Gartner clients.

Does a vendor's status as a Gartner client influence whether it appears in a Magic Quadrant? No. Sales contracts, vendor requests and client engagements do not influence or drive inclusion in Magic Quadrant research.

What does it mean if a vendor isn't included in a Magic Quadrant? It means that they did not meet the inclusion criteria defined for that Magic Quadrant. It does not imply that the vendor is not viable or competitive. It might indicate that the vendor has a slightly different strategy or functional match, or that it addresses a different target market. We publish the inclusion criteria with the Magic Quadrant to help you understand why a vendor might not have been included.

Is a market always analyzed the same way? No. As a market matures, it changes and so does the analysis. From year to year, the inclusion criteria, weightings and Gartner's opinion about what it takes to maintain market strength can change. Each report includes a detailed description of market trends and vendor dynamics for that year, and details the vendors added or dropped since the previous report.

When will a Magic Quadrant have Additional Perspectives? For most markets, the globally broad perspective of a Magic Quadrant meets the needs of Gartner's client. Where appropriate, the lead analyst for a Magic Quadrant will add a company size, industry or geographical perspective. The decision to include these perspectives will be based on whether sufficient submarket differentiation exists and is important to clients.

Will the "Notable Vendors" section of a Magic Quadrant context include vendors not in the Magic Quadrant? Yes, it is possible that a vendor that did not meet the inclusion criteria for the Magic Quadrant will be included in the "Notable Vendors" section.

Is there a schedule for when Magic Quadrants will be published? Gartner publishes a calendar of planned Magic Quadrant research on gartner.com .

How often is Magic Quadrant research updated? Gartner typically updates each Magic Quadrant annually unless it is retired. However, some may be refreshed more or less frequently in response to dynamic changes in the market.

What is the meaning of the "as-of" date and why is it different to the publication date? The "as-of" date indicated on a Magic Quadrant indicates when the research was completed and closed to all further input. Any events in the market after the "as-of" date are not considered and do not form part of the assessment or the final report. Generally the "as-of" date is prior to the publication date of the report, because vendors may release new product features after data collection has closed. However, analysts can choose to make any necessary adjustments up to the day of publication, in which case the "as-of" date will match the publication date.

How can I compare current a Magic Quadrant to a previous version? All historical Magic Quadrant research is available online as archived files. The interactive Magic Quadrant will also show the history of a specific market when past versions are available. When performing direct comparisons you should understand that the market definition and criteria may have changed from year to year. This will affect vendor positions or ratings. Also, due to mergers, acquisitions, and rebranding, some vendors can appear under different names from one year to the next. Vendor and product names are accurate on the date of publication, and are not maintained or revised to reflect market activities that may have occurred after publication of that report.

Where can I get more information about Interactive Magic Quadrants? Interactive Magic Quadrants are available only to Gartner clients on gartner.com . Further information and FAQs about the interactive experience are available on the main Magic Quadrant page .

Recommended Reading

© 2014 Gartner, Inc. and/or its Affiliates. All Rights Reserved. Reproduction and distribution of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartners research may discuss legal issues related to the information technology business, Gartner does not provide legal advice or services and its research should not be construed or used as such. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The opinions expressed herein are subject to change without notice.

Why Gartner

Gartner delivers the technology-related insight you need to make the right decisions, every day.

Find out more

Call +1 855-515-4486 or contact us

to become a Gartner client.