How Markets and Vendors Are Evaluated in Gartner Magic Quadrants


Foundational Refreshed: 21 August 2017 | Published: 22 January 2016 ID: G00298294

Analyst(s): | |

  Free preview of Gartner research

Summary

Magic Quadrants offer visual snapshots, in-depth analyses and actionable advice that provide insight into a market's direction, maturity and participants. Understanding our methodology will help you when evaluating a market, choosing a technology or service provider or managing vendor relationships.

Overview

Key Findings

  • Gartner Magic Quadrants are based on rigorous, fact-based analysis backed up by a highly structured methodology.

  • Your understanding and interpretation of a Gartner Magic Quadrant is crucial. It will enable you to get the most from the market analysis in alignment with your unique business and technology needs.

Recommendations

  • Read this document to gain insight into how Gartner can help you evaluate markets and the technology and service providers in those markets to support your investment decisions.

  • Read this document in conjunction with a Magic Quadrant to provide context to the analysis.

  • Consult a Magic Quadrant to narrow your list of choices, but don't base your decision on the model only.

  • Consider scheduling an inquiry session with a Gartner analyst who specializes in the market to gain further details and insight once you have consulted the relevant Magic Quadrant.

Analysis

Assessing the market and its technology and service providers is a business-critical task. Vendor differentiation based on variations in company size, level of complexity and strategy can make it challenging to compare vendor offerings. In addition, the market's overall direction is often unclear.

Gartner's Magic Quadrants solve these problems by offering snapshots of markets and their participants. These documents enable you to map vendor strengths against your current and future needs.

Magic Quadrants provide a graphical comparative positioning of technology and service providers where market growth is high and provider differentiation is distinct. Magic Quadrants depict markets in the middle phases of their life cycle by using a two-dimensional matrix that evaluates vendors based on their "Completeness of Vision" and "Ability to Execute." The Magic Quadrant has 15 weighted criteria that are used to plot vendors on the graph (shown in Figure 1) based on their relative strengths in the market.

Where appropriate, the Magic Quadrant will provide "Additional Perspectives" based on market differentiators identified as important to consider when selecting technology and service providers. These perspectives will provide context for either a specified geography, industry or company size depending on the market in question.

Alternatively, Gartner may provide insights about a specific market with a Gartner Market Guide rather than a Magic Quadrant.

Understanding Gartner's Qualitative Market Analysis

To evaluate vendors for Magic Quadrants, Gartner uses a comprehensive process that defines the boundaries of the market, the research focus and the steps taken to form the analysis.

Identifying the Market

To be considered for a Magic Quadrant, a market must be distinct and viable. Gartner selects a market for analysis based on the impact of emerging trends and users' need to understand changing market dynamics. Gartner analysts focus on markets in which their insights can assist clients with planning, investment decision making and the ongoing support of vendor relationships.

Selecting Vendors

A Magic Quadrant is not intended to be an exhaustive analysis of every vendor in a market, but rather a focused analysis. The criteria for inclusion may consist of market share, number of clients, installed base, types of products/services, target market or other defining characteristics. These criteria help narrow the scope of the research to those vendors that Gartner considers to be the most important — or best-suited to the evolving needs of Gartner's clients as buyers in the market.

Providers that do not meet the Magic Quadrant criteria for inclusion may be cited in the research, but are not part of the comparative evaluation. Those vendors are recognized for their ability to meet specific business needs and are included in an "honorable mentions" section of the report. Vendors with notable offerings in a specific submarket may also be highlighted in Magic Quadrant Additional Perspectives.

Defining the Rating Criteria

Magic Quadrants use standard criteria in two categories: Completeness of Vision and Ability to Execute (see Figure 1). We then adapt the rating criteria to a market by prioritizing and weighting them based on a "high," "low" or "medium" scale of importance. In some cases, a criterion may have a "no rating" weight because it has low relevance for the market or does not provide sufficient differentiation.

While standard criteria are common to all Magic Quadrants, analysts refine the standard criteria by adding subcriteria to further clarify how the standard criteria apply specifically to the market being evaluated.

Completeness of Vision

This category includes these criteria:

  • Market understanding: The ability of a vendor to understand buyers' needs and translate these needs into products and services. A vendor that shows the highest degree of vision listens to, and understands, what buyers want and need, and can use that information to shape or enhance the relationship.

  • Marketing strategy: A clear, differentiated set of messages consistently communicated throughout the organization and publicized through online presence, advertising, customer programs, events and positioning statements.

  • Sales strategy: A strategy for selling products or services that uses the appropriate network of direct and indirect sales, marketing, service and communication affiliates to extend the scope and depth of a vendor's market reach, skills, expertise, technologies, services and customer base.

  • Offering (product) strategy: A vendor's approach to product development and service delivery that emphasizes differentiation, functions, methodology and feature set in relation to current and future requirements.

  • Business model: The validity and logic of a vendor's underlying business proposition in this market.

  • Vertical/industry strategy: A vendor's strategy to direct resources, skills and offerings to meet the needs of individual market segments, including vertical industries.

  • Innovation: Marshaling of resources, expertise or capital for competitive advantage, investment, consolidation or defense against acquisition.

  • Geographic strategy: A vendor's strategy to direct resources, skills and offerings to meet the needs of regions beyond its "home" or native area — directly or through partners, channels and subsidiaries — as appropriate for that region and market.

Ability to Execute

This category includes these criteria:

  • Products/services: Core goods and services offered by the vendor that compete in and serve the market. This category includes product and service capabilities, quality, feature sets and skills (offered natively or through original equipment manufacturers), as defined in the market definition and possibly further detailed by other criteria.

  • Overall viability: Includes an assessment of the vendor's overall financial health, the financial and practical success of the relevant business unit, and the likelihood of that business unit continuing to invest in and offer the product within the vendor's product portfolio.

  • Sales execution/pricing: The vendor's capabilities in presales and sales activities and the structure that supports them in this market. This criterion also includes deal management, pricing and negotiation, presales support and the overall effectiveness of the sales channel.

  • Market responsiveness and track record: The vendor's ability to respond, change direction, be flexible and achieve competitive success as opportunities develop, competitors act, customer needs evolve and market dynamics change. This criterion also considers how responsive the vendor has been over time.

  • Marketing execution: The clarity, quality, creativity and efficacy of the execution of marketing programs designed to deliver the vendor's message to influence the market, promote its brand and business, increase awareness of its products and services, and establish a positive identification with the product, brand or vendor with buyers. These programs may include, among other elements, a combination of advertising, promotions, thought leadership, word of mouth and sales activities.

  • Customer experience: Relationships, products, and services and programs that enable clients to succeed with the products being evaluated. This criterion includes the ways in which customers receive technical support or account support for the product being evaluated. It can also include ancillary tools, customer support programs (and their quality), availability of user groups and service-level agreements.

  • Operations: The vendor's ability to meet its goals and commitments. This includes the quality of the organizational structure, such as skills, experiences, programs, systems and other vehicles that enable the vendor to operate effectively and efficiently.

Researching the Market

Research activities include, but are not limited to:

  • Attending vendor briefings and product demonstrations

  • Conducting surveys

  • Interviewing reference customers identified by the vendors

  • Interacting with industry contacts

  • Discussing pertinent topics with clients and nonclients

  • Consulting public sources, such as U.S. Securities and Exchange Commission filings, articles, speeches and published papers

  • Seeking input from other Gartner analysts

  • Assessing social media contributions, reviews and interactions, including Gartner Peer Insights.

Evaluating Vendors

Teams of analysts collaborate to evaluate and rate each vendor against each of the weighted criteria. The resulting scores are used to generate a Magic Quadrant.

Publishing the Research

The Magic Quadrant is published as a research document that explains the vendor positions and ratings, as well as any new developments in the market. During this step, the research undergoes rigorous internal peer review and validation, and a factual review by those vendors included in the research.

The Structure of a Gartner Magic Quadrant

The Magic Quadrant graph (see Figure 1) is based on two axes:

  • Completeness of Vision: Reflects the vendor's innovation, whether the vendor drives or follows the market, and if the vendor's view of how the market will develop matches Gartner's perspective.

  • Ability to Execute: Summarizes factors such as the vendor's financial viability, market responsiveness, product development, sales channels and customer base.

Figure 1. The Magic Quadrant
Research image courtesy of Gartner, Inc.

Source: Gartner (January 2016)

Understand the Four Quadrants

Vendors are positioned in the four quadrants — Leaders, Challengers, Visionaries and Niche Players.

Leaders

Leaders provide mature offerings that meet market demand and have demonstrated the vision necessary to sustain their market position as requirements evolve. The hallmark of Leaders is that they focus on, and invest in, their offerings to the point where they lead the market and can affect its overall direction. As a result, Leaders can become the vendors to watch as you try to understand how new market offerings might evolve.

Leaders typically possess a large, satisfied customer base (relative to the size of the market) and enjoy high visibility within the market. Their size and financial strength enable them to remain viable in a challenging economy.

Leaders typically respond to a wide market audience by supporting broad market requirements. However, they may fail to meet the specific needs of vertical markets or other more specialized segments.

Challengers

Challengers have a strong Ability to Execute but may not have a plan that will maintain a strong value proposition for new customers. Larger vendors in mature markets may be positioned as Challengers because they choose to minimize risk or avoid disrupting their customers or their own activities.

Although Challengers typically have significant size and financial resources, they may lack strong vision, innovation or an overall understanding of market needs. Challengers may offer products nearing the end of their lives that dominate a large but shrinking segment.

Challengers can become Leaders if their vision develops. Over time, large companies may fluctuate between the Challengers and Leaders quadrants as their product cycles and market needs shift.

Visionaries

Visionaries align with Gartner's view of how a market will evolve, but their ability to deliver against that vision is less proven. In growing markets, this is the typical status. In more mature markets, it may reflect a competitive strategy for a smaller vendor — such as selling an innovation ahead of mainstream demand — or a larger vendor trying to break out of a rut or differentiate itself.

For vendors and customers, Visionaries fall into the higher risk-higher reward category. They often introduce new technology, services or business models, and they may need to build financial strength, service and support, and sales and distribution channels. Whether Visionaries become Challengers or Leaders may depend on if customers accept the new technology or if the vendors can develop partnerships that complement their strengths. Visionaries are sometimes attractive acquisition targets for Leaders or Challengers.

Niche Players

Niche Players do well in a segment of a market, or they have a limited ability to innovate or outperform other vendors in the wider market. This may be because they focus on a particular functionality or geographic region, or because they are new entrants to the market. Alternatively, they may be struggling to remain relevant in a market that is moving away from them. Niche Players may have reasonably broad functionality, but with limited implementation and support capabilities and relatively limited customer bases. They do not demonstrate a strong vision for their offerings.

For end users, assessing Niche Players is more challenging than assessing vendors in other quadrants — because some could make progress, while others do not execute well and may not have the vision and means to keep pace with broader market demands.

A Niche Player may be a perfect fit for your requirements. However, if it goes against the direction of the market — even if you like what it offers — then it may be a risky choice because its long-term viability will be threatened.

Using a Magic Quadrant for Your Requirements

Your needs and circumstances should determine how you use the Magic Quadrant, not the other way around. We do not recommend evaluating vendors in the Leaders quadrant only, and ignoring those in other quadrants. There are many reasons for this, including that a vendor in the Niche Players quadrant could offer functions that are ideally suited to your needs. Similarly, a Leader may not offer functions that meet your requirements — for example, its offerings may cost more than those of its competitors', or it may not support your region or industry.

A vendor's position on a Magic Quadrant illustrates useful information about that vendor. For example, if you want to make a strategic investment in a technology, then a vendor's viability will be critical. In this case, you should weigh a vendor's Ability to Execute more heavily than its Completeness of Vision, and evaluate Challengers before Visionaries. Conversely, if you can gain a competitive advantage by investing in an emerging technology, then evaluate Visionaries before Challengers.

The comparative positioning of all of the vendors in a Magic Quadrant offers a high-level view of the market. In an established, mature market, most vendors may cluster in the Leaders quadrant because most will tend to offer a complete set of products and services, with little differentiation. Conversely, in a growing market most vendors will cluster in the Niche Players quadrant, and most offerings will be incomplete, yet distinctive.

Gartner's interactive Magic Quadrant features (available to entitled Gartner clients only) enable you to get additional insight into a market and its vendors. You may use these features to tailor your view of the Magic Quadrant to align with your business goals, needs and priorities.

The interactive features allow you to adjust the weightings applied to each of the evaluation criteria — to generate a new Magic Quadrant graphic for that market that is tailored to your own requirements. You can then save and share your customized Magic Quadrant as part of your internal analyses and decision making. This client-customized view of the Magic Quadrant is intended to help you gain relevant insight into a market; it does not represent Gartner's view of the market. Client-customized views of the Magic Quadrant are for internal use only and are subject to standard Gartner usage guidelines.

Some Magic Quadrants include Additional Perspectives that emphasize what is distinct in select submarkets where differentiation is relevant and provides unique considerations for technology and service selection for the given context. The Notable Vendors included in this research are a sample of vendors in the given market that clients should consider and that may or may not have been included in the full Magic Quadrant.

Distinguish Between Gartner Magic Quadrant and Market Guide Research

Markets vary in many ways, but all follow a predictable life cycle with five phases:

  • Emerging

  • High growth

  • Consolidating

  • Mature

  • Declining

When a market emerges, it lacks definition and typically has a multitude of vendors offering point solutions, so it is not always possible to comparatively assess vendors. Low barriers to entry mean that many vendors can enter the market without significant investment, and they can leave without significant implications for their broader business too.

Similarly, when the market is highly mature, the products and services on offer will typically be provided by only a few vendors and can lack differentiation in solution features or services.

In both these cases, a Gartner Market Guide provides a meaningful perspective of the underlying market, with a high-level overview of representative participants.

Frequently Asked Questions

Why does the analyst create a Magic Quadrant instead of a Market Guide? The lead analyst makes a recommendation on which research type is most appropriate to the market under consideration based on its maturity. By the very nature of these markets, determining the growth and maturity point of a rapidly evolving market is not an exact science. The key characteristics of the market outlined earlier in this document are all considered, and the decision is made using that information.

What is a Gartner Critical Capabilities document? Critical Capabilities is an essential companion to the Magic Quadrant. A Critical Capabilities helps Gartner clients gain a view of the positioning of vendors' product and service offerings, allowing comparison against a critical set of differentiators to support their strategic technology investment decisions. It shows which products or services are the best fit in various use cases and provides actionable advice to end-user clients on which products/services they should add to their vendor shortlists for further evaluation.

How does a Gartner Critical Capabilities differ from a Magic Quadrant? Critical Capabilities is a companion note to the Magic Quadrant. Critical Capabilities analyzes products or services based on a set of use cases that match client deployment scenarios, while the Magic Quadrant positions vendors in a market.

Can a Magic Quadrant be used as a vendor selection tool? A Magic Quadrant analyzes a market and can help you to focus your search on important criteria. However, it will not provide the details needed (during the vendor selection process) to align your requirements to a particular vendor. Gartner provides an extensive range of complementary research — such as Critical Capabilities and Vendor Ratings — that can assist you further in your vendor selection process. Discussions with analysts can also help during the vendor selection phase.

Why aren't all vendors from a market included in a Magic Quadrant? In most cases, it would be impossible to cover every vendor in a market — there are too many, and/or some are irrelevant based on the inclusion criteria. Gartner therefore focuses its research in a way that will provide the greatest value to Gartner clients.

Does a vendor's status as a Gartner client influence whether it appears in a Magic Quadrant? No. Sales contracts, vendor requests and client engagements do not influence or drive inclusion in Magic Quadrant research.

What does it mean if a vendor isn't included in a Magic Quadrant? It means that they did not meet the inclusion criteria defined for that Magic Quadrant. It does not imply that the vendor is not viable or competitive. It might indicate that the vendor's strategy or functions differ from the inclusion criteria, or that it addresses a different target market. We publish the inclusion criteria with the Magic Quadrant to help you understand why a vendor might not have been included.

Is a market always analyzed the same way? No. As a market matures, it changes and so does the analysis. From year to year, the inclusion criteria, weightings and Gartner's opinion about what it takes to maintain market strength may change. Each report includes a detailed description of market trends and vendor dynamics for that year, and details the vendors added or dropped since the previous report.

When will a Magic Quadrant have Additional Perspectives? For most markets, the globally broad perspective of a Magic Quadrant meets the needs of Gartner's clients. Where appropriate, Gartner analysts will add a company size, industry or geographical perspective. The decision to include these perspectives will be based on whether sufficient submarket differentiation exists and is important to clients.

Will the Notable Vendors section of a Magic Quadrant contextualization include vendors not in the Magic Quadrant? Yes, it is possible that a vendor that did not meet the inclusion criteria for the Magic Quadrant will be included in the Notable Vendors section.

Is there a schedule for when Magic Quadrants will be published? Yes. Gartner provides the Magic Quadrant publication schedule. This schedule is updated monthly to reflect any changes. Use this link to download the publication schedule (.xlsx) .

How often is Magic Quadrant research updated? Gartner typically updates each Magic Quadrant annually unless it is retired. However, some may be refreshed more or less frequently in response to dynamic changes in the market.

What is the meaning of the "as-of" date, and why is it different from the publication date? The "as-of" date on a Magic Quadrant indicates when the research was completed and closed to all further input. Any events in the market after the "as-of" date are not considered and do not form part of the assessment or the final report. Generally, the "as-of" date is prior to the publication date of the report. However, analysts may choose to make any necessary adjustments up to the day of publication, in which case the "as-of" date will match the publication date.

How can I compare a current Magic Quadrant to a previous version? Historical Magic Quadrant research is available online as archived files. The interactive Magic Quadrant also shows the history of a specific market when past versions are available. When performing direct comparisons, you should understand that the market definition and criteria may change from year to year. This will affect vendor positions and/or ratings. Also, due to mergers, acquisitions and rebranding, some vendors may appear under different names from one year to the next. Vendor and product names are accurate as of the date of publication and are not maintained or revised to reflect market activities that may occur after publication of that report.

Where can I get more information about Interactive Magic Quadrants? Interactive Magic Quadrants are available only to Gartner clients on gartner.com . Further information and FAQs about the interactive experience are available on the main Magic Quadrant page .

What is the relationship between Magic Quadrants, Critical Capabilities and Gartner Peer Insights? Gartner Peer Insights provides ratings and reviews of software and services contributed by IT professionals. Contributors share their firsthand experience of a vendor and product through the various stages of IT life cycle management, from evaluation through deployment. Gartner Peer Insights leverages the structure of the Market Description, Vendor List, Product List and Critical Capabilities (from the Magic Quadrant and Critical Capabilities) for its user review survey structure and the client experience. With this innovative design (analyst-led research and advice through Magic Quadrants and Critical Capabilities, now complemented by the perspective of Gartner Peer Insights), organizations can count on Gartner as their must-have source for a full-spectrum perspective on markets, vendors, products and services.

What if there is a difference between how a vendor and product is rated in Peer Insights and Gartner research such as Magic Quadrants and Critical Capabilities? Peer Insights reflects the individual opinions and experiences of end users who have submitted reviews and ratings through a rigorous verification, validation and moderation process. Analysts draw upon a large base of information from multiple sources to contribute to their research. While end-user feedback is important, so is feedback and input from vendors, references, public sources, investors, the press and social media. Lead authors of Magic Quadrants and Critical Capabilities will get access to Peer Insights review data that has been verified, validated and moderated. Analysts will be able to see data throughout the year to review crowd sentiment. A single Gartner Peer Insights review will not affect the placement of a vendor or product in a Magic Quadrant or Critical Capabilities. The Magic Quadrant and Critical Capabilities research process is based on a structured methodology to maintain independence and objectivity and provide unbiased advice to clients.

Recommended Reading

© 2016 Gartner, Inc. and/or its Affiliates. All Rights Reserved. Reproduction and distribution of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartners research may discuss legal issues related to the information technology business, Gartner does not provide legal advice or services and its research should not be construed or used as such. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The opinions expressed herein are subject to change without notice.

Why Gartner

Gartner delivers the technology-related insight you need to make the right decisions, every day.

Find out more