peer-insights-customers-choice-distinctions-methodology

Gartner Peer Insights Customers’ Choice Methodologies

Recognition for Top Customer-Rated Companies

Since starting Gartner Peer Insights in October of 2015, we have collected more than 120,000 customer reviews across over 320 markets. In a growing number of markets, we’ve collected enough data to help our clients with a top-level synthesis of which vendor products are the most valued by customers in a given market. This synthesis takes into account both the number of reviews and the overall user ratings. We recognize them with a distinction showcasing top vendors in a market.

Gartner Peer Insights Customers’ Choice (Updated 6 September 2018)

Customer Choice methodology for the following markets to be announced in November 2018:

  • Analytics and Business Intelligence Platforms
  • Cloud HCM Suites for Midmarket and Large Enterprises
  • Data Center Backup and Recovery Software
  • Data Integration Tools
  • Endpoint Protection Platforms
  • IT Service Management Tools
  • Solid State Arrays
  • Unified Communications as a Service
  • Web Application Firewalls

Recognition Criteria

Recognition will be given to a maximum of seven vendors in a market according to these criteria:

  • At least one vendor and up to seven may be named a Customers’ Choice in a given market. In markets where more than seven vendors meet the above criteria, the seven vendors with the highest number of published reviews within the submission period will be named.
    • Excessive concentration in geographic region is defined as having less than 20% of all eligible deployments (or 25 eligible deployments, whichever is smaller) reported by the reviewers from all the non-Primary regions (Primary region is defined as region with maximum number of eligible deployments for that vendor)
    • Excessive concentration in industry is defined as having less than 20% of all eligible reviews (or 25 eligible reviews, whichever is smaller) reported from all the non-primary industries (Primary industry is defined as industry with maximum number of eligible reviews for that vendor).
    • Excessive concentration in non-enterprise users is defined as having less than 25 published reviews reported by enterprise end-users. (Enterprise end-user is defined as reviewer’s company size >$50M USD).
  • Reviews for products that are in production / general availability, along with beta products are included in the analysis of this distinction. Reviews for legacy products (no longer being sold by the vendor) were not included in the analysis in this distinction.
  • During the submission period — defined as 12 months prior to the eligibility cutoff date — a vendor must have:
    • 50 or more published reviews
    • An average overall rating of 4.2 stars or greater, when rounded to the nearest tenth.
  • At least one vendor and up to seven may be named a Customers’ Choice in a given market. In markets where more than seven vendors meet the above criteria, the seven vendors with the highest number of published reviews within the submission period will be named.

Ratings and reviews must first pass our validation, conflict of interest and moderation process — please see our Peer Insights Review FAQ here.

For more questions, please see our Customers’ Choice FAQ page or contact PeerInsights@gartner.com.

For vendor guidelines on how to use the Gartner Peer Insights Customers’ Choice Logo and the quote policy, please click here.

Gartner Peer Insights Customers’ Choice (Archived)

Customer Choice methodology for the following markets to be announced in October 2018:

  • Content Collaboration Platforms (formerly EFSS)
  • Data Center Networking
  • Mobile App Development Platforms
  • Sales Force Automation
  • Unified Communications
  • Unified Endpoint Management Tools
  • Web Content Management

Customer Choice methodology for the following markets to be announced in September 2018:

  • Access Management, Worldwide
  • Cloud Infrastructure as a Service, Worldwide
  • Enterprise High-Productivity Application Platform as a Service
  • Unified Threat Management
  • Warehouse Management Systems

Customer Choice methodology has been updated since the announcement of results in August 2018 in the following markets:

Customer Choice methodology has been updated since the announcement of results in July 2018 in the following markets:

Customer Choice methodology has been updated since the announcement of results in May 2018 in the following markets:

Recognition Criteria

Recognition will be given to a maximum of seven vendors in a market according to these criteria:

  • The vendor must have at least one product designated by our research analysts as relevant to the market. Vendors with excessive concentrations from a specific geographic region*, industry** or non-enterprise users*** are not considered.
  • During the submission period — defined as 12 months prior to the eligibility cutoff date — a vendor must have:
    • 50 or more published reviews
    • An average overall rating of 4.2 stars or greater.
  • In markets where more than seven vendors meet the above criteria, the seven vendors with the highest number of published reviews within the submission period will be selected for the designation.

Ratings and reviews must first pass our validation, conflict of interest and moderation process — please see our Peer Insights Review FAQ here.

For more questions, please see our Customers’ Choice FAQ page or contact PeerInsights@gartner.com. For vendor guidelines on how to use the Gartner Peer Insights Customers’ Choice Logo and the quote policy, please click here.

Gartner Peer Insights Customer Choice (Archived)

Customer Choice methodology has been updated since the announcement of results in February 2018 in the following markets:

Recognition Criteria

Recognition will be given to a maximum of seven vendors in a market according to these criteria:

  • The vendor must have at least one product designated by our research analysts as relevant to the market. Vendors with excessive concentrations from a specific geographic region*, industry** or non-enterprise users*** are not considered.
  • During the submission period — defined as 12 months prior to the eligibility cutoff date — a vendor must have:
    • 50 or more approved ratings
    • An average overall rating of 4.200 stars or greater.
  • In markets where more than seven vendors meet the above criteria, the seven vendors with the highest number of approved ratings within the submission period will be selected as winners.

Ratings and reviews must first pass our validation, conflict of interest and moderation process — please see our Peer Insights Review FAQ here.

For more questions, please see our Customers’ Choice FAQ page or contact PeerInsights@gartner.com. For vendor guidelines on how to use the Gartner Peer Insights Customers’ Choice Logo and the quote policy, please click here.

Gartner Peer Insights Customer Choice Awards (Archived)

Customer Choice methodology has been updated since the announcement of results in October 2017 in the following markets:

Market Criteria

A market needs to have at least 300+ reviews, and 7+ vendors with 20+ reviews each for us to give a Customer Choice Award in that market. In addition, we’ll consider other factors like the market’s popularity, timing of Gartner Research refresh (we won’t likely do an award in a market that’s about to get an overhaul), etc. to determine when to launch awards in which markets.

Award Criteria

Awards will be given to vendors in a market according to these criteria:

  • The vendor must have at least one product designated by our Research analysts as relevant to the market.
  • The vendor must have 50 or more reviews published in the submission period.
    • Submission period is defined as 12 months prior to the award eligibility cut off date.
    • Vendor-incentivized reviews will not be counted for award eligibility and will not be included in the score calculation.
    • Reviews published must first pass our validation, conflict of interest, and moderation process – please see our Review FAQ here. Reviews that do not pass the process are not considered.
  • To ensure that the awards are given to vendors who represent the Peer Insights’ end user base, vendors will be eligible for Gold, Silver and Bronze awards subject to three criteria:
    • Maximum 75% of the deployments reported by the reviewers from non-North America regions.
    • Maximum 75% of reviews from one industry.
    • Maximum 50% of reviews from non-Enterprise end users. Enterprise end user is defined as reviewer’s company size >$50M USD.
  • If vendors qualify for the award but don’t meet the three criteria above, they may still be eligible for an honorable mention for their focus.

Award Score

  • Gold, silver and bronze awards will be given to the top 3 vendors based on award score (defined below), and honorable mention will be given to every other vendor over 4.000 stars.
    • Award score is based on a Bayesian weighting of overall score vs. the number of reviews published and the Bayesian prior is a Dirichlet distribution with an average of 4 stars and 50 reviews, so:

Gartner Peer Insights CCA Formula

Where N = Total number of reviews.

  • Award score will be rounded to the nearest tenth of a star. If there’s a tie, the number of published reviews will be the tie breaker. If the number of published reviews is also tied, both vendors will receive the same award.
  • All winners must have an average award score greater than 4.000 stars. If there are less than 3 vendors that meet the criteria we’ll give fewer awards for this market.
  • For more questions, please see our FAQ page, or contact PeerInsights@gartner.com.
  • For Vendor Guidelines on how to use Gartner Peer Insights Customer Choice logo and quote policy please click here.

Gartner Peer Insights Customer Choice Awards – SIEM 2017 (Archived)

Customer Choice methodology has been updated since the announcement of results in June of 2017 in  the following market:

Market Criteria

A market needs to have at least 300+ reviews, and 7+ vendors with 20+ reviews each for us to give a Customer Choice Award in that market. In addition, we’ll consider other factors like the market’s popularity, timing of Gartner Research refresh (we won’t likely do an award in a market that’s about to get an overhaul), etc. to determine when to launch awards in which markets. Vendors will be informed of a new award at least 8 weeks before we announce the winners for their given markets.

Award Criteria

Awards will be given to vendors in a market according to these criteria:

  • The vendor must have at least one product designated by our Research analysts as relevant to the market.
  • The vendor must have 50 or more reviews published in the submission period.
    • Reviews considered must strictly adhere to our guidelines – please see out Technology Provider FAQ here.
    • Reviews published must first pass our validation, conflict of interest, and moderation process – please see our Review FAQ here.
  • Gold, silver and bronze awards will be given to the top 3 vendors based on probable score (defined below), and honorable mention will be given to every other vendor over 4.000 stars.
    • Probable score is based on a Bayesian weighting of overall score vs. the number of reviews published and the Bayesian prior is a Dirichlet distribution with an average of 4 stars and 50 reviews, so:

Formule

Where NReview = Total number of reviews.

  • Probable score will be rounded to the nearest tenth of a star. If there’s a tie, the number of published reviews will be the tie breaker. If the number of published reviews is also tied, both vendors will receive the same award.
  • All winners must have an average probable score greater than 4.000 stars. If there are less than 3 vendors that meet the criteria we’ll give fewer awards for this market.
  • For more questions, please see our FAQ page, or contact PeerInsights@gartner.com.
  • For Vendor Guidelines on how to use Gartner Peer Insights Customer Choice logo and quote policy please click here.