Gartner Research

Use Holdout Tests to Measure Marketing Channel ROI

Published: 28 May 2019

ID: G00378831

Analyst(s): Joseph Enever


Marketing leaders seek to determine their marketing channels’ ROI, but often overlook a simple and cost-efficient way to achieve it — holdout testing. Prove marketing channel ROI and make better financial decisions by designing, executing and measuring strong holdout tests.

Table Of Contents
  • Key Challenges



  • Gain Testing Buy-In From Stakeholders
    • Use Examples and Case Studies to Describe Holdout Testing
    • Explain How Holdout Testing Can Help Justify New Initiatives and Assist Budget Prioritization
    • Allay Stakeholders’ Concerns About Losing Revenue
  • Take Five Steps to Design and Measure Great Holdout Tests
    • Step 1: Align Your Tests With a Business Outcome
    • Step 2: Segment Similar Audiences Into Testing Groups
    • Step 3: Choose Your Testing Window and Test Duration
    • Step 4: Effectively Measure Your Holdout Tests
    • Step 5: Commit to Abide by Results

Gartner Recommended Reading

©2022 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form without Gartner’s prior written permission. It consists of the opinions of Gartner’s research organization, which should not be construed as statements of fact. While the information contained in this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartner research may address legal and financial issues, Gartner does not provide legal or investment advice and its research should not be construed or used as such. Your access and use of this publication are governed by Gartner’s Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its research is produced independently by its research organization without input or influence from any third party. For further information, see Guiding Principles on Independence and Objectivity.

Already have a Gartner Account?

Become a client

Learn how to access this content as a Gartner client.