Survey Analysis: Customers Rate Their BI Platform Implementation Costs

Key Findings

• Implementation costs can range from as much as 100% of the initial license costs in the smallest deployments to around 60% of license costs in the largest ones. The average across all deployment sizes equates to approximately 80%.
• Business intelligence (BI) platform usability (particularly for BI developers), the level of integration among platform components, the level of product quality and the complexity of migrations are all factors that affect implementation cost and effort as well as achievement of business benefits. Consider these factors in addition to cost in your vendor selection decision.
• Ease of use (for both developers and business users) of a particular platform expands the complexity of analysis that users can perform on their own and the breadth of product features used, while lowering implementation costs.

Recommendations

• When evaluating vendors on BI platform ownership costs (BIPOC), balance any cost consideration with functional requirements, expected adoption and business benefits. Low-cost tools that do not meet requirements will not deliver the expected business benefits.
• Be just as rigorous in the selection process for service providers, as implementation costs make up a sizable (between 60% and 100% of license cost depending on deployment size) component of the overall cost.

Each year, Gartner evaluates the BI platforms market with the ultimate goal of publishing the BI platforms Magic Quadrant. The 2012 version of "Magic Quadrant for Business Intelligence Platforms" was published in February 2012. Part of the research process involves a large user survey of vendor-supplied references and other organizations — IT, business or hybrid IT-business leaders disclose their experiences with vendors' BI and analytics products and how those products contributed to overall business success.

The format of the Magic Quadrant research limits the details of the survey data Gartner can disclose. The purpose of this document is to give additional insight into how survey respondents evaluated the experiences they have with 34 vendors/products. To be included in this research, a vendor product must have at least 12 completed reference surveys for the cost of ownership questions in particular. Twenty-five products have met this criterion.

A recent Gartner survey of 1,364 BI professionals worldwide found that cost, while not a top criteria, continues to be a significant factor in their buying decisions. From discussions with clients, most BI leaders primarily equate the cost of a BI platform with what they spend on license fees. While license costs are often the most visible types of cost, in actual fact, license and maintenance, and hardware and implementation costs combined represent less than 30% of the total three-year cost of a BI platform (see "Survey Analysis: Customers Rate Their BI Platform Ownership Costs"). Implementation costs in particular can range from as much as 100% of the initial license costs in the smallest deployments to around 60% of license costs in the largest ones. Within a broader cost context, initial implementation costs represent 5% to 10% of total three-year BIPOC, depending on deployment size and whether or not business user developers are included in the BIPOC calculation.

This research provides an overview of initial implementation cost results overall, by vendor type and for individual products surveyed. It also shows how platform usability (particularly for BI developers), the level of integration among platform components, product quality and migration complexity are factors that affect implementation cost and effort. Table 1 describes how we have categorized vendors for this research.

BI leaders can use these results to evaluate the costs of BI platforms they are considering, but should not focus solely on costs. A cost-benefit analysis that includes an assessment of functional fit with requirements, usability to drive adoption and an assessment of business benefits should also be critical pieces of the purchasing decision equation. It is misguided to invest in low-cost software solely because it is inexpensive. If it does not meet business and usability requirements, it will achieve limited adoption and will fail to deliver expected business benefits.

Source: Gartner (September 2012)

Three other research notes provide details on the components of BI platform ownership cost and should be read in addition to this overview:

• "Survey Analysis: Customers Rate Their BI Platform Ownership Costs" breaks down cost categories of BIPOC by deployment size and compares these by vendor type. In general, paradoxically, easier-to-use platforms that are more pervasively deployed to business users deliver the highest business benefits and also tend to have the highest three-year BIPOC.
• "Survey Analysis: Customers Rate Their BI Platform License and Hardware Costs" compares license and hardware costs across vendors and vendor types based on Gartner survey data. It also shows factors that affect the cost of software licenses and hardware costs, including vendor pricing models, product packaging, product scalability and size of deployment. Most BI leaders primarily equate the cost of a BI platform with what they spend on license fees. While license costs are often the most visible types of cost, in actual fact, license and maintenance, and hardware and implementation costs combined represent less than 30% of the total three-year cost of a BI platform.
• "Survey Analysis: Customers Rate Their BI Platform Ongoing Development and Administration Costs" compares these costs, which make up at least 70% of the total and three-year BIPOC across vendors and vendor types based on Gartner survey data. This research shows that, like implementation costs, platform ease of use, product quality, and size of deployment affect the number of days to create reports with different levels of complexity and development, and development and administration costs. It also highlights and details the paradox that vendors that are successfully achieving more pervasive adoption of BI among business users will tend to have a higher total BI platform ownership cost when we include business user developers and administrators in the calculation.

Implementation Costs by User Count

On average, initial implementation costs (see Figures 1 and 2), such as those for external consultants and system integrators, can represent as little as 60% of license cost in the largest deployments (or an average of $143 per user), to as much as 100% of license costs (or an average of $3,038 per user) in the smallest ones. As such, initial implementation represents a significant factor in total cost and should be procured and managed as rigorously as licenses costs.

N = 504 implementation cost
Chart represents customer perceptions and not Gartner's opinion

Source: Gartner (September 2012)

N = 504 implementation cost
Chart represents customer perceptions and not Gartner's opinion

Source: Gartner (September 2012)

Factors That Affect Implementation Costs

The survey results imply a link between the following:

• Integration and product quality
• Ease of use for developers, platform integration and implementation cost per user

Ease of use appears to translate into lower implementation costs, in part because easy-to-use tools allow IT developers and other BI authors to develop BI content more quickly. It also allows more business users with fewer technical skills to create their own reports and analysis, thereby saving costs incurred by contracting IT specialists to design them. Figure 3 shows each vendor's product quality score on the X axis, their ease of use for developers score on the Y axis, while the size of the dot represents the average implementation cost per user. The color of the dot — orange is above average; blue is below average — shows the average platform integration score (see Note 1). The data shows that vendors with strong ease of use for developers, high product quality and a good degree of platform integration tend to have lower average implementation costs per user. In general, ease of use (for developers) is in part a function of BI platform integration and BI platform developer productivity features (particularly for the full range of simple to complex types of analysis). Well integrated platforms have less platform "moving parts" and they have integrated user tools rather than multiple user interfaces, which tend to require less training even for diverse user groups.

As a net conclusion, platforms that enable easier content development tend to have lower implementation costs per user.

N = 504 implementation cost
Ease of use for developers is scored on a scale of 1 to 7, where a score of 1 to 2 = poor, 3 to 5 = average, and 6 to 7 = outstanding
Product quality is scored on a scale of 1 to 7, where a score of 1 to 2 = poor, 3 to 5 = average, and 6 to 7 = outstanding
Average integration score: see Note 1 for calculation The orange dots are above average integration scores; the blue dots represent below average integration scores.
Average implementation cost per user is the average across deployment sizes
Chart represents customer perceptions and not Gartner's opinion

Source: Gartner (September 2012)

The survey results also suggest that the ease of use (for both developers and consumers) of a particular platform affects the complexity of analysis users can perform on their own, as well as implementation costs and breadth of product functionality used. Enterprises tend to use BI platforms with higher scores on ease of use for a broader range of activities (for example, reporting, ad hoc analysis and dashboards) rather than for a single function. Figure 4 shows each product's composite ease-of-use score versus complexity of analysis conducted by users. The size of the bubble represents average implementation cost per user. Orange bubbles represent those platforms with above average breadth of function use, while blue bubbles are below the survey average. Users beyond traditional power analysts adopt intuitive tools more easily and for more functions. Moreover, ease of use reduces the cost of training and change management. This is evident in the results of the four data discovery tools, which have above average ease-of-use scores, while enabling users of these platforms to conduct the most sophisticated types of analysis. This paradox — ease of use combined with support for complex analysis — has given them momentum in the market and has caused traditional vendors to attempt to imitate their success with similar offerings. The data discovery vendors, with the exception of Tibco Spotfire (which supports the highest levels of complexity of analysis of the data discovery vendors, and is the only data discovery vendor to offer integrated predictive analytics), also tend to have below average implementation costs per user.

A number of factors drive ease of use:

• Many data discovery tool offerings (such as QlikViewh, Tableau and Tibco Spotfire) do not require a traditional IT modeled semantic layer — although they do offer (QlikView recently with its acquisition of Expressor) optional reusable data components and metadata. These tools provide easy-to-use capabilities for business analysts to access, blend, mash up and manipulate data with minimal IT assistance or for IT to develop content more rapidly than with traditional approaches. This approach reduces the deployment and maintenance costs associated with a semantic layer, but can increase the potential for creating personal, workgroup or departmental silos, which can cost more in terms of level of effort for IT to manage from a governance perspective.
• Intuitive BI content authoring tools include a graphical user interface and design environment, and out-of-the-box objects and wizards, which reduce the coding required for all levels of analytical complexity.
• Widely available skills make it easier and often less costly to develop analytic content (certainly than for hard-to-find skills).

N = 504 implementation cost
"Breadth of product use score" is the sum of user activity percentages across reporting, ad hoc analysis (all levels of complexity), dashboards, scorecards and predictive analytics for each vendor Orange dots represent an above average score, while blue dots represent a below average score on breadth of use.
Composite ease of use score is a combined measure of ease of use for business users and ease of use for developers, each scored on a scale of 1 to 7, where a score of 1 to 2 = poor, 3 to 5 = average, and 6 to 7 = outstanding
Composite complexity of analysis/usage is a weighted average score based on percentage of respondents reporting use of the platform. Activities are weighted as follows: viewing static reports = 1, monitoring performance via a scorecard = 1, viewing parameterized reports = 2, doing simple ad hoc analysis = 3, interactive exploration and analysis of data = 4, doing moderately complex to complex ad hoc analysis = 5, using predictive analytics and/or data mining models = 5
Average implementation cost per user is the average across deployment sizes
Chart represents customer perceptions and not Gartner's opinion

Source: Gartner (September 2012)

Products with lower migration complexity tend to realize above average business benefits and lower implementation costs per user. Figure 5 shows migration complexity scores versus implementation costs per user, while the color of the dot (orange is above average; blue is below) show average business benefits achieved by product. With the exception of Alteryx, Quiterian, Tibco Spotfire, Quiterian and Prognoz, these vendors also tend to have below average implementation costs per user.

N = 504 implementation cost
Business benefits score: see Note 2 for calculation
Migration complexity is calculated on a scale of 1 to 4, where 1 = extremely straightforward; 2 = straightforward; 3 = somewhat complex; and 4 = extremely complex
Average implementation cost per user is the average across deployment sizes
Chart represents customer perceptions and not Gartner's opinion

Source: Gartner (September 2012)

Implementation Costs by Vendor Type and Product

A high-level view of average implementation costs versus deployment size by vendor type and product shows which vendors have the largest and most costly deployments (see Figure 6). This view shows that only a handful of vendors — LogiXML, Actuate BIRT, Tableau and Jaspersoft — have customers with both above average deployment sizes and below average implementation costs. Microsoft is the only vendor with a below average number of reported users responding to the cost question, but above average implementation total costs. In general, niche vendors tend to have among the smallest average deployment sizes (outliers in this regard are arcplan, LogiXML and Prognoz) and correspondingly small implementation costs, while megavendors (with the exception of Microsoft, whose customers responding to the cost question on the survey tended to be smaller) and large independents tend to have above average deployment sizes and above average implementation costs, although ratios and rank of these measures vary widely among specific vendors.

N = 504 implementation cost
Average implementation cost is the average across deployment sizes
Average deployment size is calculated for those survey respondents that also provided implementation cost information. Average deployment sizes for each vendor for the survey as a whole will may vary from this number since these respondents are a subset of the overall survey
Chart represents customer perceptions and not Gartner's opinion

Source: Gartner (September 2012)

Implementation cost per user goes down substantially as deployment size increases. Those vendors with very large deployments tend to have skewed lower average per-user implementation costs, while vendors with smaller average deployments tend to have skewed higher average per-user costs. To accurately account for this and to provide more actionable insight for users of this research, we represent most of the cost data that follows by user count. Figure 7 shows average implementation cost per user by individual product, by user count sorted by vendor type. Figure 8 shows average implementation cost per user by vendor type, by user count.

The follow represents analysis of this data by vendor type.

Megavendors

Collectively, customers of megavendors reported above average implementation costs per user across all deployment sizes. IBM Cognos 8 customers reported above average per-user implementation costs across all user sizes. OBIEE customers reported above average implementation costs in all but the largest, above 1,000 user deployments. SAP Business Objects customers reported above average per-user implementation costs only in deployments below 100 users.

Microsoft customers report above average implementation costs in all deployment sizes, with the exception of below 100 users. Microsoft BI customers have unique challenges. They need to deploy three products — Office, SQL Server and SharePoint — to meet BI platform requirements. They often use these three components for non-BI functions. They need to build an optimization layer using Microsoft Analysis Services cubes, which are often part of a Microsoft deployment. Microsoft customers often don't separate out data warehouse implementation costs from SQL Server BI implementation costs — because BI and data warehouse capabilities are deployed using the same product.

Large Independent Vendors

As a group, large independent vendors have above average implementation costs per user for deployment sizes that are below 249 users, and below average implementation costs for deployment sizes that are greater than 250 users. SAS and MicroStrategy tend to enable users to conduct more complex analysis, while Information Builders customers responding to this question in the survey had larger deployments than MicroStrategy and SAS, but support less complex types of analysis.

Data Discovery Tools

Data discovery tools enable business users to develop more of their own analytic content without the assistance of the IT department. For example, Tableau, Tibco Spotfire and Advizor tend to have a higher proportion of business user developers versus IT developers. Most enterprise BI platforms require more specialized and often higher cost IT skills.

As a group, data discovery tools have below average implementation costs per user, particularly at the largest deployment sizes. Individually, this is the case with the exception of Tableau and QlikTech customers, which report above average per-user implementation costs in the 100 to 249 user range of deployment sizes.

Niche Vendors

With the exception of Prognoz, all niche vendors have below average implementation costs per user. Prognoz also tends to have larger deployments on average than the other niche vendors and above average ease of use and complexity of analysis scores. Implementation costs vary widely among the remaining vendors. Phocas, Bitam and Salient have high ease of use, product quality, integration and breadth of use scores. They also have among the lowest implementation costs per user across deployment sizes, which tend to be smaller than average. Alteryx and Quiterian, which provide specialized software for location intelligence and predictive analytics, respectively across smaller numbers of business analyst users, tend to have higher implementation costs per user than the other niche vendors (except for Prognoz), but still fall below the survey average in each deployment size. High ease of use, particularly for developers, contributes to low implementation costs per user for LogiXML across all deployment sizes. Board and Targit have smaller deployment sizes on average, targeted at business users and, despite below average product quality and integration scores, have below average implementation costs per user. Arcplan has implementation costs per user that are below (but close to) the survey average for below 100 user counts, but fall significantly below the survey average in above 1,000 user deployments.

Open-Source Tools

Actuate BIRT and Jaspersoft fall below the survey average for implementation cost per user, although Actuate BIRT customers report costs that are comparable to many commercial vendors at the below 100 user deployment size. Pentaho implementations cost above the survey average on a per-user basis. Pentaho also scores below average on ease of use, product quality and BI platform integration, and above average on migration complexity, all of which can affect implementation costs.

N = 504 implementation cost
Chart represents customer perceptions and not Gartner's opinion

Source: Gartner (September 2012)

N = 504 implementation cost
Vendor type averages are calculated based on the vendor categories defined in Table 1
Chart represents customer perceptions and not Gartner's opinion

Source: Gartner (September 2012)

Methodology

BI platform ownership costs (license, hardware, implementation, and IT and business user ongoing development and administration) by vendor and vendor category by deployment size are aggregated from responses to the 2012 BI Platform Magic Quadrant customer survey. The survey included vendor-provided references, as well as survey responses from BI users from Gartner's BI Summit, as well as respondents from last year's survey. There were 1,364 survey responses, with 120 (8.8%) from non-vendor-supplied reference lists. To ensure the integrity of the survey data, each survey response was checked by company respondent email. For survey responses from non-identifiable email accounts such as Gmail or Yahoo accounts, the respondent was contacted and had to provide Gartner with a company email address, a company role and other contact information (this amounted to fewer than five responses, all of which were vetted and ultimately included).