Magic Quadrant for Data Warehouse Database Management Systems
The data warehouse DBMS market is transforming due to the rise of "big data" and logical data warehouses. Unexpectedly, many organizations entered the data warehouse market in 2012 for the first time, increasing demand for professional services and causing important changes in vendors' positions.
This document was revised on 6 February 2013. The document you are viewing is the corrected version. For more information, see the Corrections page on gartner.com.
The market considered in this Magic Quadrant is specifically that for relational database management systems (DBMSs) used as platforms for data warehouses (see Note 1). It is important to note that a DBMS does not in itself constitute a data warehouse — rather, a data warehouse can be deployed on a DBMS platform.
A data warehouse solution architecture often uses many different data constructs and repositories. The market now considers broader technology solutions for information management platforms. The intersection of analytics, cloud and big data forms a new impetus to expand data warehouse architectures and seek alternative strategies in addition to the relational DBMS at the data warehouse's core. For example, cloud infrastructure as a service (IaaS) and platform as a service (PaaS) providers that offer technology to transport, manage, analyze and store data may elect to use NoSQL solutions in combination with a variety of other data management technologies. Rumors of the demise of the data warehouse are misguided, however — a data warehouse is simply a warehouse of data, not a specific class or type of technology.
For this Magic Quadrant, we define a DBMS as a complete software system that supports and manages a database or databases in some form of storage medium (which can include hard-disk drives, flash memory, solid-state drives or even RAM). Data warehouse DBMSs are systems that, in addition to supporting the relational data model, are extended to support new structures and data types, such as XML, text, audio, image and video content, and access to externally managed file systems. They must support data availability to independent front-end application software, include mechanisms to isolate workload requirements (see Note 2) and control various parameters of end-user access within managed instances of the data. A data warehouse DBMS is now expected to coordinate virtualization strategies, and distributed file and/or processing approaches, to address changes in data management and access requirements.
This Magic Quadrant covers only the DBMS supplier portion of the overall data warehouse market. Vendors in the data warehouse DBMS market supply DBMS products for the database infrastructure of a data warehouse and the required operational management controls.
Defining the size of a warehouse is less important in 2013, as the ability to access information assets, regardless of location, has become more important. It remains important, however, to distinguish the actual data volume in a data warehouse from the total database size because data volumes under management are one driver toward the logical data warehouse. In the data warehouse market, the management of databases often becomes more complex as data warehouse size increases (see Note 3). The true measurement of data warehouse platforms will become a combination of volume management and performance optimization for distributed data assets.
For the purposes of this analysis, we treat all of a vendor's products as a set. If a vendor markets more than one DBMS product that can be used as a data warehouse DBMS, we note in the section specific to that vendor, but we evaluate all of that vendor's products together as a single entity. This is especially important because the influence of the logical data warehouse has created a situation in which multiple repository strategies are now expected, even from a single vendor. Strengths and Cautions relating to a specific offering or offerings are also noted in the individual vendor sections. It may become important in the future to evaluate separate vendor offerings as the logical data warehouse becomes more pervasive and implementers more frequently pursue best-of-breed strategies.
To be included in this document's analysis, a DBMS product must have been part of the vendor's product set for the majority of the calendar year in question — in this case, 2012. If a product has been acquired from another vendor, that acquisition must have occurred before 30 June 2012. Any vendor or product acquired mid-year will be represented by a separate dot until publication of the following year's Magic Quadrant.
There are many different delivery models, such as stand-alone DBMS software, certified configurations, cloud (public and private) offerings and data warehouse appliances (see Note 4). These are also evaluated together in the analysis of each vendor.
Source: Gartner (January 2013)
1010data (www.1010data.com) was established 13 years ago as a managed service data warehouse provider with an integrated DBMS and business intelligence (BI) solution primarily for the financial sector. More recently, it has branched into the retail/consumer packaged goods (CPG), telecom, government and healthcare sectors. 1010data offers the current release of its offering, version 6 (released in November 2011), as a managed service, a licensed database and a certified configuration. At the time of writing, over 250 customers use 1010data's database or solution.
- Increasingly diverse customer base. In prior years, 1010data has shown significant capabilities in supporting securities, financial and banking analytic needs. Throughout late 2011 and into 2012, 1010data also added customers in the retail and consumer packaged goods sectors, and achieved similarly high levels of satisfaction. This demonstrates that the DBMS's strong ability to execute in one market now translates well into other markets. It also indicates improved vision overall.
- Seamless version migration. Customers find it relatively simple to get 1010data's DBMS up and running. They often report that moving their analytics from other platforms to 1010data's is straightforward. Similarly, they report that version upgrades are seamless. This is important, as 1010data releases new functionality approximately every other week. Since 1010data provides a software as a service (SaaS) solution to many of its customers, this should be the case — if it were not, 1010data's business model would be in trouble. Whether customers operate 1010data's offerings themselves or buy them as SaaS, they report no additional licensing costs other than for regular maintenance and support for upgrades. Additionally, 1010data's approach includes a full platform offering, from data ingest and integration to analytics, and even supports participation in data consortiums.
- Load and query performance. Most 1010data customers report that the majority of their data is loaded on a daily basis or even more frequently. Most reference customers report that query performance is exceptional, with comments ranging from appreciation of the ability to run complex queries on large datasets to the ease with which end users can access the analytic tools to answer ad hoc queries.
- Support and maintenance. Customers report timely, thorough and intimate product support, from knowledgeable professional staff. For the first time, however, we have heard about issues of scaling and that 1010data's staffing needs to keep pace with 1010data's growth.
- North America-centric. 1010data's growth and customers remain heavily concentrated in North America — it has almost no demonstrated customers elsewhere. This, however, is understandable given its present size and reach. 1010data's clients are predominantly very large organizations, with most of its reference customers reporting annual revenue of over $1 billion.
- Silo access. Many customers report that although 1010data's solution delivers superior performance, access to the data and the ability to use the engine from other front-end analytic tools is less than satisfactory. In cases where other analytic tools need to access the data, 1010data is restricted by its interface and, in some respects, by a lack of transparency about performance statistics and performance-tuning capability.
- Vulnerability to "standard" vendors. Reference customers report that 1010data's solution is used in two or more departments as a key analytics platform, but that it is often not their enterprise standard. This raises the threat of incumbent vendors forcing out 1010data by offering corporate standards. We consider this a minor risk, however, as customers consider that 1010data provides an analytic support capability that is unique to their use case — and that it challenges the supposed need for a "standard." Alternative "standard" vendors generally include all the Leaders in this Magic Quadrant. Stated reasons for selecting other vendors range from high pricing by 1010data to, in a few cases, better performance by the customer's standard vendor.
- A different solution approach. 1010data is something of a maverick innovator in the field of analytics, but this does not make it immune to effects of major market forces. Many end users are demanding the components of a logical data warehouse, which 1010data is able to introduce behind the scenes and transparently — but it is a different matter if customers want to license these components from 1010data and run a logical data warehouse themselves.
Actian (www.actian.com) (formerly Ingres) offers two products: the commercially licensed Vectorwise aimed at analytic data warehouses (version 2.5 was introduced in June 2012) and the general-purpose open-source Ingres DBMS (version 10 was introduced in May 2012). Actian claims over 65 customers for Vectorwise. Ingres, one of the original relational DBMS (RDBMS) engines, has a 30-year history and over 14,000 customers running mission-critical applications, including data warehouses.
- Increasing mind share. Actian rebounded significantly in 2012 from the disruption it experienced the previous year as it reorganized and introduced new products. It introduced new DBMS versions, added experienced executives, and expanded and increased the leverage of its partner network. Actian is now delivering logical models with independent software vendors and system integrators in several vertical markets.
- Focus on performance. Vectorwise remains Actian's focus for analytic use cases, and its excellent performance dominated Gartner's customer interviews and surveys. Vectorwise exploits hardware assists, turning columns into vectors and processing them in x86 chip registers to capitalize on instruction parallelism and on-chip caching. This can have dramatic impact on performance, as noted by one customer interviewed for this research who was referred to Vectorwise by Intel. Vectorwise delivered leading 1TB nonclustered TPC Benchmark H (TPC-H) results in 2012, but this should be viewed in perspective as this is a highly specific sector of the market. As hardware prices drop, Vectorwise's price/performance continues to make it very attractive for smaller projects — most of the interviews and surveys Gartner conducted concerned those at the low end of the size spectrum.
- Continued improvements to functionality and delivery options. In 2012, Actian added more analytic SQL functionality to Vectorwise and revealed a road map for additional capabilities, including online analytical processing (OLAP) features and a Hadoop connector. Actian also introduced Vectorwise appliances with Lenovo and Huawei, certified configurations with Dell, HP and Cisco, a managed service option with remote monitoring, and a cloud offering with Rackspace. Ingres contains most of the features necessary for data warehousing, such as partitioning, compression, parallel query, multidimensional structures and a community-driven geospatial offering.
- Aggressive pricing to gain new customers. Vectorwise has continued to win customers for new installations and furthered its long tradition of having loyal supporters. Vectorwise's core- or data-based licensing model has proved attractive, and several customers mentioned price as a key feature. Some Actian customers mentioned data warehouse and analytics deployments in which technologies from open-source partners Pentaho and Yellowfin are combined with Vectorwise to lower the cost even further.
- Lack of functionality. Vectorwise is key to Actian's ability to support analytic data marts, but the company must continue to enhance its data warehouse, storage management and mixed workload management capabilities if it is to compete with larger, more mature vendors and meet the needs of the broader data warehouse DBMS market. Vectorwise has gained more analytic SQL constructs, but still needs stored procedures, user-defined functions and data types, table partitioning and other features to close the gap with competitors. Actian's road map to add these capabilities looks aggressive and will require a strong focus on execution. The lack of some of these capabilities causes problems when deploying logical data warehouses.
- Good support for "bugs" not enough. The maturity of Vectorwise presents a challenge. Several customers noted early challenges with bugs, though they all praised Actian for its rapid response in addressing them. Still, "fixing bugs for customers" is not a scalable product support model — Actian needs to put more emphasis on product quality.
- Market perception recovery under way. Although Actian has begun to raise awareness of its new brand name and portfolio expansions, it has yet to regain much of its former market traction. Its aggressive new vision must be matched by strong marketing and sales execution. With over 65 customers to call on as references to help sell its innovative product, Actian can and must continue to change the market's perception. It must accelerate its revenue and numbers of new customers more quickly than it has so far.
Calpont (www.calpont.com), a private corporation based in Frisco, Texas, launched InfiniDB, an analytic columnar DBMS platform with parallel query performance, in February 2010. It currently has about 50 named customers. This is a software solution — Calpont does not offer appliances, but it does work with partners offering prebuilt solutions. The latest release of InfiniDB is version 3.5, which became generally available in 2012.
- Targeted regions and markets. Regional-specific vendors often do not fare well — and we continue to view a lack of global presence as a drawback — but Calpont does a good job of focusing its delivery resources on North America, and it does also have a small but significant practice in Germany representing EMEA. Calpont addresses vertical markets differently from many vendors (especially through its partnership network), with more reference customers in the IT services, wholesale, agriculture and online media sectors than in the traditional fields of telecommunications, financial services/banking and retail. Overall, this constitutes a Strength as it demonstrates focus and an ability to identify specific use cases suited to the functionality of its product. From a competitive perspective, Calpont most frequently finds itself up against HP (Vertica), EMC (Greenplum) and Oracle (Exadata), while keeping its resources focused on the analytics data mart sector.
- Technology partners and big data. Being a small company gives Calpont some opportunity to try new ideas. InfiniDB uses a "MySQL front-end" to attract the huge number of MySQL users worldwide. For big data, Calpont has adopted a MapReduce parallel programming paradigm to support parallelism across multiple servers for performance and scalability. Its road map includes many new features and enhancements for performance, stability, scalability and high availability, and the beginnings of support for a logical data warehouse with connectors to Hadoop.
- Multiple pricing options. Of the vendors in this Magic Quadrant, Calpont has one of the more flexible approaches to pricing, with three options: (1) the free InfiniDB Community Edition for single-server deployments; (2) cloud pricing per terabyte; (3) on-premises deployments priced by terabyte or number of processor cores. This enables customers to choose the best deployment option for them in light of their location and the amount of data.
- Ease of deployment and use. Our survey of reference customers asked them to answer a variety of positive sentiment and negative sentiment questions. Calpont scored low for negative sentiment (a good result) and came in the top third for positive sentiment. By far the most common observation made by customers was that Calpont's offering could be "dropped in" quickly to replace MySQL — with some reporting that no changes were required to any of their analytics, and others that they now use the same GUI tools for InfiniDB as they did for MySQL.
- Above-average value. In customer ratings, Calpont scored well above average in terms of value for price paid, although Calpont's customers generally were not on the current version and did not plan to become current in the next nine months. Calpont states that its 3.0 version was deployed primarily to enhance the solution's scaling and that customers on version 2.x therefore had no immediate reason to upgrade. Reference customers report high satisfaction regardless of the version they are using, which seems to support Calpont's assertion that version upgrades are driven more by demands for the new functionality.
- Technical challenges, functional weaknesses. Customers report a lack of some basic SQL and analytical functions (first/last, variance, ranking). As with most columnar DBMSs, insert operations are reported to be slow, but Calpont's bulk loader (CPImport) compensates for this.
- Complex query issues. Users reported that even a small number of users issuing contending complex queries can cause the DBMS to stop functioning properly, even to the point of requiring a restart. Some of these problems appear to emerge from JOIN issues. Calpont says these issues are addressed in newer versions, but, as noted above, many customers have yet to move to these versions.
- Lack of high availability and disaster recovery (HA/DR). According to reference customers, Calpont lacks traditional, robust HA/DR operations, incremental backup and restore being identified as a specific area of weakness. Calpont offers HA with its multiserver configuration, but most customers have not purchased this configuration (hence their comments, probably). After considering the technical challenges, functional weaknesses, complex query issues and lack of robust HA/DR, Gartner takes the view that Calpont is not ready to manage or drive a logical data warehouse, although InfiniDB could play a part in one.
- Continued struggle to find vision. Calpont has been in business for over 12 years, but did not release InfiniDB until 2010. Its customer base remains small, and it has almost no presence in Gartner's customer inquiry base. End users should consider Calpont a "new" vendor when evaluating its products.
Greenplum (www.greenplum.com), part of EMC's Data Products division, offers a massively parallel processing (MPP) data warehouse DBMS that runs on Linux and Unix. Version 4.2 of Greenplum's database was released in November 2011. EMC does not publish a count of named customers but Gartner estimates there are over 400, based on our existing information and analysis of calls to our client inquiry service. EMC offers the Greenplum database for license, as an appliance and as a certified configuration model (it cooperates with partners to deliver certified configurations). EMC (Greenplum) was one of the first vendors to combine DBMS processing with distributed processing clusters (Hadoop).
- Increasingly global presence. EMC has data warehouse customers in Australia, Canada, China, Indonesia, Japan, Malaysia, South Korea and the United States. It offers Greenplum customers a worldwide sales and support organization, and the product's reach and presence are expanding rapidly worldwide. That EMC's Ability to Execute is lower on this year's Magic Quadrant reflects the company's promising return to a focus on visionary leadership. Nevertheless, following EMC's recent repositioning for cloud offerings and distributed infrastructure, it will take more evidence of customer adoption to recover in the execution ratings.
- Broad appeal to vertical markets. To complement its global diversity, EMC has a broad base of customers in vertical markets: telecommunications, financial services, oil and gas, utilities, software and IT, among others. EMC's announcements claim multiple customers with annual revenue of over $1 billion, though most of the reference customers it identified for our Magic Quadrant survey reported annual revenues of $1 billion or less. Additionally, there were fewer new customers among EMC's references this year — but the existing ones increased their investments in its solutions. The same mixed picture emerges from Gartner's inquiry data, so we can make no conclusion about EMC's customer growth in 2012.
- Reliability and performance. Customers report that EMC's system is self-healing in the event of disk or server failures. A few of the reference customers volunteered, without prompting, the information that any failures could be attributed directly to hardware faults and were not the fault of the DBMS. There were instances of advice from EMC customers in the customer experience section of our survey that recommended the use of resource queuing functions to overcome some database performance issues. Customers reported a wide range of data warehouse/database sizes, from 1TB to over 300TB, and at least one customer reported that it was approaching 2PB. Interestingly, the largest number of connected users was in the 100TB range and a small number of users in the very large size range — the latter indicates strategic use by experienced and intensive analysts in very large warehouses. Gartner expects the combination of EMC's stable market delivery and the expertise of former Greenplum staff in designing and developing software to produce a solid recovery by 2014.
- Winning value proposition. Even customers who reported that EMC's pricing is high were impressed by the value proposition. References added that there are very few separately priced and licensed add-ons to the DBMS. One reason for this winning combination is the Greenplum database's roots in managing and analyzing big data assets — EMC's reference customers reported the highest rate of production of any of the vendors in this Magic Quadrant for the implementation of NoSQL, MapReduce and other big data technologies managed or adjacent to Greenplum data warehouses. In addition, EMC claims rapid adoption of Greenplum Chorus, a platform that supports self-service provisioning of the data warehouse architecture "sandboxes" that analysts use when exploring data in the warehouse.
- Reportedly inconsistent HA/DR. EMC's reference customers were less appreciative of its resource queue management in relation to the overall operation of its DBMS — several commented that this made setting up HA an issue. Some also reported problems due to the absence of incremental backup capability, with large databases specifically affected. EMC's resource management also attracted criticism, with some noting that hardware balancing is the user's job and not managed well by the DBMS. In November 2012, EMC acquired Israeli company More IT Resources with a view to using its flagship MoreVRP product to improve its performance monitoring, historical playback of issues and real-time resource allocation. This indicates that EMC is fully aware of the situation and determined to address it. EMC has incremental backup capabilities on its road map for 2013, which is also likely to include user-configured data retention policies.
- Compatibility and loading issues. Customers report that loading issues exist in relation to data type compatibility and extraction, transformation and loading tool compatibility.
- EMC's support for its DBMS. Greenplum customers reported some difficulty in moving to EMC as the supporting vendor, due to a loss of intimacy and a lack of product understanding, especially from low-level support staff. Recent indications are that EMC's support model is improving, but, given the additional HA/DR, compatibility and loading issues, this remains a factor that prospective clients should treat with low level but insistent concern. It is here that the real challenge of combining the capabilities of EMC and Greenplum lies.
Exasol (www.exasol.com) is a small DBMS vendor based in Nuremberg, Germany. It has been in business since 2000. Its first in-memory column-store DBMS, EXASolution, became available in 2004. EXASolution is still used primarily as a data mart for analytic applications, but also occasionally in support of logical data warehouses. Exasol reports 38 customers in production and expects to have 50 customers by January 2013.
- Support for the logical data warehouse. Clients have always praised Exasol's in-memory DBMS for its speed of query processing. However, Exasol's hybrid model, which manages data both in memory and on disk, is not always sufficient to support the diversity of use cases arising from the need to manage data large volumes of varied data at high speed. Exasol has recognized this shortcoming and, with the release of EXAPowerlytics, has been adding support for external functions such as MapReduce and support of the R statistical programming language. Exasol's support for the logical data warehouse will be further extended by the addition of database link capabilities in the next release.
- Customer adoption. Exasol won new customers throughout 2012 as it benefited from the market hype about in-memory computing. This hype, together with Exasol's effective reselling strategy in Europe and Israel, proved a winning combination.
- Growing maturity of product offerings. As Exasol attracts more customers, it is also maturing its products. EXAStorage, currently in beta testing and available as a controlled release, is a complementary storage layer that manages data persistence to disk in order to add support for more redundancy, failover and disaster recovery. It is available as a stand-alone storage solution in addition to EXASolution.
- Availability of skills and support for third-party tools. Like other small vendors, Exasol suffers from the limited availability of skills in this market and limited support for third-party tools, such as BI, data integration and database management tools from Dell (which acquired Quest Software in September 2012). However, Exasol offers standard SQL connectors and APIs that enable interoperability with the offerings of major vendors.
- Completeness of offering. Respondents to our customer survey indicated that completeness of offering remains an issue for Exasol, specifically in relation to auditing, monitoring and administration capabilities. Some also mentioned that upgrading to newer versions could be a challenge. Even so, the vast majority of Exasol customers surveyed indicated that they planned to purchase additional software licenses from the company in the next 12 months.
- Increased competition. Although in-memory DBMSs are benefiting from the hype surrounding SAP Hana, the number of offerings entering the market is also growing as other small vendors emerge. This means Exasol must improve its marketing and communications in order to differentiate itself from its competitors.
- Viability. As for many small vendors, customers reported concerns about the long-term viability of Exasol's technology and the risk associated with this vendor's status as a potential acquisition target. But Exasol's plans to open new offices in the U.K. and Brazil, and its growing customer adoption, are reassuring signs.
HP's (www.hp.com) portfolio of platforms for data warehousing is anchored by Vertica, a column-store analytic DBMS it acquired in May 2011. Vertica is delivered as software for standard platforms (excluding Windows), and as a Community Edition (free for up to 1TB of data, three nodes). Also available are the HP Vertica Analytics System appliance, cloud versions (via HP Cloud Services and Amazon Machine Images), and Datawarehouse On Demand, a hosted, managed service. HP Factory Express (in predefined certified configurations based on its ProLiant DL380 servers) are available for Vertica and Cloudera Distribution including Apache Hadoop (CDH). HP now claims over 2,250 direct, OEM and channel customers for Vertica, an increase from the 500 customers Gartner reported in the 2012 edition of this Magic Quadrant, due primarily to the inclusion of OEM and channel customers previously excluded from this count.
- Price/performance value. Vertica customers we interviewed identified performance, compression and ease of deployment as key advantages; ease of setup and automated database design were mentioned frequently (one respondent's organization, located in a country without technical support personnel at the time of deployment, set up its system entirely unaided). Vertica's pricing model, which is based on the amount of source-system-extracted data (SSED) loaded into the DBMS, rather than the number of users, servers, chips or processor cores, also remains in favor. We expect that the value customers attribute to the solution will be a significant benefit to HP as it tries to recover in terms of overall execution.
- Analytics use case optimization. Vertica was an early leader in analytic platforms. It designed for scalable shared-nothing deployment on commodity MPP clusters with built-in HA, active replicas and automated node recovery, and its separation of an in-memory write-optimized row store from a read-optimized disk-based column store improved load speed (a traditional problem for columnar databases), pointing the way to more general use of in-memory database technology. The Vertica team delivered on its planned road map with the release of version 6 in June 2012. It has doubled the size of its team since acquisition, ramped up its professional services effectively, and extended its vision based on emerging projects with HP Labs' substantial R&D resources.
- Logical data warehouse ("getting to the unstructured stuff"). Adding to Vertica's data compression, multitier storage model and extensibility, version 6's new Flex architecture supports querying across additional data stores, such as Autonomy, ArcSight and Hadoop. This enhances Vertica's flexibility and supplements its in-database execution capabilities, which now include expanding support for R parallelization, pattern matching, statistical and linear regression, geospatial analysis and social graphing. An external tables facility and the ability to materialize tables during loading add reach to these features. Version 6 points to the future use of federation across other RDBMSs, as well as detection of external metadata for "late binding," laying the foundation for key elements of the logical data warehouse vision. Again, while the vision for this solution is present, its execution has yet to be achieved, and we expect this area to be another contributor to HP's recovery on the Ability to Execute axis of the Magic Quadrant.
- Platform optimization tools. Vertica has begun to benefit from HP's hardware and system management resources. The HP Vertica Management Console and HP Cluster Management Utility (CMU) provide Web-based management, configuration and monitoring platforms for Vertica deployments that include cluster visualization, alerting, performance and diagnostic graphs, and "push-button" cluster operations. The HP CMU offers real-time 3D visualization of physical assets, time-series performance views, and a "single pane of glass" for the entire hardware stack.
- Partners and alternative delivery models. Vertica's SaaS and embedded capabilities have begun to attract the attention of HP partners, including large system integrators like Accenture and Capgemini, as well as smaller vertically focused shops. Independent software vendors are creating products using Vertica, and HP Enterprise Services now provides data models, filling a gap to enhance the data warehouse value proposition. HP is supporting the MyVertica community's participation in open-source code sharing via GitHub.
- Poor coordination of unstructured and structured data strategy. Throughout 2012, HP lacked comprehensive integration road maps for the multiple products in its portfolio. These remain under separate management organizations, but a new relationship between Autonomy and Vertica, which sees peer general managers reporting to the executive vice president of HP Software, seems to have stabilized matters. HP needs to capitalize on this to attract customers seeking to incorporate unstructured data, a key driver of the logical data warehouse. Although HP has mitigated some of its organizational problems, its progress toward the logical data warehouse still lags behind.
- Risk to overall market confidence. At the time of writing, HP's acquisition of Autonomy is under financial review. There is a risk that the adverse publicity associated with the recent multibillion-dollar write-downs related to the acquisitions of EDS and Autonomy will add to the concerns Gartner clients have raised about their confidence in HP's ability to manage acquisitions effectively.
- Slow to exploit global presence. In 2012, Vertica's customer base was still overwhelmingly North American, a reflection of HP's slowness to capitalize on its worldwide sales and marketing reach. Appliance marketing has been underwhelming (few Gartner customers even know that HP appliances exist), while Oracle's Exadata messaging continues to dominate the airwaves. HP needs to make a substantial effort — including substantial investments — to raise its profile. Inquiries to Gartner about Vertica remain low, reinforcing the impression of a troubling slowdown in Vertica's momentum. However, new sales force training and compensation arrangements seem to be accelerating global adoption.
- Need for additional features. High performance alone is no longer a differentiator. Vertica's competitive posture could be improved by the addition of HP's system management leadership, but, despite progress, customers continue to point to missing DBMS features, such as stored procedures. To keep pace, HP must validate its strategic information platform ambitions by continuing to improve its workload management, volume credentials, user-defined functions and vision for the logical data warehouse.
IBM (www.ibm.com) offers stand-alone DBMS solutions, as well as data warehouse appliances, marketed as the IBM Smart Analytics System (ISAS) family and under the Netezza brand. IBM's data warehouse software, InfoSphere Warehouse, is available on Unix, Linux, Windows and z/OS. IBM also has a version of the Netezza product that attaches to System z products as an analytic accelerator — the IBM DB2 Analytics Accelerator (IDAA). In late 2012, IBM released the PureData System line of DBMS appliances, in the process rebranding Netezza as the PureData System for Analytics, the Power-based ISAS as the PureData System for Operational Analytics, and the System z ISAS with IDAA as the zEnterprise Analytics System. IBM has thousands of DBMS customers worldwide, but does not report specific customer and/or license counts. There are more than 1,000 appliance customers (Netezza and ISAS combined).
- Logical data warehouse practices. As early as 2010, IBM began to reposition its data management and integration stack to align it more closely with the (then unnamed) logical data warehouse concept. In addition, technical collaboration between IBM and some of its partners (notably Composite Software with the Netezza product line) is bringing this concept into the DBMS and appliance spaces. Close coordination between IBM's big data processing and integration practices and data warehouse offerings forms the third "leg" of its logical data warehouse proposition, the other two being repositories and virtualization. In 2012, IBM aligned its sales, product marketing, product management and product positioning around the Big Data Platform, its physical architecture for the logical data warehouse. In addition, IBM's Global Services organization continues to develop delivery methodology for new data warehousing practices. We expect that IBM will continue along this trajectory and that in the coming years it will vie with other Leaders for visionary leadership. When it comes to making sure that different solutions and platforms are increasingly integrated both technically and from a support and management perspective, IBM continues to make progress. For example, the System-z-based models of the ISAS family (9700/9710) were updated in 4Q12 to include the IDAA (powered by Netezza technology) and rebranded the zEnterprise Analytics System. This is a significant milestone in the evolution of System z as a multiworkload platform that includes high-performance analytics.
- New PureData System appliances. In 2012 IBM released the PureFlex System, followed by the PureData System line of DBMS appliances. Notably, the PureData systems — PureData System for Analytics, powered by Netezza technology, PureData System for Operational Analytics (DB2 with data partitioning) and PureData System for Transactions (DB2 online transaction processing [OLTP] with pureScale clustering) — are geared to data warehousing and analytics. The PureData System for Analytics uses the next generation of Netezza hardware and software, while the PureData System for Operational Analytics replaces the Power 7 Smart Analytics System. Additionally, IBM has delivered a multitemperature storage solution for data warehousing in DB2 10 that automatically manages flash memory and disk storage for high performance. Netezza, along with the Power-based Smart Analytics, joined the Pure Systems family under the PureData System brand name in October 2012.
- Global presence, sales, delivery and support. With offices or distributors in virtually every region of the world, IBM delivers not only software support but also implementation expertise and services. IBM's customer count includes those with warehouses more than three years old and those with warehouses less than three months old. IBM reports 24% growth for its Netezza products alone. IBM's reference customers include organizations in the government, healthcare, financial services, energy and utilities, wholesale, oil and gas, and business and consumer services sectors, as well as many others. With customers ranging from those with less than $100 million in annual revenue to those with over $5 billion, IBM serves both small organizations and large global organizations. Additionally, the introduction of per-terabyte pricing is helping IBM gain traction with customers expecting incremental growth from their warehouse. IBM appeared more frequently in the competitive situations discussed by Gartner clients in 2012 than in 2011, which indicates a gradual increase in mind share. We expect this to continue.
- Extent of use and commitment. IBM often exists side-by-side with competitors, but organizations that use IBM as their data warehouse DBMS standard become almost exclusive users of IBM solutions — these customers typically reach a point at which they exclude other top competitors from consideration. This gives IBM an entrenched customer base that seems ready to use even more IBM products. Sizes of databases reported by customers range from 1TB to 2PB. For all sizes, loading occurred most frequently on a daily basis, with about 20% of customers reporting loading more frequently than daily.
- Enhanced performance, pricing and end-user involvement. The traditional method of data warehouse and analytics deployment usually involves extended requirements specification between warehouse end users and the IT department that deploys the warehouse. In 2012, IBM introduced a series of DBMS product enhancements focused on allowing end users to deploy their own solutions — in effect, empowering end users to do more for themselves. At the same time, IBM's new features audit these activities and enable IT professionals to monitor the self-service environment. InfoSphere Data Click combines data access management with data export design, enabling users to self-serve their data access. Temporal data, multitemperature storage, adaptive compression and real-time load enhancements (what IBM calls "continual ingest") were introduced. Many of these new features and products are aimed at enhancing IBM's Big Data Platform. Finally, IBM's partner network was active in 2012, with new analytic solutions that combine appliance delivery with best practices.
- Complexity, which demands improved field experience. Customers reported a distinct preference for IBM to provide direct support for its products in all cases. Issues were reported concerning incomplete documentation for advanced features. Additionally, customers reported that IBM's partner network demonstrates weakness in supporting products and that up-to-date expertise on the latest features is difficult to obtain. During 2012, IBM made major changes to its Partner program, including training, prebriefings on products and closer alignment with the IBM sales force in an effort to improve customer experience in the field. We believe the new Partner training will have a positive effect in future years, but that at present it leaves IBM vulnerable to partners "knowing too much" about the complexity of implementations and therefore not selecting IBM for their customers.
- Need for further improvement in IBM's mind share. With Netezza, Watson and InfoSphere BigInsights (an Apache Hadoop-based offering), IBM has begun to win the mind share it needs to increase its sales in the data warehouse and analytics data management space. Gartner noted a small increase in the number of mentions of IBM in competitive situations during calls to our inquiry service and bid reviews in 2012, and we will continue to monitor its progress.
- No market awareness of attractive pricing and good support. Overall, IBM scored well in customers' references and in terms of our inquiry service data. But even with high marks for technology, implementation and vision/road map, IBM's issues with support and pricing cause confusion, with reference customers awarding these aspects of IBM's business scores from excellent to neutral and, in isolated cases, very low. This range indicates an inconsistency that IBM must address as it starts to win mind share; otherwise it will fail to benefit from any improvements in n the market's momentum. Judging from IBM's references, the company is on an upward trend for both pricing and support — but IBM must focus on making sure the market knows that these actually are IBM advantages.
- Lagging in-memory strategy. IBM has a vast number of products that use in-memory computing, including solidDB, but its only in-memory solution for the data warehousing and analytical market is Informix Warehouse Edition. By contrast, IBM's main competitors all delivered additional in-memory DBMS capabilities in 2012. IBM must take a leadership position in in-memory computing if it is to stay competitive with the other vendors.
Infobright (www.infobright.com) is a global company with a steadily increasing customer base for its column-vectored, highly compressed DBMS. Open-source (Infobright Community Edition [ICE]) and commercial (Infobright Enterprise Edition [IEE]) versions are available, and the company announced a new Infopliance in September 2012. Infobright claims to have 300 customers, a 50% increase over its 2011 numbers.
- Successful niche focus. Infobright has continued to focus on operational technology data (what it calls "machine-generated data"); its president blogged in 2012 that "We are not designed to be an enterprise data warehouse." Nonetheless, Infobright can play a significant role in the logical data warehouse, managing growing amounts of data from sources such as customer data records (in the telco space, where it has had strong success with partners), smart meter data (in the utilities space) and clickstream data from Internet interactions. Infobright's continuing pursuit of OEMs and partnerships with software vendors including Pentaho, Jaspersoft and Talend has proved successful. In 2012 the company began a new initiative with the introduction of its Infopliance, which is designed to tap the growing interest in appliance form factors. All of this relates to the rise of big data.
- Compression and other performance technology. Infobright's early positioning based on adding a column-store file system to MySQL was successful, but the company has moved beyond that position, and only two of the Infobright customers interviewed for this Magic Quadrant referred to MySQL. The Knowledge Grid in-memory metadata store gives Infobright the ability to enhance performance through selective data decompression in processing queries. This is one reason for the excellent performance cited by customers; another is the superior compression technology that makes Infobright such an effective solution with machine-generated data. Delivering both these benefits without conflict is one of the product's strongest differentiators.
- Price/performance balance. Infobright also features a Distributed Load Processor for parallel loading of data into the system at very high speeds, and was quick to provide connectivity to Hadoop Distributed File System (HDFS) data. Use cases for machine-generated data stored in Hadoop and extracted into Infobright are prominent in the company's customer base, and they represent a compelling price/performance opportunity.
- Simplicity. Customer interviews and surveys highlighted not only the speed and compression ability of Infobright's products, but also their simplicity. The column-store design reduces dependence on special index and aggregate creation, while the Knowledge Grid's contribution to optimization reduces time requirements for specialist tuning activities. This simplicity will continue to attract OEMs, which will base vertically specialized systems on Infobright products. The simplicity of management, the scalability and the compression ability all interest OEMs looking to embed a DBMS that requires little support on their part.
- Low pricing. Infobright remains one of the few open-source column-store DBMS vendors, with ICE — and, as the market now knows, "open source" does not mean free. Infobright also generates revenue from IEE using a commercial license rather than a General Public License, with a subscription support model based on the amount of SSED stored in the system. Reference customers continue to report low prices and lowered barriers to adoption with this model, and Infobright's substantial growth rate shows its strategy to be a success. Infobright is, however, something of an outlier in that its financial and technical success is based on a very good performance overall in the Niche Players quadrant.
- Focus that limits market expansion. Infobright has done well with its focus on machine-generated data, and has made it clear that this focus will continue. But both existing and prospective customers need to address other data management use cases. Our interviews uncovered dissatisfaction with update speed (not a typical requirement for machine data, which is typically append-only), and with the need for a better HA capability. Again, HA has not typically been a focus in machine-generated data use cases, but growth in Infobright's customer base will bring this issue forward. The Infopliance may well prove to be a solution to this issue, but it will take time to validate its effectiveness. Infobright is not partnering for the sales and support of this appliance, and this will limit its early visibility and the ability to provide support for hardware issues in early deployments.
- Absence of management and optimization tools. Prior issues with new releases seem to have been eliminated, but the lack of management software remains an issue — a typical one for small vendors. Third-party software vendors are not quick to commit to working with small software companies, and this puts more pressure on Infobright to develop its own management software.
- Looming MySQL issues. In prior research we have highlighted the risks associated with Infobright's use of MySQL, as MySQL is owned by Oracle. Oracle has continued to enhance this product, but the expiration of Oracle's agreement with the EU is drawing closer, and with it the possibility that Oracle will seek to change agreements with OEMs. Infobright and its customers should remain vigilant.
Kognitio (www.kognitio.com) started out offering a database management system for data warehouses as Whitecross in 1992 and a managed service in 1993. It now has customers using the Kognitio Analytical Platform either as an appliance, a data warehouse DBMS engine, data warehousing as a managed service (hosted on hardware located at Kognitio's sites or those of its partners), or as a data warehouse platform as a service using Amazon Web Services (AWS). Kognitio's total customer count is less than 50.
- Pricing model. Kognitio has always led with innovative pricing models. It pioneered database SaaS offerings of a complete data warehousing service, not just cloud hosting of a data warehouse. Kognitio also has an "on demand" cloud offering, which is available on AWS. Most of its new customers are adopting the Kognitio Cloud offering. The company recently announced a free version for small deployments with up to 128GB of data in RAM (persistent storage is free and unlimited). What makes Kognitio more interesting still is the ability to switch between options.
- Early elements of the logical data warehouse. Kognitio has been progressing toward the logical data warehouse through Hadoop integration that combines the speed of query processing and analytical capabilities of the Kognitio Analytical Platform with management of large volumes of data hosted on a Hadoop distribution. Kognitio's logical data warehousing strategy is also supported by dedicated partnerships with Hortonworks and MetaScale.
- Great performance. Respondents to our customer reference survey continued to support Kognitio's performance capabilities. Kognitio offers an in-memory, row-based DBMS that has been on the market for many years and that has consistently satisfied clients with its great performance, as it makes it easier to use and run ad hoc queries.
- New positioning. In 2012, Kognitio recognized the shifts in the market from traditional warehousing to what we call the logical data warehouse. The new features Kognitio released for its product, and their positioning, have extended the product's capability by enabling it to manage external processes and data.
- Long-term viability challenged by low market visibility. Despite having a good product, Kognitio grows slowly, typically adding fewer than 10 clients a year. This shows that the vendor faces a challenge to improve its visibility in the market, despite all the hype about in-memory DBMSs generated by SAP. Consequently, reference customers had concerns about the company's long-term viability. Any small vendor must address visibility issues and balance this need with an appropriate focus of resources. However, it should be remembered that Kognitio has been in business for 25 years and demonstrated financial viability throughout that time.
- Integration with third-party tools. Clients reported that interoperability with third-party data integration and business intelligence tools remains an issue. This problem is compounded by the challenges posed by the lack of investment by software integrators in interoperability with Kognitio's technology and the limited availability of skills, as is often the case with smaller vendors.
- Lack of global execution, so far. Kognitio, a U.K.-based company, has been trying to enter the North American market by hiring U.S. staff and opening offices in New York. Although this effort is starting to pay off with the addition of a few North American customers, Kognitio is still mainly Europe-based, with most of its customers located in the U.K. It has not significantly grown its partnerships with service providers or the reseller agreements that would enable it to draw more on its partners to mitigate the problems with the availability of skills. While we are encouraged by Kognitio's new vision and small improvements in 2012, we believe the company needs to improve its execution significantly in 2013 with regard to sales and signing contracts.
Microsoft (www.microsoft.com) markets the SQL Server 2012 Fast Track reference architecture, SQL Server 2008 R2 Parallel Data Warehouse Appliance (Update 3), HP Business Decision Warehouse and Dell Quickstart Data Warehouse Appliance. Microsoft does not report customer or license counts, but there are thousands of SQL Server customers worldwide (including analytical and transactional systems).
- Broad market usage. With customers in the manufacturing, telecommunications, insurance, government, retail, financial services, transportation and other industries, Microsoft's solution demonstrates wide applicability to vertical markets. In addition, customers come from all across the world, from North America to EMEA to Asia/Pacific, including countries such as Belgium, Denmark, New Zealand, South Korea, Ukraine and the U.S. Furthermore, customers report annual revenues ranging from under $100 million to over $10 billion. Microsoft is a major rival to any vendor just in sheer numbers of deployed data warehouses (although not in revenue, which, though rising, remains much lower).
- Competitive capability and customer loyalty. Microsoft customers tend to grow with Microsoft. Microsoft also holds its own when competing with other leading vendors, its usual rivals in competitive situations being IBM and Oracle. Its competitors are most often rejected on the basis of price, which means Microsoft's pricing is considered favorable. Reference customers indicated that half of Microsoft's competitive wins were directly related to its price and ease-of-use advantages over competitors. Importantly, although customers identified issues of product maturity (such as with metadata and monitoring tools), they considered Microsoft's skills availability and pricing compensated for these weaknesses — though they still encourage Microsoft to improve on its shortcomings. In addition, Microsoft can claim the highest percentage of customers performing data warehousing on a current software release, according to our survey data.
- Product form factors and partners. By offering DBMS software, providing reference architectures, prebuilding and preloading implementations of reference architectures, offering an appliance and offering professional services (and partner connections), Microsoft offers almost every configuration of data warehouse deployment. In addition, Microsoft has an impressive mix of partners for colead management and lead development, hardware partners, codevelopment agreements with other suppliers and implementers, and reseller agreements. Moreover, Microsoft's Parallel Data Warehouse appliance, despite a slow start, has been adopted by approximately 100 organizations in the past 18 months. Further adoption of this appliance is likely as Dell continues to enter the data warehouse space and sells the Dell Parallel Data Warehouse Appliance.
- Completeness of solution approach. With so much of the software needed for BI and data warehousing initiatives included in a single license, Microsoft makes the processes of purchasing and license management very easy. In addition, Microsoft offers data models (through partners) and provides assistance (directly and through partners) at the time of implementation. The most notable change by Microsoft in 2012 regarding data warehousing was its new vision for combining structured and unstructured data, desktop analytics tools (such as PowerPivot), enterprise warehousing, fast implementation strategies and the cloud.
- Logical data warehouse practices. Microsoft started early on unstructured and structured data in combination by using SharePoint, PowerPivot and technology acquired from Fast Search & Transfer — all coordinated by the SQL Server engine. It has since added in-memory capabilities across the stack by using xVelocity in SQL Server 2012, SQL Server Analysis Services (SSAS) and PowerPivot. In addition, Microsoft has been catching up with the announcement of HDInsight and a solution offering in partnership with Hortonworks. Finally, with Microsoft's StreamInsight complex event processing solution, customers can address the velocity requirements of big data.
- Perceived lack of enterprise-readiness. As SQL Server grows to enable its expanded use for data warehousing, it is generating more questions from clients about its readiness to support large data warehouses and HA/DR capabilities. However, with the release of SQL Server 2012, Microsoft has continued to mature this offering with the addition of Always On features to improve clustering, failover and active-active data warehouses. Growth in the number of large SQL Server data warehouses will also reduce this perception.
- Complete data management capabilities. SQL Server offers a complete set of data management capabilities, including data integration with SQL Server Integration Services and OLAP capabilities with SSAS. However, as data warehousing projects have matured and grown in complexity, clients have reported (both through Gartner's direct interactions and through the survey we conducted for this Magic Quadrant) limitations with the capabilities offered with SQL Server, specifically in the areas of metadata management and data integration. Microsoft is still developing its capabilities in these areas.
- Business model driven by the "majority market." Distributed process management and execution is one aspect of the logical data warehouse. However, although Microsoft was quick to respond to its customers' fast-developing need to cope with data in high volume, in great variety and at fluctuating velocity, and to offer integration with major Hadoop distributions, this left it behind some of the Leaders in terms of vision. But, then, being a "fast follower" for some features and functions has always been part of Microsoft's business model.
Oracle (www.oracle.com) offers a range of products. Customers can choose to build a custom warehouse using Oracle's DBMS software on implementer-chosen hardware and infrastructure, use a certified configuration comprising Oracle products on Oracle-recommended hardware, or purchase an appliance or appliances ready for a warehouse design and load. Oracle offers the following data warehouse solutions, which are included in this evaluation: (1) DBMS software; (2) certified configurations; (3) Oracle Big Data Appliance; (4) Oracle Exadata X3 (X3-2 and X3-8); (5) Oracle SPARC SuperCluster T4 systems with Oracle Exadata Storage Servers. Oracle reports over 390,000 DBMS customers worldwide (it is not possible to determine which licenses are used for transactional systems or data warehousing systems).
- Appeal to current demand. Oracle offers data warehouse data models (for example, for retail and communications and airlines) and uses its acquisitions (such as Endeca) in applications and vertical solutions to achieve broad appeal in the data warehouse market. The breadth of Oracle's offerings, and the fact that they meet the needs of a wide range of customers, means that Oracle scores high for execution.
- DBMS market leader. Oracle's DBMS versions account for approximately 49% of the RDBMS market by revenue (see "Market Share: All Software Markets, Worldwide, 2011"). Overall, 73% of Oracle customers in our reference survey reported that they are on the current version of Oracle, and 59% reported that they will invest in additional Oracle licensing in the next 12 months. We interpret this as a tendency to continue to use Oracle products. Although Oracle has stopped officially reporting the number of Exadata machines installed, Gartner estimates it to be 2,200 at the end of Oracle's fiscal 2012 (which ended on 31 May 2012).
- Continual advances in features and functions. In 2012, Oracle released big data connectors (R Connector for Hadoop, an SQL connector for HDFS and Oracle Loader for Hadoop — HDFS) and spatial data and graph capability. It also launched its in-memory analytics appliance (Oracle Exalytics), Oracle Exadata X3 with 22.4TB of flash storage per full rack (including a Write Back Flash Cache feature for high performance) and many other innovations (see "Oracle Embraces Flash Memory in New Exadata Database Machine"). Oracle's Big Data Appliance specifically addresses MapReduce using a Hadoop distribution — a key driver in the current market. Some customers identified ease of integration as a strength of their "all Oracle" appliance strategy.
- Good penetration of most vertical markets. Oracle delivers data warehouses to many of the standard industries for analytics: telecommunications, financial services and banking, insurance and so on. Moreover, of all the vendors in this analysis, Oracle reports the highest incidence of nontraditional analytics customers: sectors such as hospitality, energy trading, life sciences and food distribution show up in its reference base. This is evidence of Oracle's ability to execute with broad appeal. Many of the vertical markets where Oracle has the greatest success contain traditional implementers or late adopters of data warehousing. Oracle's customers range in annual revenue from $100 million to over $10 billion. However, hundreds of calls to Gartner's inquiry service show that Oracle's approach to pricing, with a variety of appliance form factors and contract mechanisms, creates so many pricing levels that it can confuse customers.
- Version-currency. Oracle customers generally report that they are on current or recent versions. This is due in part to Oracle's stringent version requirements for the maintenance of support, but also because customers find migrating to the current version of Oracle extremely straightforward (14%) or straightforward (40%), with only about one-quarter indicating that version migration is somewhat complex. As a side note, Oracle reports that 30% of Oracle Exadata customers are repeat customers, and it claims this is due to customer satisfaction. Some Oracle customers report that their version-currency is a simple response to data volume and scaling requirements, which are partly met by keeping their version current.
- Customers' experiences of account management. Most of the reference customers identified by Oracle (reference customers were identified by the vendors) reported that their warehouse has been in operation for more than a year. Only a small fraction reported going into production less than six months before responding to our survey. Our survey asked the reference customers a battery of questions (couched in positive and negative terms) to elicit information about their overall sentiment. Oracle's customers continued to award low scores for ease of use and implementation. They also continued to award it low scores for value, pricing and the overall experience. Furthermore, they reported a slight increase in technical issues, year over year, in relation to scaling, missing functionality (unnamed) and the ability to integrate with components of the overall architecture. We should note that Oracle customers reported a contrary, positive experience when asked if there were "no issues" to report, with 20% reporting an issue-free implementation.
- Customers' uncertainty about strategic compatibility. In our previous Magic Quadrant, we said that Oracle's appliance strategy was becoming an area of concern. We believe the appliance strategy has become a significant influence on Oracle's approach to marketing and delivery as part of its hardware revenue strategy. This strategy is important to Oracle, but sometimes runs contrary to customers' designated interests. Our interactions with clients indicate that the "engineered system" approach is commonly perceived as "vendor lock-in," but it is unclear whether they consider this good or bad; it is especially important to remember that vendor lock-in with a preferred vendor in good standing is a positive thing. Customers also find it disconcerting that some capabilities are available only with engineered systems and not with Oracle's standard software products — Hybrid Columnar Compression and Smart Scan, for example, are available only in Exadata Storage Servers. This has given some customers an impression of growing inequity in product delivery, uneven support and disparate "best practices." The logical data warehouse approach to analytics implies flexible, best-of-breed solutions, so any uncertainty about which products have which features complicates decisions about whether to use Oracle products. It is Gartner's position that separate functionality is a natural function of building appliances ("engineered systems" as Oracle calls them) in that appliances require and benefit from specific functionality. We raise the point here only because customers may not understand that this is a predictable and normal phenomenon in mature appliance delivery. Oracle's approach serves primarily to create opportunities for it to further its own efforts to combine hardware with software and Oracle-tuned practices. From a market execution perspective, this will yield positive results, but it is uncertain if the market will agree with Oracle's vision.
- Difficulty expanding large customer base. Reference customers indicated that the most common reason for choosing Oracle over another vendor was simply that Oracle was their standard vendor, and they added that they were reluctant to access new skills to support tools from other vendors. The reference survey indicates that 40% of Oracle respondents also considered IBM in a competitive situation. This rate is also borne out by Gartner's inquiry service data. The growing number of competitive situations shows that the default approach of adhering to simple standards is losing influence.
ParAccel (www.paraccel.com) offers a high-performance analytic platform based on an MPP column-vectored database with strong in-database analytic functions. ParAccel 4.0, which shipped in August 2012, remains a software-only offering, but certified configuration partnerships with Dell, Cisco, HP and NetApp, and cloud offerings with MicroStrategy and Birst, extend the company's portfolio. ParAccel claims over 60 customers worldwide, the overwhelming majority in North America.
- Ample funding and good partners. ParAccel's executive team has been in place for two years now and has secured substantial new funds to support a major effort to capitalize on the market's growing receptivity to analytic platforms that span traditional and big data initiatives. Amazon's investment, an expanding commitment by MicroStrategy, relationships of varying depth with Dell, NetApp and Cisco, and agreements with 42 new resellers and service providers announced in September 2012 indicate a company determined to achieve a new level of growth. ParAccel's licensing model — a single license charge, based on intended use, regardless of type and number of servers and storage devices — remains a useful weapon for it to use against many of its larger competitors.
- Integration and support of analytics in a logical data warehouse. ParAccel has customers with data warehouses that have been in production for over five years, and our interviews revealed satisfaction with their performance and ease of design. ParAccel 4.0 and the company's short-term road map continue to focus on a performance theme, but they extend it to the world of big data by adding On Demand Integration (ODI) high-performance connectors for other data warehouse DBMSs, Open Database Connectivity and Hadoop MapReduce, treating them as engines beneath ParAccel's highly optimized SQL analytic front end (now called the Omne Optimizer). ParAccel now supports the WITH construct in SQL syntax, a very useful capability that some of its competitors still lack. It also supports Apache HCatalog, the emerging metadata facility for the Hadoop stack.
- Advanced analytics. ParAccel's expanded vision for the distributed processing dimensions of the logical data warehouse comes at a key moment for this market. Its support for the Financial Information eXchange (FIX) Protocol and New York Stock Exchange data integration will strengthen its appeal in core vertical markets. ParAccel ships with Fuzzy Logix libraries and supports Numerix as well, reinforcing its in-database analytics leadership; over 500 advanced functions are available to simplify development of complex analytics.
- High-volume analytics. Customers report that ParAccel's support for multilevel, highly recursive analytics on self-joined datasets has been highly effective for market basket analysis and clinical trial data. Significant capabilities for social analytics and many of the new big data dataset issues are already in place, with more to come according to a road map that helped ParAccel enter the Visionaries quadrant in this year's Magic Quadrant. Its claimed support for 5,000 simultaneous clients also positions it well for use by sizable enterprises in larger roles, as does its support for HDFS.
- Time to grow revenue. After seven years, ParAccel remains a very small company. Last year, we noted that it had significantly expanded its experienced sales staff, but growth remains slow in absolute terms. However, its latest efforts — the hiring of a senior executive and an expansion in channel activity — should increase its appeal to large IT vendors as an alternative to inviting a "stack" vendor into their accounts; evidence of this is clear in the strong relationship ParAccel has formed with MicroStrategy. ParAccel has reported a significant improvement in its sales pipeline and has substantially expanded its presence in conferences and marketing channels. Establishing partnerships was a good first step, and in 2013 and 2014 we expect to see these partnerships to produce revenue and mind share.
- Possibility of loading issues. ParAccel customers continue to identify challenges with loading and operating the database (though some of these are addressed in the 4.0 release), with Hadoop ODI (which shares the processing execution and management of processing tasks with Hadoop clusters), and with a new graphical user interface console. These issues have been left largely unaddressed by ParAccel as the company focuses on its performance-related efforts. This situation is somewhat uncharacteristic of a market in which data usually goes into the warehouse without difficulty and where problems more often arise in getting it out — ParAccel, by contrast, can get any data out, but reportedly has problems loading it in the first place. Last year's issues, including those with online scaling and reorganization and Lightweight Directory Access Protocol integration, are operability and data warehousing problems that are not always a focus when analytic platform thinking is the priority. There are positive signs, however: ParAccel has stabilized operationally, as is reflected in our interviews with customers, and last year's reports of crashes and other maturity issues were almost absent this year.
- Demand for more features. ParAccel earned some breathing room in 2011 and has an opportunity to address its issues with loading, protocol integration, online scaling and organization. However, continued deflection of discussion about logical models and metadata will hold ParAccel back, as logical data warehouse designers think beyond execution to data curation (for example, the alignment of multitemporal interpretations and recognition of misaligned data definitions) and operation of the entire analytics data management fabric. Importantly, ParAccel does not need to provide these capabilities directly but can do so by developing partnerships, especially if it finds managed services providers in the AWS cloud.
- Narrow window of cloud opportunity. The continued maturation of cloud offerings could be another opportunity for ParAccel. MicroStrategy has made a clear commitment to ParAccel technology by offering its database as part of the MicroStrategy Cloud portfolio. Others could follow, but ParAccel's window of opportunity could be closing as its big competitors begin to mobilize, especially as most of ParAccel's initiatives have yet to take off. The company needs to accelerate its cloud strategy and publicize some big wins. Amazon (AWS), for example, invested in ParAccel in 2011, and in 2012 announced that it had licensed ParAccel components for use in its planned Redshift offering, to deliver a data warehouse as a cloud service.
This year, our analysis of SAP (www.sap.com) covers both its offerings in this market: SAP Sybase IQ and SAP Hana. SAP Sybase IQ, was the first column-store DBMS, and is SAP/Sybase's primary data warehouse DBMS. It is available as a stand-alone DBMS, a data warehouse appliance and on an OEM basis through system integration vendors (such as BMMsoft). SAP Sybase IQ has over 2,000 customers worldwide. SAP Hana, the company's in-memory DBMS, became generally available in June 2011, and we estimate it had about 500 customers at the end of June 2012 and about 1,000 at the end of December 2012.
- In-memory leader. SAP leads the market with its messaging about in-memory computing, driven by the SAP Hana (which attained general availability in 2011) offering. Customer uptake of SAP Hana has been fast: it attracted 500 customers in just 12 months, a great achievement for a new product. The two use cases that have driven adoption of SAP Hana are in-memory data marts and SAP Business Information Warehouse (BW) powered by Hana. SAP continues to execute on its in-memory strategy, with Hana also supporting other products.
- Support for the logical data warehouse. The release of SAP Sybase IQ 15.4 in 2011 moved SAP Sybase IQ from the traditional data warehouse sector toward the logical data warehouse space by adding support for big data and the ability to create separately managed storage and processing virtual work units that are managed by the DBMS (it already had substantial mixed-workload management features, faster loading capabilities, query parallelism and support for in-DBMS MapReduce capabilities and connectors to Hadoop). SAP Sybase IQ in combination with SAP Hana's in-memory capabilities, R integration and text analytics offers customers tools with which to build a logical data warehouse architecture supporting a wide range of use cases. The company's vision of an "SAP real-time data platform" goes beyond current demand in the data warehouse DBMS market by including the SAP Business Suite and real-time analytics capabilities for transactional data.
- Effective sales and marketing. SAP led the market for DBMS revenue growth in 2011, with a 48% increase year over year, which was much higher than the overall market's 14.8% rise. This success shows that SAP's strategy to become a leading player is having an impact on the DBMS market (see "Forecast: RDBMS by Operating System, Worldwide, 2010-2015").
- Confusing data warehousing strategy. Throughout 2012, Gartner's interactions with SAP customers showed they were unsure whether to use SAP Sybase IQ or SAP Hana for data warehousing. Although SAP's messaging articulated the relative positions of SAP Sybase IQ and SAP Hana as part of a SAP real-time data platform, Gartner knows of customers that have compared SAP Hana with products from other vendors when SAP Sybase IQ would have been a better alternative.
- Newness of SAP Hana. Our customer survey indicates that although customers are very satisfied with SAP Hana's performance and capabilities, as a new product it comes with expected challenges, such as those relating to stability and overall maturity. Our interactions with clients have also revealed that SAP Hana deployments remain small (for less than 1TB of data). SAP states that SAP Hana deployments are split between SAP BW powered by Hana (45%), SAP Hana data marts (45%), and other uses (10%). SAP has pursued an aggressive and responsive product release schedule to mature and evolve the SAP Hana platform, with five major service pack releases offering significant new features introduced in a single year (the latest, SPS5, arrived in November 2012).
- Concerns about ease of implementation and use. SAP's customer survey ratings for ease of implementation, ease of use and satisfaction with support reveal that the company needs to make overall improvements. Although satisfaction with SAP's professional services scored better, this did not fully compensate for the other ratings. Although these ratings may be partly due to the fact that SAP Hana is a recent product and although it will doubtless improve over time, SAP still needs to focus on remedying it shortcomings. Then again, these do not seem to have been strongly limiting factors so far, given the strong adoption of Hana.
- Little interest in big data among SAP customers. SAP's customer base shows the lowest rate of adoption of big data technologies such as NoSQL, Hadoop MapReduceand processing clusters. The behavior of its customers indicates that they are not pursuing big data projects at the same rate as those of other vendors. This could create a false impression of the market's demand — it could be misinterpreted as an indication that big data technology is not wanted.
Teradata (www.teradata.com) has a 30-year history in the data warehouse market, which it supplies with a combination of tuned hardware and analytics-specific database software. With a variety of products, which include the Teradata database (version 14.0 was released February 2012) on various appliance form factors and the Aster Database (available as a DBMS, an appliance or via the cloud using the Amazon Elastic Compute Cloud), Teradata offers both traditional and emerging logical data warehouse solutions. Teradata continued to grow in 2012, and it claims over 1,200 customers worldwide, almost all for production data warehouses.
- Market presence and experience. Teradata delivers its products in such a way that it always leads, or is at least competitive, in overall execution and constantly pushes the market toward emerging best practices and product innovation. Teradata's traditional market includes advanced data warehousing. Dominant Teradata customers in this field are analytics organizations with significant data warehousing requirements that use many of Teradata's built-in functions. The recent influx of inexperienced practitioners creates a demand for education as users' requirements quickly outpace the initial expectations of their organizations when implementing data warehouses. For Teradata, however, this is familiar ground, as it has over 30 years' experience of solving problems with its management tools and of load-balancing systems. Teradata also has an opportunity to win new customers from this base of newcomers as they develop advanced (and well-known) requirements — and discover there are reasons for all the technologies that comprise a data warehouse.
- Product diversity and growth of features and functions. Teradata has four different appliance form factors, ranging from a tiny proof of concept (POC) solution up to an all-flash-memory, enterprise-capable solution. Recent additions to the company's road map and the ongoing incorporation of acquired intellectual property (such as Unity) give Teradata the edge over most of its competitors. Its family of products is further expanded by the Teradata Aster Big Analytics Appliance, a packaged offering combining the Aster Database and Hortonworks Data Platform, a Hadoop distribution. This product diversity firmly addresses the market's big data demands and creates an educational path to guide new entrants to this field.
- Vision for the logical data warehouse. Teradata is a leader in terms of vision for the logical data warehouse with its Unified Data Architecture that combines Teradata, Aster and Hadoop technology and is complemented by its Unity products. The Hortonworks partnership for Hadoop supports a wide variety of data and the changing nature of metadata across various use cases. The Teradata Aster Big Analytics Appliance is starting to address metadata management challenges that are central to the logical data warehouse with support for Apache HCatalog; this is helping Teradata make it easier to share and reuse data stored in external file systems such as HDFS.
- Customer loyalty and durability of purchases. Our survey of Gartner clients and reference customers shows that Teradata's customers have been using Teradata for periods ranging from 10 months to over 10 years. We also acknowledge that Teradata has a multiyear track record of durable implementations, customer satisfaction, and upgrades across different generations of hardware and software.
- Diversity of industries and wide geographic coverage. Healthcare, transportation, financial, insurance, telecommunications, manufacturing, retail, IT services and media are among the industries that Teradata supports. Additionally, its customer base includes organizations with annual revenues ranging from $200 million to $20 billion and more. Nearly all of Teradata's surveyed customers reported that it was their de facto standard vendor for data warehouse products. Teradata is present in every world region, with direct or appropriately capable local support available to customers.
- Upgrades and staffing requirements. Teradata customers reported they incurred either low or no additional staffing costs when implementing its products, indicating that it is easy to move from an older version to a new version. A small but meaningful number of references reported a temporary 50% increase in staffing costs when performing upgrades, but the same customers said they used the temporary increase in staff numbers as an opportunity to change their data warehouse's design.
- Unexplained resistance to version upgrades. Almost 50% of Teradata's reference customers, and a similar number of Teradata customers that use Gartner's inquiry service, report delaying upgrades to new versions. Although they give no reason for this (recent) trend, other survey questions and inquiry questions indicate that Teradata's costs are coming under increasing scrutiny. However, many data warehouse managers report this is because Teradata appears as a specific line-item cost in their budget for the analytics use case. Nevertheless, customers do report that this scrutiny is causing concerns — not because the cost is high, but because it stands out — and the recent high number of "not on current version" responses might reflect these concerns. The reluctance to upgrade might also be due to the ability of older versions to continue to meet expected performance requirement, so reducing the urgency of upgrades or physical scale-outs.
- Near absence of skilled staff. Over half Teradata's reference customers and a similar percentage of Teradata clients who use Gartner's inquiry service indicate that the market does not have enough skilled Teradata professionals. This results in clients complaining that their use of third-party database administration tools, which are difficult to use with Teradata, makes it even more challenging to scale their internal teams to support Teradata products. This could be a major issue for Teradata when competing with other leading vendors for dominance in emerging logical data warehouse practices and even in traditional data warehouses. Skilled personnel with deep knowledge of competing technologies are widely available for customers of those vendors.
- Support for extended environments. A small but meaningful minority of Teradata's customers reported issues with new features (for example, temporal features), long durations for migrations, and the use of Teradata's other offerings (such as Aprimo). In light of the skills issue, Teradata risks offering even more advanced products while having even less ability to support them in the broad market.
We review and adjust our inclusion criteria for Magic Quadrants and MarketScopes as markets change. As a result, the mix of vendors in a Magic Quadrant or MarketScope may change over time. A vendor's appearance in a Magic Quadrant or MarketScope one year and not the next does not necessarily indicate that we have changed our opinion of that vendor. It may reflect a change in the market, and therefore changed evaluation criteria, or a change of focus by that vendor.
Exclusion from the Magic Quadrant should not disqualify a vendor or its products from consideration. Customers should examine vendors and products in light of their specific needs and circumstances.
Gartner's Magic Quadrant process includes research on a wider range of vendors than appear in the published document. This wide remit enables us to spot vendors that need to be added to the Magic Quadrant, as in the case of Calpont this year.
There are many vendors of data warehouse DBMSs in addition to those featured in this Magic Quadrant. However, they did not meet our inclusion criteria. Excluded vendors often also participate in appliance form factors, but do not actually produce their own DBMS — instead they offer other vendors' products on an OEM basis. Some offer technological solutions but failed to meet our reference requirements (by not providing enough active references, whether because they are new companies with few customers or for other reasons). Others provide a range of functionality, but operate only in a specific technical environment. Still others operate only in a single region or support only narrow, departmental implementations. Another reason for excluding some vendors was that, although they met all the functional, deployment and geographic requirements, as very new entrants they have limited revenue and few production customers.
In addition to the vendors featured in this Magic Quadrant, Gartner clients sometimes consider the following vendors, when their specific capabilities match the deployment needs (this list also includes recent market entrants with relevant capabilities). Unless otherwise noted, the information provided on these vendors derives from responses to Gartner's initial request for information for this document or from reference survey respondents. This following list is not intended to be comprehensive:
- BMMsoft. The BMMsoft EDMT solution includes the SAP/Sybase IQ database as its data management base system, and as BMMsoft is an OEM supplier, it is not considered a stand-alone DBMS. Nevertheless, prospective clients should note that the product has unstructured data, content, email analytics and other capabilities in addition to the DBMS. BMMsoft is based in San Francisco, California.
- Objectivity. This vendor, which has a lineage dating back to 1989 when it first began to offer an object-oriented database, now offers InfiniteGraph and Objectivity/DB (version 10.2.1). It claims to have thousands of direct licensed customers as well as thousands of embedded licenses worldwide. Objectivity did not supply the minimum number of reference customer contacts to be included in this research. It is based in Sunnyvale, California.
- ParStream. ParStream 1.8 became available in June 2012 (the first version launched in October 2011) and offers a High Performance Compressed Index (HPCI) on an MPP architecture. However, ParStream did not have enough production customers to qualify for this research. It is based in Cologne, Germany and Redwood City, California.
- XtremeData. This privately owned company targets organizations that need a massively scalable DBMS solution for mixed read and write workloads in the cloud. Organizations pay for their usage with elastic scalability, and they benefit from low entry costs and fast time-to-market. XtremeData is available on AWS and other clouds. Our information on this vendor was obtained not from the RFI we sent to many vendors or from reference customers but from direct communications with XtremeData's technical personnel and the ongoing research of Gartner analysts. XtremeData is based in Schaumburg, Illinois.
- SAND Technology. We were unable to contact anyone willing to take responsibility for communications at this vendor. We were also unable to collect sufficient information, including client references and details of market, product road map and strategic direction, to include SAND Technology in this Magic Quadrant.
To be included in this Magic Quadrant vendors had to meet the following criteria:
- Vendors must have DBMS software that has been generally available for licensing or
supported download for at least a year (that is, since 10 December 2011).
- We use the most recent release of the software to evaluate each vendor's current technical capabilities. We do not consider beta releases. For existing data warehouses and direct vendor customer references and reference survey responses, all versions currently used in production are considered. For older versions, we consider whether later releases may have addressed reported issues, but also the rate at which customers refuse to move to newer versions.
- Product evaluations include technical capabilities, features and functionality present in the product or supported for download through midnight, U.S. Eastern Daylight Time on 7 December 2012. Capabilities, product features or functionality released after this date can be included at Gartner's discretion and in a manner Gartner deems appropriate for ensuring the quality of our research product on behalf of our nonvendor clients. We also consider how such later releases can reasonably impact the end-user experience.
- Vendors must have generated revenue from at least 10 verifiable and distinct organizations with data warehouse DBMSs in production that responded to Gartner's approved reference survey questionnaire. Revenue can be from licenses, support and/or maintenance. Gartner may include additional vendors based on undisclosed references in cases of known use for classified but unspecified use cases. For this year's Magic Quadrant, the approved questionnaire was produced in English only.
- Customers in production must have deployed data warehouses that integrate data from at least two operational source systems for more than one end-user community (such as separate business lines or differing levels of analytics).
- Support for these data warehouse DBMS product(s) must be available from the vendor. We also consider products from vendors that control or participate in the engineering of open-source DBMSs and their support. We also include vendors' capabilities to coordinate data management and processing from additional sources beyond the DBMS, but continue to require that a DBMS meets Gartner's definition (see Notes 1 to 4 for defining parameters).
- Vendors participating in the data warehouse DBMS market must demonstrate their ability to deliver the necessary services to support a data warehouse via the establishment and delivery of support processes, professional services and/or committed resources and budget.
- Products that exclusively support an integrated front-end tool that reads only from the paired data management system do not qualify for this Magic Quadrant.
For details of our research methodology, see Note 5.
Ability to Execute is primarily concerned with the ability and maturity of the product and the vendor. Criteria under this heading also consider the product's portability, its ability to run and scale in different operating environments (giving the customer a range of options), and the differentiation between data warehouse DBMS solutions and data warehouse appliances. Ability to Execute criteria are critical to customers' satisfaction and success with a product, so customer references are weighted heavily throughout.
Product/service includes the technical attributes of the DBMS, as well as features and functionality built specifically to manage the DBMS when used as a data warehouse platform. We include HA/DR, support and management of mixed workloads, the speed and scalability of data loading, and support for new hardware and memory models. These attributes are evaluated across a variety of database sizes and workloads. We also consider the automated management and resources necessary to manage a data warehouse, especially as it scales to accommodate larger and more complex workloads. The abilities to respond to new varieties of information assets, incorporate distributed processing and manage distributed data are also part of this criterion.
Overall viability includes corporate aspects such as the skills of the personnel, financial stability, research and development (R&D) investment, and merger and acquisition activity. It also covers the management's ability to respond to market changes and the company's ability to survive market difficulties (crucial for long-term survival). Vendors are further evaluated on their capability to establish dominance in meeting a specific market demand.
Under sales execution/pricing we examine the price/performance and pricing models of the DBMS, and the ability of the sales force to manage accounts (judging from feedback from our clients). We also consider DBMS software's market share. Also included is the diversity and innovative nature of packaging and pricing models, including the ability to promote, sell and support the product around the world.
Market responsiveness and track record — for example, the vendor's ability to meet demand for vertical-market solutions, general market perceptions of the vendor and its products, and the diversity of its delivery models in response to demand (for example, its ability to offer appliances, software solutions, data warehouse as a service offerings or certified configurations). We also consider the vendor's ability to adapt to market changes during the previous 18 months and its flexibility in response to market dynamics over a longer period.
Marketing execution includes the ability to generate and develop leads, channel development through Internet-enabled trial software delivery, partnering agreements (including co-seller, co-marketing and co-lead management arrangements). Also considered are the vendor's coordination and delivery of education and marketing events throughout the world and across vertical markets.
The customer experience criterion is based primarily on customer reference surveys and discussions with users of Gartner's inquiry service during the previous six quarters. Also considered are the vendor's track record of POCs, customers' perceptions of the product, and customers' loyalty to the vendor (this reflects their tolerance of its practices and can indicate their level of satisfaction). This criterion is sensitive to year-to-year fluctuations, based on customer experience surveys. Additionally, customer input regarding the application of products to limited use cases can be significant, depending on the success or failure of the vendor's approach in the market.
Operations covers the alignment of the vendor's operations, as well as whether and how this enhances its ability to deliver. Aspects considered include field delivery of appliances, manufacturing (including the identification of diverse geographic cost advantages), internationalization of the product (in light of both technical and legal requirements) and adequate staffing.
Source: Gartner (January 2013)
Completeness of Vision encompasses a vendor's abilities to understand the functions needed to support data warehouse workload design, develop a product strategy that meets the market's requirements, comprehend overall market trends, and influence or lead the market when necessary. A visionary leadership role is necessary for the long-term viability of both product and company. A vendor's vision is enhanced by its willingness to extend its influence throughout the market by working with independent third-party application software vendors that deliver data-warehouse-driven solutions (for BI, for example). A successful vendor will be able not only to understand the competitive landscape of data warehouses but also to shape the future of this field.
Market understanding covers a vendor's ability to understand the market and shape its growth and vision. In addition to examining a vendor's core competencies in this market, we consider its awareness of new trends, such as the increasing sophistication of end users, the growth in data volumes, and the changing concept of the data warehouse and analytics information management in the enterprise (see also "Data Warehousing Trends for the CIO, 2011-2012").
Marketing strategy refers to a vendor's marketing messages, product focus, and ability to choose appropriate target markets and third-party software vendor partnerships to enhance the marketability of its products. For example, we consider whether the vendor encourages and supports independent software vendors in its efforts to support the DBMS in native mode (via, for instance, co-marketing or co-advertising with "value-added" partners). This criterion includes the vendor's responses to the market trends identified above and any offers of alternative solutions in its marketing materials and plans.
Sales strategy is an important criterion. It encompasses all channels and partnerships developed to assist with selling, and is especially important for younger organizations as it can enable them greatly to increase their market presence while maintaining lower sales costs (for example, through co-selling or joint advertising). This criterion also covers a vendor's ability to communicate its vision to its field organization and, therefore, to clients and prospective customers. Also included are pricing innovations and strategies, such as new licensing arrangements and the availability of freeware and trial software.
Offering (product) strategy covers the areas of product portability and packaging. Vendors should demonstrate a diverse strategy that enables customers to choose what they need to build a complete data warehouse solution. Also covered are partners' offerings that include technical, marketing, sales and support integration. Additionally, we consider the availability of certified configurations and appliances based on the vendor's DBMS.
Business model covers how a vendor's model of a target market combines with its products and pricing, and whether the vendor can generate profits with this model, judging from its packaging and offerings. Additionally, we consider reviews of publicly announced earnings and forward-looking statements relating to an intended market focus. For private companies and to augment publicly available information, we use proxies for earnings and new customer growth, such as the number of Gartner clients indicating interest in, or awareness of, a vendor's products during calls to our inquiry service.
In recent years, organizations seeking traditional data warehousing solutions have become interested in vertical/industry strategy. This strategy affects a vendor's ability to understand its clients. Aspects such as vertical-market sales teams and specific vertical-market data models are considered for this criterion.
Innovation is a major criterion when evaluating the vision of data warehouse DBMS vendors in developing new functionality, allocating R&D spending and leading the market in new directions. This criterion also covers a vendor's ability to innovate and develop new functionality in its DBMS, specifically for data warehouses. The use of new storage and hardware models is one key to this. Users expect a DBMS to become self-tuning, reducing the resources required to optimize the data warehouse, especially as mixed workloads increase. Also addressed here is the maturation of alternative delivery methods, such as IaaS and cloud infrastructures. Vendors' awareness of new data warehousing methodologies and delivery trends is also considered.
We evaluate a vendor's worldwide reach and geographic strategy by considering its ability to use its own resources in different regions, as well as those of subsidiaries and partners. This criterion considers a vendor's ability to support clients throughout the world, around the clock and in many languages. Anticipation of regional and global economic conditions is also considered.
Source: Gartner (January 2013)
The Leaders quadrant generally contains the vendors that demonstrate the greatest support for data warehouses of all sizes, with large numbers of concurrent users and management of mixed data warehousing workloads. These vendors lead in data warehousing by consistently demonstrating customer satisfaction and strong support, as well as longevity in the data warehouse DBMS market, with strong hardware alliances. Hence, Leaders represent the lowest risk for data warehouse implementations in relation to, among other things, performance as mixed workloads, database sizes and complexity increase. Additionally, the market's maturity demands that Leaders maintain a strong vision for the key trends of the past year: mixed-workload management for end-user service-level satisfaction and data volume management. Finally, Leaders have the ability to align their messaging, product placement and pricing models to support both traditional data warehousing practices and emerging practices for distributed processing, data virtualization, metadata management and dynamic optimization, which are essential for the logical data warehouse and big data to become the "new normal." Importantly, with so many Leaders aligned with our current criteria, the scale of differentiation will shift even more strongly in coming years toward combining structured and unstructured analytics, support for external files and processing, and the ability to support a diverse set of end-user applications via metadata and performance auditing and tuning.
The Challengers quadrant includes stable vendors with strong, established offerings but a relative lack of vision — in 2012 it was possible for very strong traditional vendors to have high scores for Ability to Execute but lower scores for Completeness of Vision. Challengers have presence in the data warehouse DBMS space, proven products and demonstrable corporate stability. They generally have a highly capable execution model. Ease of implementation, clarity of message and engagement with clients contribute to these vendors' success. Challengers in this year's Magic Quadrant focus on superior execution relative to pricing; with customers demanding discernible value for price paid, Challengers offer simplicity, fast implementation and less functionality. Organizations often purchase Challengers' products initially for limited deployments, such as a departmental warehouse or a large data mart, with the intention of later scaling them up to enterprise-class deployments. Increasingly, Gartner clients consider exceptional performance on structured data only to be evidence of "limited" vision.
Visionaries take a forward-thinking approach to managing the hardware, software and end-user aspects of a data warehouse. However, they often suffer from a lack of a global, and even strong regional, presence. Frequently, their messaging to align benefits with market use cases is unclear. They normally have smaller market shares than Leaders and Challengers. New entrants with exceptional technology may appear in this quadrant soon after their products become generally available. But, more typically, vendors with unique or exceptional technology appear in this quadrant once their products have been generally available for several quarters. The Visionaries quadrant is often populated by entrants with increasing functional capabilities combined with new concepts that are unproven in the market.
To qualify as Visionaries, vendors must demonstrate that they have customers in production, in order to prove the value of their functionality and/or architecture. Our requirements for production customers and general availability for at least a year mean that Visionaries must be more than just startups with a good idea. Frequently, Visionaries will drive other vendors and products in this market toward new concepts and engineering enhancements. During 2011, the Visionaries quadrant was populated with a larger number of vendors because overall market demand for strategies for specific functions was not being met. But many vendors answered that demand partially or fully in 2012 — such as the use of MapReduce for large-scale data analytics and massive process scaling in heterogeneous hardware environments.
Niche Players generally deliver a highly specialized product with limited market appeal. Frequently, a Niche Player provides an exceptional data warehouse DBMS product, but is isolated or limited to a specific end-user community, region or industry. Although the solution itself may not have limitations, adoption is limited.
This quadrant contains vendors in several categories:
- Those with data warehouse DBMS products that lack a strong or a large customer base.
- Those with a data warehouse DBMS that lacks the functionality of those of the Leaders.
- Those with new data warehouse DBMS products that lack general customer acceptance or the proven functionality to move beyond niche status. Niche Players typically offer smaller, specialized solutions that are used for specific data warehouse applications, depending on the client's needs.
In 2012 the data warehouse began to fulfill its new role as an overall analytics data management platform, and we expect this to continue in 2013. Gartner predicts that from 2015 "big data" concepts will become part of the "new normal" in analytics and data management and integration, and solidified as pro forma requirements by 2018. In "Big Data Drives Rapid Changes in Infrastructure and $232 Billion in IT Spending Through 2016," we state that big data will cease to be a distinguishable technological approach in 2015 and that it will be wholly obscured by 2018. However, for now, big data remains a specific requirement that the data warehouse must address as it evolves into a new form — the logical data warehouse.
In addition, organizations demanding advanced capabilities have amplified their requirements to include extended query-driven analytics, time series analysis and causal analysis of extended data structures beyond relational data. Distributed processing, which is best represented by MapReduce distributed processing solutions and solutions that emulate this type of processing (such as those of Cloudera, Hortonworks, MapR and even LexisNexis's High Performance Computing Cluster), are rising in demand. Organizations that were previously content to complete such distributed processes on separate clusters of servers and storage devices now demand that this capability is integrated with existing analytic engines. In some cases, BI platforms are being used to address this demand, as are data virtualization tools.
In 2012, many data warehouse programs actually regressed in terms of capability and demand. Thus, our Leaders are a mixed group, some with technology-leading vision across the board, and others more focused on meeting technologically regressive demand. It is important to remember that market leadership does not necessarily equate to technology leadership. As big data becomes "normal" and the logical data warehouse becomes the best practice, vision will become increasingly important. As the market continues to evolve, we expect to see a significant split between vendors that execute strongly in fulfillment of less sophisticated demand and vendors with the vision to provide advanced technologies for leading-edge implementers.
Increasingly, for reasons of HA, visibility into performance, resource management and other demands, data warehouse DBMS vendors are being required to address enterprise-readiness by "hardening" extended environments. Many of these vendors have already adopted practices and deployed features and functions to support big data management and processing as well as the virtualization capabilities required for new analytics practices — but they are ahead of market adoption in this respect. Being ahead of demand indicates strong vision.
This Magic Quadrant deals with the key capabilities of business analytics and information capabilities frameworks. It should therefore interest anyone involved in defining, purchasing, building or managing a data warehouse environment — notably CIOs, chief technology officers, members of BI competency centers, infrastructure, database and data warehouse architects, database administrators and IT purchasing managers.
For this Magic Quadrant, Gartner based most of its analysis on information collected between December 2011 and November 2012. But we also considered information from a longer historic period, as well as any late-breaking news relating to vendors' products, customers and finances.
In 2012, the DBMS market revenue continued to grow. The traditional vendors plugged most of the gaps in their product portfolios for big data and distributed processing. In the process they regained much of the demand that had dropped away in the previous year. Much remains to be done, however, in relation to customers' demands for new data warehouse practices and capabilities. For example, vendors need to offer customers the ability to shift between relational, virtual and distributed data management and processing seamlessly, rather than via sometimes clumsy manual change-overs.
Additionally, the market recognized that a data warehouse need no longer be defined by only by a DBMS running a primary repository. Data warehouses now can include virtual data stores, federation and even distributed processing across clusters. We use the term "logical data warehouse" to refer to these new data warehouses and the practices associated with them.
Incorporating data stored in HDFS and emerging NoSQL stores such as MongoDB and Couchbase will become increasingly common. NoSQL data stores are typically used by programmers, not "database people," so care is required to ensure data governance is not circumvented and users of the new tools are brought into the data management community along with the development of the logical data warehouse.
Importantly, we expect the DBMS market to have grown by about 10% in 2012, despite the troubled economic situation (see "Forecast Analysis: Enterprise Infrastructure Software, Worldwide, 2011-2016, 2Q12 Update"). This underlines the investment organizations are making in information management technologies.
While leaders in data warehouse implementation and utilization are demanding new architectures with extended data management capabilities, many entirely new users are at long last entering the data warehouse market. In 2012 Gartner recorded a significant increase in inquiries from organizations seeking to deploy data warehouses and analytic data stores for the first time. (This might sound incredible over 25 years into the data warehousing era, but we asked these clients several questions to confirm that they were contemplating truly "greenfield" data warehousing initiatives.)
There is cause for concern about these new implementations. History shows the periodic resurgence of discredited practices by disreputable or ignorant implementers, which have led organizations' data warehouse practices astray. At the same time, vendors have an opportunity to oversell infrastructure. Clients should therefore be cautious and conduct POC exercises that reflect their expected use case over the next three to five years (no longer than this because use cases will grow and require new technology, not oversized aging technology). Additionally, new adopters of data warehousing technology should carefully evaluate how their workload changes and could be managed by different solution architectures. Large-volume queries on structured data have very different performance requirements and data design from highly recursive queries and subselection analysis, and these in turn differ from blended structured, unstructured and distributed data processes. Organizations new to this field must learn the lessons of the past 25 years, not repeat the painful mistakes of their predecessors.
For most of the past 20 years, fewer than 20% of organizations have tended to adopt visionary architectures five to seven years before they became widely accepted. In 2012, however, the emergence of the logical data warehouse — and its associated set of best practices for warehousing and analytics data management — had a significant impact on customers' vision. Gartner clients increasingly reported that the logical data warehouse was their preferred future architecture for data warehouses. Clearly, an effective data warehouse remains a mission-critical requirement.
One alternative to the logical data warehouse as an analytics information architecture is wholesale replacement of the data warehouse with search, content analytics and MapReduce clusters, and the elimination of the centralized repository. The year also saw the rise of the concept of cloud-deployed analytics data management (sometimes referred to as data warehouses, sometimes contrasted with them).
In light of these developments, it is self-serving and disingenuous, even intentionally misleading, to continue to promote the idea of a centralized repository as the only solution for data warehousing. A data warehouse is first and foremost a data integration and rationalization engine. It is becoming increasingly important for vendors and implementers to recognize that a data warehouse uses multiple technologies. The misdirection only grows worse when proposals are made to replace entire architectures based solely on the excuse of adding one new type of technology (for example, NoSQL or MapReduce). However, the impact of the information management challenges arising from what Gartner calls the Nexus of Forces — Cloud, Social, Mobility and Information — is undeniable.
In 2012, Gartner observed that IBM, Oracle and Teradata began to deliver, in very different fashion, on their visions and road maps for revised offerings. It is becoming increasingly difficult for these vendors to demonstrate a clear lead over each other as they each have an advantage in different areas of analysis, and they all struggle to sell to each other's customers while easily defending their own customer base. It is entirely possible for one of them to lead in terms of execution by delivering only to the more traditional customers, and for another to lead the market toward a new vision of, for example, logical data warehouse practices.
Microsoft began to deliver on its vision for the data warehouse market in 2012. Implementers responded with increased interest, judging from the number of inquiries Gartner received.
Meanwhile, SAP completed the transition of Sybase from acquisition to fully enabled technology delivery unit within the overall SAP corporation. SAP's Data Services, BI platform and Hana were the subject of more frequent calls to Gartner's enquiry service from organizations seeking advice about competitive situations, so these offerings are looming large on the horizons of the market's traditional leaders.
2012 also brought the return to the Visionaries quadrant of Paraccel and Kognitio. However, their continued success will depend on their execution in 2013, rather than their improved vision.
Actian, for its part, recovered in terms of both vision and execution. Its aggressive positioning helped expand the customer base for its new products.
1010data branched into new vertical markets and has therefore improved its position on the Completeness of Vision axis. But 1010data remained somewhat difficult to position on the Magic Quadrant as it differs considerably from other vendors in that, although it offers a DBMS, its strongest offering is an outsourced and complete technology stack.
A data warehouse is a collection of data in which two or more disparate data sources can be brought together in an integrated, time-variant information management strategy. Its logical design includes the flexibility to introduce additional disparate data without significant modification of any existing entity's design.
A data warehouse can be much larger than the volume of data stored in the DBMS, especially in cases of distributed data management. Gartner clients report that 100TB warehouses often hold less than 30 terabytes of actual data (SSED).
For the purposes of this evaluation, the workloads we expect to be managed by a data warehouse include batch/bulk loading, structured query support for reporting, views/cubes/dimensional-model maintenance to support OLAP, real-time or continuous data loading, data mining, significant numbers of concurrent access instances (primarily to support application-based queries at the rate of hundreds if not thousands per minute) and management of externally distributed processes.
The data warehouse volume managed in the DBMS can be of any size. For the purpose of measuring the size of a data warehouse database, we define data volume as SSED, excluding all data-warehouse-design-specific structures (such as indexes, cubes, stars and summary tables). SSED is the actual row/byte count of data extracted from all sources. The sizing definitions of traditional warehouses are:
- Small data warehouse: less than 5TB
- Midsize data warehouse: 5TB to 40TB
- Large data warehouse: more than 40TB
A data warehouse appliance consists of a DBMS mounted on specified server hardware with an included storage subsystem specifically configured for analytics use case performance characteristics. In addition, a single point of contact for support of the appliance is available from the vendor, and the pricing for the appliance does not include separate prices for the hardware, storage and DBMS components.
Gartner uses multiple inputs to establish the positions and scoring of vendors in our Magic Quadrants. These are adjusted to account for maturity in a given market, market size and other factors. For this update of the Magic Quadrant, the following sources of information were used:
- Original Gartner published research, often utilizing our market share forecasts to establish the breadth and size of a market.
- Publicly available data, such as earnings statements, partnership announcements, product announcements and published customer cases.
- Gartner inquiry data collected by the authors and the wider analyst community within Gartner: inputs such as use cases, issues encountered, license and support pricing, and implementation plans.
- RFI surveys issued to the vendors, in which they were asked to provide specifics about versions, release dates, customer counts, distribution of customers worldwide and other data points. Vendors could refuse to provide any information in this survey, at their discretion.
- Customer reference surveys. Vendors were asked to identify a minimum number of references. Gartner augmented the vendor-provided population by adding Gartner inquiry client contacts as potential respondents. Responses were voluntary, for all participants. These surveys included questions to validate customers as current license holders. Additionally (especially in the case of open-source utilization), customers provided information that validated the size and scope of their implementations. In addition, customers were asked to provide information about issues and software bugs, overall and specific sentiments about their experience of the vendor, the use of other software tools in the environment, the types of data involved, and the rate of data refresh or load. They were also asked about their deployment plans.
- Gartner customer engagements in which we provide specific support were aggregated and anonymized to add additional perspective to the other, more expansive research approaches.
It is important to note that this is qualitative research and, as such, forms a cumulative base on which to form the opinions expressed in this Magic Quadrant.
Ability to Execute
Product/Service: Core goods and services offered by the vendor that compete in/serve the defined market. This includes current product/service capabilities, quality, feature sets, skills, etc., whether offered natively or through OEM agreements/partnerships as defined in the market definition and detailed in the subcriteria.
Overall Viability (Business Unit, Financial, Strategy, Organization): Viability includes an assessment of the overall organization's financial health, the financial and practical success of the business unit, and the likelihood of the individual business unit to continue investing in the product, to continue offering the product and to advance the state of the art within the organization's portfolio of products.
Sales Execution/Pricing: The vendor's capabilities in all pre-sales activities and the structure that supports them. This includes deal management, pricing and negotiation, pre-sales support and the overall effectiveness of the sales channel.
Market Responsiveness and Track Record: Ability to respond, change direction, be flexible and achieve competitive success as opportunities develop, competitors act, customer needs evolve and market dynamics change. This criterion also considers the vendor's history of responsiveness.
Marketing Execution: The clarity, quality, creativity and efficacy of programs designed to deliver the organization's message in order to influence the market, promote the brand and business, increase awareness of the products, and establish a positive identification with the product/brand and organization in the minds of buyers. This "mind share" can be driven by a combination of publicity, promotional, thought leadership, word-of-mouth and sales activities.
Customer Experience: Relationships, products and services/programs that enable clients to be successful with the products evaluated. Specifically, this includes the ways customers receive technical support or account support. This can also include ancillary tools, customer support programs (and the quality thereof), availability of user groups, service-level agreements, etc.
Operations: The ability of the organization to meet its goals and commitments. Factors include the quality of the organizational structure including skills, experiences, programs, systems and other vehicles that enable the organization to operate effectively and efficiently on an ongoing basis.
Completeness of Vision
Market Understanding: Ability of the vendor to understand buyers' wants and needs and to translate those into products and services. Vendors that show the highest degree of vision listen and understand buyers' wants and needs, and can shape or enhance those with their added vision.
Marketing Strategy: A clear, differentiated set of messages consistently communicated throughout the organization and externalized through the website, advertising, customer programs and positioning statements.
Sales Strategy: The strategy for selling product that uses the appropriate network of direct and indirect sales, marketing, service and communication affiliates that extend the scope and depth of market reach, skills, expertise, technologies, services and the customer base.
Offering (Product) Strategy: The vendor's approach to product development and delivery that emphasizes differentiation, functionality, methodology and feature set as they map to current and future requirements.
Business Model: The soundness and logic of the vendor's underlying business proposition.
Vertical/Industry Strategy: The vendor's strategy to direct resources, skills and offerings to meet the specific needs of individual market segments, including verticals.
Innovation: Direct, related, complementary and synergistic layouts of resources, expertise or capital for investment, consolidation, defensive or pre-emptive purposes.
Geographic Strategy: The vendor's strategy to direct resources, skills and offerings to meet the specific needs of geographies outside the "home" or native geography, either directly or through partners, channels and subsidiaries as appropriate for that geography and market.