Strong user demand has resulted in continued, steady growth in the AO services market. We believe this growth — driven by the need for skilled IT application labor — will continue. To assist prospective buyers of AO services, Gartner has created a Magic Quadrant designed to evaluate vendors based on their completeness of vision and ability to execute that vision. The AO Magic Quadrant positions six of the largest AO providers (see Figure 1 and Note 1) based on their application service offerings and delivery capabilities. At a minimum, these application services (see Note 2) include application management, but they often extend to new development, conversions, migrations and Internet development. Although several of these six vendors also offer IT infrastructure outsourcing services, all will sign stand-alone AO deals. This Magic Quadrant evaluation process only examined the vendors' AO offerings.

Figure 1

AO Magic Quadrant, North America

Source: Gartner Research

Previous research (see Research Notes M-10-1093, M-10-1958 and M-10-1978) describes the Magic Quadrant process and evaluation criteria for the Completeness of Vision and Ability to Execute axes for AO suppliers (see Note 3, Note 4 and Note 5). Although there are a number of other AO vendors that are delivering excellent AO services (see Note 6), this process only reviewed the capabilities of the largest external services providers (ESPs).

• Client Satisfaction
• Effective Resourcing
• Global Delivery and Resourcing
• Relationship Management and Flexibility
• Service Delivery
• Measuring Results

Magic Quadrant Results: Based on our review of the vendors' capabilities and our conversations with client references, all six providers were placed in the Leaders segment. This is not particularly surprising. AO is a fairly mature offering (although market growth is robust), with known, proven processes and procedures that can enable successful service delivery. Most participating vendors have spent years fine-tuning their methodologies, increasing their quality initiatives, building their delivery capabilities and supporting infrastructure and fine-tuning their vision to keep up with the changing IT landscape. Thus, most of these ESPs have significant experience and understanding of the dynamics needed for the successful delivery of AO services, and most have delivered these services for at least five years. Although CGE&Y — the newest entrant in the United States — has more-limited AO experience in North America, it has a number of core competencies that can be effectively used in delivering AO services.

Client Satisfaction: The evaluation results suggest that, despite the wide variety of complaints we have heard during the years from many (if not most) IT outsourcing customers, most AO outsourcing customers are basically satisfied with their deals. Through well-executed AO services, vendors can add value to an enterprise, as well as clearly quantify these benefits (e.g., reduced costs, improved productivity and quality, and decreased time to market). Experienced AO vendors have the experience, knowledge and infrastructure to deliver AO services more effectively than most customers can do on their own. This disparity will only increase as these vendors increasingly employ quality initiatives (e.g., CMM certification) to use more-mature processes across the application life cycle and additionally build a global delivery model that enables them to access and use the most cost-effective labor available.

Areas of Improvement: ESPs are delivering reliable AO service, as evidenced by client feedback. However, there is a risk that they will become complacent, not actively prepare for future market directions and miss major market opportunities. The evaluation process highlights several areas where vendors should focus attention (most notably along the Vision axis) and which users should consider during the evaluation process.

• Vision: Despite the expectations surrounding the application service provider (ASP) industry, most of these vendors did not have a well-defined, proactive ASP strategy. Although ASP delivery is not what most customers are demanding, we believe that the market will slowly gravitate toward ASP models and AO vendors need to have an attractive offering. (EDS, which views its ASP capabilities as an important element of its continuum of AO services, was the exception.)
•      Innovation: Although AO services are mature, continued innovation is necessary for vendors to provide global delivery capabilities and resourcing, quality initiatives, process and vertical industry expertise, and portfolio transformation. Although many of the ESPs have employed innovative technologies, tools and processes, customers (and in some cases an ESP's project executive) were unaware of these capabilities and often expressed the view that innovation was not required, thus limiting their vision of what could be accomplished by their ESP. The customer and the ESP need to creatively envision how new innovative techniques can be effectively applied against an enterprise's application portfolio (current and future).
•      Process and Vertical Expertise: Such domain expertise can further re-engineering and portfolio transformation efforts, aid in developing new markets and help vendors win deals. Although most of the six vendors had strong pockets of expertise, most enterprises were similarly unaware that this expertise existed or could add value. Most felt that such expertise, if it did exist, resided solely with the original employees transferred when the deal was signed.
•      Quality initiatives: Few customers reported ongoing CMM initiatives (see Note 7), despite our belief and statements of most vendors that CMM initiatives will be essential in the future. (Keane's customers were the exception). This was true even when most of the participating ESPs emphasized their CMM certification levels and efforts. Again, this suggests that the ESPs need to be more proactive in educating their customers about these efforts.

---

Note 7

CMM Certification

The Capability Maturity Model (CMM) was developed by the Software Engineering Institute (SEI) at Carnegie Mellon University. It is a way to assess the maturity (i.e., strength) of the processes used by an organization to develop and support its information systems. Because these processes are the tools by which an organization accomplishes its work, the higher their quality, the higher the performance of the organization.

---

Along the Execute axis, the areas of improvement that vendors will most need to address involve their resourcing efforts.

• Effective Resourcing: Most of the clients expressed some dissatisfaction with their ESPs' ability to produce the right staff, at the right time, with the right skills. (However, the clients were quite charitable to the vendors, citing the known IT labor shortages in the United States).
•      Global Sourcing: In response to the growing number of offshore vendors that employ lower-salaried employees than those in the United States and the increased competitiveness of this market, all six vendors were exploring or developing offshore labor pools to provide appropriate resources at price competitive levels. Yet, few of the customers reported any such activities on their behalf, and their vendors had not proposed this alternative staffing alternative.

Supplier Positioning: Although all six ESPs ended up in the Leaders segment, differences emerged as seen through the suppliers' respective Magic Quadrant placement. Keane, which received the most-favorable client references and feedback, has a heritage deeply rooted in application management and sees AO as its core business and competency. (On average, Keane's deals are somewhat smaller than their competitors' — although deal sizes are growing — which may tend to favorably skew customer feedback, because there are typically fewer business and IT interface requirements in smaller deals). Similarly, although Accenture has many core offerings, the company has always emphasized its application methodologies and processes, something absolutely essential to AO success. While CSC, EDS and IBM are all experienced IT outsourcing vendors (i.e., the top three global IT outsourcing vendors in terms of revenue), IBM customers felt that its methodologies did not enable its deals as much as expected. Finally, CGE&Y is still digesting the merger of Cap Gemini and Ernst & Young and needs some time to capitalize on the strengths of each partner. Furthermore, CGE&Y is the newest entrant to the U.S. AO market of these six suppliers.

Bottom Line: The AO Magic Quadrant is a useful tool for enterprises looking for assistance in evaluating prospective AO providers. The ESPs included in this Magic Quadrant have stable financial resources, established reputations and long-standing client relationships, providing the foundation to provide effective application services. However, enterprises should actively track whether these ESPs will be able to accommodate future market directions. Vendor selection should also be based on a detailed evaluation of an enterprise's unique AO objectives and technical requirements vs. an ESP's capacity to execute against those expectations.

Entire contents (C) 2001 by Gartner Group, Inc. All rights reserved. Reproduction of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner Group disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner Group shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice.